Lock-in is decided at the storage format, compute API, and catalog layers, years before the renewal meeting. A layer-by-layer risk table, an own/federate/never-buy-locked framework, and a checklist for the next cycle.
By
Billy Allocca
Table of Contents
The CDO's Guide to a Vendor-Neutral Data Estate: What to Own, What to Federate, What to Never Buy Locked
A vendor-neutral data estate stores tables in an open format (Apache Iceberg on Parquet) in storage you control, exposes them through an open catalog API, and queries them with standard SQL from any engine. Proprietary services sit above that foundation as removable accelerators. Lock-in is an architecture outcome, decided at the foundation layer, and renegotiation cannot undo it.
The regulatory floor moved in 2025. Regulation (EU) 2023/2854, the Data Act, applied from 12 September 2025 and requires cloud providers to start switching after at most two months' notice and complete it within a 30-day transitional window [1][3]. Article 29 caps switching charges at direct cost today and abolishes them from 12 January 2027 [1][2]. Google, AWS, and Microsoft each announced free egress for departing customers between January and March 2024, and Microsoft's terms require you to cancel every Azure subscription and finish within 60 days [4].
Cheaper egress removes one exit cost and leaves the larger one untouched. Flexera's 2026 survey of more than 750 cloud decision-makers found 76 percent of large enterprises spend over $5 million a month on cloud [5], and in a November 2025 Parallels survey of 540 IT professionals, 94 percent said they were concerned about vendor lock-in [6]. The bytes can now leave for free while the format, the compute API, and the catalog schema stay behind.
The post on the real cost of cloud-era platforms covers what those platforms charge, and Why Composability Is Required at Every Layer covers the design principle; this guide covers how to structure the estate so the next renewal is a choice.
How Do You Avoid Data Platform Vendor Lock-In?
You avoid it by deciding, layer by layer, where a proprietary dependency is allowed to exist. Vendor lock-in is the condition in which the cost of moving a workload to another provider exceeds the benefit of doing so, at any price. That cost is set by three foundation decisions: the format the data is written in, the API the compute is called through, and the schema the catalog uses to describe tables and grants. Everything above those three is cheaper to replace, provided the foundation stays open.
Vendor documentation shows the penalty attached to the open path. A Delta Lake table can be read by Iceberg clients through UniForm, but "Iceberg client support is read-only. Writes are not supported" [9]. A Snowflake table on an external Iceberg catalog "provides limited Snowflake platform support," and Fail-safe is unavailable on customer-managed volumes [8].
Stack layer | Lock-in mechanism | Cost to undo | Open choice |
|---|---|---|---|
Storage format | Vendor-internal table format in vendor-managed storage | Rewrite of every table with dual-running; highest in the stack | Apache Iceberg on Parquet [18] in your buckets, readable by Spark, Trino, Flink, and Hive concurrently [15] |
Compute API | Dialect extensions, proprietary UDF runtimes, notebook APIs | Rewrite and re-test of every pipeline that used an extension | ANSI SQL (ISO/IEC 9075) [19], Apache Spark APIs, Apache Arrow for in-memory exchange [17] |
Catalog schema | Table pointers, grants, and lineage in a vendor catalog with no export | Re-creation of every pointer and grant; loss of history | Iceberg REST catalog specification, one client for any compliant server [16] |
Semantic layer | Metric definitions inside one BI or warehouse tool | Re-authoring metrics; definition drift during cutover | Vendor-neutral specs such as Open Semantic Interchange, announced September 2025 [20] |
Orchestration | DAGs written against a scheduler's proprietary operators | Operator rewrite; moderate, mostly testing | Open schedulers running plain SQL or Python against open APIs |
BI and serving | Extracts bound to one tool's storage engine | Dashboard rebuild; low if the semantic layer is open | Tools that query the semantic layer live over standard SQL or Arrow |
What Should a CDO Own, Federate, and Never Buy Locked?
Own the layers that hold the asset, federate the layers that read it, and refuse any purchase that puts the asset inside the vendor's format. Sovereignty is a control property: you own the data, you control the tooling, and you can run the same stack in your cloud, your data center, or an air-gapped site. Openness and composability are prerequisites, because a closed format cannot be moved and a monolith cannot be partially replaced.
Streaming already runs this pattern. BYOC (bring your own cloud) is a deployment model in which the vendor operates the control plane and the data plane runs in your account. WarpStream, acquired by Confluent in September 2024, runs its agents on your VMs and writes to your object storage, and its control plane "only hosts the cloud control plane, so all it ingests is metadata" [13][14]. Databricks classic compute runs in your AWS account on the same principle, while serverless compute runs inside the Databricks account [11]. A data plane in your account is one you can point a different engine at.
Decision | Own | Federate | Never buy locked |
|---|---|---|---|
Object storage | Your buckets, account, and keys | ||
Table format | Iceberg on Parquet, written by any engine | A format only one vendor can write | |
Catalog | Open catalog you can back up and re-host | A catalog with no REST export of tables and grants | |
Identity and policy | One identity provider, one policy engine, every engine | Per-tool permission models with no export | |
Compute engines | Trino, Spark, warehouse engines as interchangeable readers | An engine whose write path produces a closed format | |
Semantic layer | Definitions in an open, versioned spec | Metrics that exist only inside one BI tool | |
Orchestration | Any scheduler whose tasks call open APIs | Operators with no equivalent elsewhere | |
BI and applications | Several tools reading one semantic layer | Tools that require extracts into their own engine | |
Managed acceleration | Rented above the foundation, with a tested off-ramp | Acceleration that requires converting tables first |
Why Is Lift-and-Shift to Multi-Cloud Not a Vendor Independence Strategy?
Moving a proprietary platform to a second cloud copies the dependency. If the tables are in a vendor-internal format and the grants live in a vendor catalog, running it in AWS and Azure gives you two regions of the same lock plus cross-cloud egress. Data portability is the ability to move data to another provider and use it without conversion; a copy in a format only the originating vendor can write is portable in bytes and locked in practice.
The 37signals exit shows what a portable storage layer is worth. The company cut its cloud bill from $3.2 million to $1.3 million a year after moving compute to about $700,000 of owned hardware, projected savings above $10 million over five years, and in October 2024 still held almost 10 petabytes in S3 costing "well over a million dollars" a year, with 18 petabytes of replacement storage priced at roughly one year of that spend [7]. The exit was possible because object storage is an open interface: the files could be copied to any S3-compatible target and read by the same applications. A warehouse table in a closed format cannot be copied that way, which is why format and catalog decisions dominate exit cost.
The guide on zero lock-in compute covers separating storage, compute, and serving without an integration burden, and the guide on Apache Iceberg for enterprise AI covers what to build around the format.
Why Not Just Negotiate a Better Contract at the Next Renewal?
Negotiation recovers price and leaves the dependency in place, so the following renewal starts from the same position. Procurement extracts a discount when it has a credible alternative; if the estate's tables can only be read by the incumbent, there is none, and both sides know it. Microsoft's egress waiver arrives only after every subscription is cancelled and the transfer completes within 60 days [4], a window a multi-petabyte estate in a closed format cannot meet without a rewrite.
Open formats carry a real cost too: Snowflake lists features unavailable on externally cataloged Iceberg tables, and Databricks generates Iceberg metadata read-only and asynchronously [8][9]. The trade is worth making because a managed convenience is rentable later and the format decision is close to irreversible.
What Should Be on the Checklist for the Next Renewal Cycle?
Preparation starts twelve months before the renewal, because most items below take a quarter or more.
Inventory every table by format and location: open format in your bucket, open format in vendor storage, or vendor-internal format.
Measure the share of production reads and writes that use vendor-specific SQL extensions or runtimes, and estimate rewrite hours.
Export the catalog (pointers, schemas, grants, lineage) through an open API, re-host it, and test the restore.
Run one production workload end to end on a second engine (Trino or Spark against the same Iceberg tables) and record the gaps.
Convert one high-value dataset to Iceberg in your own storage and re-point its consumers, so the off-ramp is a repeatable procedure.
Put semantic definitions in a versioned, tool-independent spec and confirm two BI tools read them.
Federate identity to one provider and enforce policy from one engine across every compute path and the object store.
Time the exit under the Data Act terms: two-month notice, 30-day transition, plus your provider's waiver conditions [3][4].
Write the switching cost as a number and bring it to the negotiation as the alternative.
NexusOne Preserves Optionality at the Middle of the Stack
NexusOne is the layer between platforms, and it keeps the foundation open by construction. Tables are stored as Apache Iceberg in object storage you control; Trino, Apache Spark, and Apache Kyuubi are consumers of those tables, so any engine can be added or retired without touching the data. Identity is federated from Active Directory, Okta, LDAP, or SAML into one Keycloak layer, and a policy defined once in Apache Ranger applies to Trino, Spark, Kyuubi, and S3 object storage per object at the same time. Tagging a dataset in DataHub and mapping a role to that tag generates the Ranger policies wherever the data can be reached, so grants live in a model you can export.
Federated query reaches every source in the estate, including cloud-era platforms, which become repositories inside it. The AI & Data Control Plane governs agent and MCP requests at the boundary with per-role token budgets, PII redaction before egress, and a log of every request. The same software runs identically on-prem, in your cloud, hybrid, or air-gapped, and if you leave, you take your data in open formats. To work through your own lock-in risk table with an architect, request an expert consultation.
Key Takeaways
Lock-in is decided at three foundation layers (storage format, compute API, catalog schema), and the renewal meeting cannot reverse it.
The EU Data Act abolishes switching charges from 12 January 2027, which lowers the transfer bill and leaves format and catalog conversion costs untouched.
Own storage, format, catalog, identity, and semantic definitions; federate compute, orchestration, and BI; refuse any purchase that converts tables into a single-vendor format.
Lift-and-shift of a proprietary platform to a second cloud duplicates the dependency and adds cross-cloud egress.
Vendor Iceberg support is real and carries functional penalties, so test the off-ramp on a production dataset and bring the measured switching cost to the negotiation.
FAQ
What Does a Vendor-Neutral Data Architecture Look Like for a CDO?
You own the object storage, the table format (Apache Iceberg on Parquet), the catalog (exposed through the Iceberg REST specification), identity and policy, and the semantic definitions. Query engines, orchestration, and BI are federated consumers that can be swapped without rewriting data, and proprietary managed services run above that foundation with a tested off-ramp.
How Do You Avoid Data Platform Vendor Lock-In?
Keep the storage format, compute API, and catalog schema open, because those three layers set the exit cost. Write every table as Iceberg in a bucket you control, keep pipelines on ANSI SQL and open Spark APIs, hold grants in a catalog you can export, and test the exit on one production dataset each year.
Is Open Source Data Infrastructure Realistic for the Enterprise?
Yes, and the major vendors have conceded the format layer: Snowflake supports Iceberg tables in customer storage [8], and Databricks exposes Unity Catalog managed Iceberg tables over the Iceberg REST catalog with read, write, and create access [10]. The remaining gap is operational (identity, policy, and upgrades across many engines), and that is where a governed layer over the open tools belongs.
What Is a Multi-Cloud Data Strategy That Delivers Vendor Independence?
One in which the same open tables, catalog, identity, and policy run in every cloud, and compute in each cloud reads those tables. Running a proprietary platform in two clouds is redundancy with added egress; independence comes from the format and catalog being open everywhere, so any cloud or your own data center can host the next engine.
What Does BYOC Change for Vendor Neutrality?
BYOC keeps the data plane in your account and moves only the control plane to the vendor, so the storage is yours at exit. WarpStream and Redpanda run this model for streaming [12][13], and Databricks classic compute follows the same principle [11]. A proprietary table format written into that storage stays proprietary, so pair BYOC with an open format.
References
Regulation (EU) 2023/2854 (Data Act), EUR-Lex. https://eur-lex.europa.eu/eli/reg/2023/2854/oj/eng
Data Act explained, European Commission, Shaping Europe's digital future. https://digital-strategy.ec.europa.eu/en/factpages/data-act-explained
Cloud Switching Under the EU Data Act: Implications for IaaS, PaaS, and SaaS Providers, Greenberg Traurig, September 2025. https://www.gtlaw.com/en/insights/2025/9/cloud-switching-under-the-eu-data-act
Microsoft joins AWS, Google Cloud on path to free data egress, CIO Dive, March 14, 2024. https://www.ciodive.com/news/microsoft-azure-eliminates-cloud-data-egress-fees/710367/
Flexera Finds Cloud Value is Rising While AI Waste Grows (2026 State of the Cloud Report), Flexera, March 18, 2026. https://www.flexera.com/about-us/press-center/flexera-finds-cloud-value-is-rising-while-ai-waste-grows
94% of IT Leaders Fear Vendor Lock-In as AI Reality Check Forces EUC Strategy Reset, Parallels Survey Finds, Parallels, February 2026. https://www.parallels.com/newsroom/news/press-releases/20260217-cloud-survey/
Our cloud-exit savings will now top ten million over five years, David Heinemeier Hansson, HEY World, October 17, 2024. https://world.hey.com/dhh/our-cloud-exit-savings-will-now-top-ten-million-over-five-years-c7d9b5bd
Apache Iceberg tables, Snowflake Documentation. https://docs.snowflake.com/en/user-guide/tables-iceberg
Read Delta Lake tables with Iceberg clients (UniForm), Databricks Documentation. https://docs.databricks.com/aws/en/delta/uniform
Unity Catalog managed tables for Delta Lake and Apache Iceberg, Databricks Documentation. https://docs.databricks.com/aws/en/tables/managed
Databricks architecture overview (control plane and compute plane), Databricks Documentation. https://docs.databricks.com/aws/en/getting-started/overview
Redpanda BYOC, Redpanda. https://www.redpanda.com/data-streaming/bring-your-own-cloud-byoc
BYOC Kafka Streaming, WarpStream. https://www.warpstream.com/bring-your-own-cloud-kafka-data-streaming
Confluent Acquires WarpStream to Advance Next-Gen BYOC Data Streaming, Confluent, September 9, 2024. https://www.confluent.io/press-release/confluent-acquires-warpstream-to-advance-next-gen-byoc-data-streaming/
Apache Iceberg, The Apache Software Foundation. https://iceberg.apache.org/
Iceberg Catalogs, Apache Iceberg Documentation. https://iceberg.apache.org/concepts/catalog/
Apache Arrow Overview, The Apache Software Foundation. https://arrow.apache.org/overview/
Apache Parquet, The Apache Software Foundation. https://parquet.apache.org/
ISO/IEC 9075-1:2023, Information technology, Database languages SQL, Part 1: Framework, ISO. https://www.iso.org/standard/76583.html
Snowflake, Salesforce, dbt Labs, and More, Revolutionize Data Readiness for AI with Open Semantic Interchange Initiative, Snowflake, September 23, 2025. https://www.snowflake.com/en/news/press-releases/snowflake-salesforce-dbt-labs-and-more-revolutionize-data-readiness-for-ai-with-open-semantic-interchange-initiative/
