Data Contracts for AI: Why Agent Systems Break Without Them and How to Define Them

A data contract for AI is a versioned, machine-readable agreement between a data producer and an AI consumer (a model, feature pipeline, or agent) that specifies the schema with types, the meaning of every field, which agents may use the data for which purposes, maximum acceptable data lag, and the quality checks and change policy that keep those promises true.

The cost of running without contracts is measurable. A 2023 Wakefield Research survey of 200 data professionals, commissioned by Monte Carlo, found organizations averaging 67 data incidents per month, with 68 percent reporting four hours or more to detect an incident and an average of 15 hours to resolve one [1]. Chad Sanderson's 2022 example still holds: a column is dropped upstream and "the pricing algorithm that generates 80% of your company's revenue is broken" before anyone downstream is told [2]. Agents make the same failures worse because no analyst is in the loop to notice a number that looks wrong.

If you are still building the shared meaning layer these contracts depend on, the posts on the semantic layer that travels with the data and the 90-day path to an AI-ready data estate cover the prerequisites.

What Is a Data Contract in Enterprise AI?

A data contract is the interface between the team that generates a dataset and the systems that depend on it. Andrew Jones defines a data contract as "an agreed interface between the generators of data, and its consumers" that sets expectations and governs how the data is produced [3]; Sanderson calls them "API-like agreements between Software Engineers who own services and Data Consumers" [2].

The engineering contract covers schema and service levels. The Open Data Contract Standard (ODCS), a Linux Foundation AI & Data incubation project maintained by Bitol, is the YAML specification most tooling has converged on; v3.2.0 organizes a contract into fundamentals, schema, data quality, SLA, team, roles, servers, and custom properties [4][5], and the rival Data Contract Specification was deprecated in its favor in 2025 [6]. Schema properties carry logicalType, classification, and criticalDataElement [7]; slaProperties express latency, frequency, and timeToDetect with a driver of regulatory, analytics, or operational [8].

An AI consumer needs three more dimensions, because it cannot ask a colleague what a column means and has no way to know that yesterday's snapshot is too old for today's decision.

Why Do AI Agents Break Without Data Contracts?

An agent is the most demanding data consumer in your estate because it acts on what it reads with no judgment step in between. An analyst who sees tenure_days drop from an average of 900 to 0 stops and asks; a feature pipeline, the code that turns raw records into model inputs, backfills zeros, and the churn model predicts that every long-standing customer is brand new.

Three failure classes recur. Schema drift: a source team renames cust_id to customer_id, the join returns zero rows, and the agent answers from its prior. The NULL surprise: a field that was 99.8 percent populated drops to 60 percent after an upstream form change, and a tree-based model routes 40 percent of traffic down a branch it never trained on. Semantic ambiguity: revenue in the finance mart is recognized revenue in USD, revenue in the sales mart is booked ARR in local currency, and an agent joining the two is wrong by construction.

Freshness is a fourth class that appears at inference time. Jumio's real-time feature store on AWS, described in an August 2026 engineering post, requires that "features must be served in under 100 milliseconds" and reaches a 95th-percentile response of 16.9 milliseconds [9]; a fast read of a value three hours stale is still a wrong answer. Tecton's expected_feature_freshness marks data stale when now - most_recent_feature_value_timestamp exceeds the threshold, and monitor_freshness raises the alert [10]. The sibling guide on what multi-agent systems need from your data layer covers the wider requirements; the contract makes them checkable.

What Should a Data Contract for an AI System Specify?

A contract an agent can be held to has to exist as a file that tooling can read. The example below is a complete contract for a customer churn prediction feature set, written in the ODCS v3.2.0 shape so datacontract-compatible tooling can lint and test it, with the three AI dimensions carried in glossary links, a permissibleUse block under customProperties, and explicit slaProperties [4][8].

apiVersion: v3.2.0
kind: DataContract
id: 7f3c2a10-9b4e-4d61-8f2a-churn-features-v2
name: customer_churn_features
version: 2.1.0
status: active
domain: customer_success
dataProduct: churn_prediction
tenant: retail_bank_na

description:
  purpose: Model-ready features for 30-day customer churn prediction.
  usage: Read-only input to the churn scoring model and the retention agent.
  limitations: Not a source of record for balances; do not use for credit decisions.

team:
  - username: owner.churn-features@example.com
    role: owner
  - username: oncall.customer-success-data@example.com
    role: steward

schema:
  - name: churn_features
    logicalType: object
    physicalType: table
    physicalName: gold.customer_churn_features
    properties:
      - name: customer_id
        logicalType: string
        physicalType: varchar(36)
        businessName: Customer Identifier
        description: Golden-record ID from the MDM hub. Never the CRM account number.
        required: true
        unique: true
        primaryKey: true
        classification: confidential
        criticalDataElement: true
        authoritativeDefinitions:
          - url: https://catalog.example.com/glossary/customer_id
            type: businessDefinition
      - name: feature_ts
        logicalType: timestamp
        businessName: Feature Snapshot Time
        description: UTC time the feature row was computed. Freshness SLA is measured on this column.
        required: true
        criticalDataElement: true
      - name: tenure_days
        logicalType: integer
        businessName: Customer Tenure
        description: Whole days since first funded account. NULL means unknown, never zero.
        required: false
        criticalDataElement: true
      - name: txn_count_30d
        logicalType: integer
        businessName: Transactions, Trailing 30 Days
        description: Count of posted (not pending) transactions in the 30 days before feature_ts.
        required: true
      - name: support_tickets_90d
        logicalType: integer
        businessName: Support Tickets, Trailing 90 Days
        description: Closed and open tickets, all channels, deduplicated by case ID.
        required: true
      - name: churn_flag
        logicalType: boolean
        businessName: Churned in Next 30 Days (label)
        description: True if all accounts closed within 30 days after feature_ts. Training label only.
        required: false
        classification: restricted
        authoritativeDefinitions:
          - url: https://catalog.example.com/glossary/churn_definition_v3
            type: businessDefinition

customProperties:
  - property: permissibleUse
    value:
      - agent: churn-scoring-model
        purpose: batch_and_online_inference
        fields: [customer_id, feature_ts, tenure_days, txn_count_30d, support_tickets_90d]
      - agent: retention-outreach-agent
        purpose: recommend_next_action
        fields: [customer_id, tenure_days, txn_count_30d]
        forbidden: [churn_flag]
      - agent: model-training-pipeline
        purpose: supervised_training
        fields: all
  - property: prohibitedPurposes
    value: [credit_decisioning, marketing_to_minors, third_party_sharing]
  - property: policyBinding
    value: ranger-tag:churn_features_v2

slaProperties:
  - property: latency
    value: 15
    unit: m
    element: churn_features.feature_ts
    driver: operational
    description: Newest feature_ts must be within 15 minutes of read time for online inference.
  - property: frequency
    value: 5
    unit: m
    element: churn_features.feature_ts
  - property: availability
    value: 99.9
    unit: percent
  - property: timeToDetect
    value: 10
    unit: m
  - property: timeToNotify
    value: 15
    unit: m
  - property: retention
    value: 2
    unit: y

quality:
  - type: library
    metric: nullValues
    element: churn_features.customer_id
    mustBe: 0
  - type: library
    metric: duplicateValues
    element: churn_features.customer_id
    mustBe: 0
  - type: library
    metric: nullValues
    element: churn_features.txn_count_30d
    mustBeLessThan: 0.5
    unit: percent
  - type: sql
    query: SELECT COUNT(*) FROM gold.customer_churn_features WHERE tenure_days = 0
    mustBeLessThan: 100
    description: Guards against the zero-backfill failure mode.
  - type: custom
    engine: greatExpectations
    implementation: expect_column_values_to_be_between(column="txn_count_30d", min_value=0, max_value=5000)

changePolicy:
  breaking:
    definition: Column removal, rename, type narrowing, or a change to any authoritativeDefinitions URL.
    noticePeriod: 30d
    requires: major version bump and written acknowledgement from every agent in permissibleUse
  nonBreaking:
    definition: Column addition, type widening, description edits.
    noticePeriod: 0d
    requires: minor version bump and changelog entry
  onViolation

apiVersion: v3.2.0
kind: DataContract
id: 7f3c2a10-9b4e-4d61-8f2a-churn-features-v2
name: customer_churn_features
version: 2.1.0
status: active
domain: customer_success
dataProduct: churn_prediction
tenant: retail_bank_na

description:
  purpose: Model-ready features for 30-day customer churn prediction.
  usage: Read-only input to the churn scoring model and the retention agent.
  limitations: Not a source of record for balances; do not use for credit decisions.

team:
  - username: owner.churn-features@example.com
    role: owner
  - username: oncall.customer-success-data@example.com
    role: steward

schema:
  - name: churn_features
    logicalType: object
    physicalType: table
    physicalName: gold.customer_churn_features
    properties:
      - name: customer_id
        logicalType: string
        physicalType: varchar(36)
        businessName: Customer Identifier
        description: Golden-record ID from the MDM hub. Never the CRM account number.
        required: true
        unique: true
        primaryKey: true
        classification: confidential
        criticalDataElement: true
        authoritativeDefinitions:
          - url: https://catalog.example.com/glossary/customer_id
            type: businessDefinition
      - name: feature_ts
        logicalType: timestamp
        businessName: Feature Snapshot Time
        description: UTC time the feature row was computed. Freshness SLA is measured on this column.
        required: true
        criticalDataElement: true
      - name: tenure_days
        logicalType: integer
        businessName: Customer Tenure
        description: Whole days since first funded account. NULL means unknown, never zero.
        required: false
        criticalDataElement: true
      - name: txn_count_30d
        logicalType: integer
        businessName: Transactions, Trailing 30 Days
        description: Count of posted (not pending) transactions in the 30 days before feature_ts.
        required: true
      - name: support_tickets_90d
        logicalType: integer
        businessName: Support Tickets, Trailing 90 Days
        description: Closed and open tickets, all channels, deduplicated by case ID.
        required: true
      - name: churn_flag
        logicalType: boolean
        businessName: Churned in Next 30 Days (label)
        description: True if all accounts closed within 30 days after feature_ts. Training label only.
        required: false
        classification: restricted
        authoritativeDefinitions:
          - url: https://catalog.example.com/glossary/churn_definition_v3
            type: businessDefinition

customProperties:
  - property: permissibleUse
    value:
      - agent: churn-scoring-model
        purpose: batch_and_online_inference
        fields: [customer_id, feature_ts, tenure_days, txn_count_30d, support_tickets_90d]
      - agent: retention-outreach-agent
        purpose: recommend_next_action
        fields: [customer_id, tenure_days, txn_count_30d]
        forbidden: [churn_flag]
      - agent: model-training-pipeline
        purpose: supervised_training
        fields: all
  - property: prohibitedPurposes
    value: [credit_decisioning, marketing_to_minors, third_party_sharing]
  - property: policyBinding
    value: ranger-tag:churn_features_v2

slaProperties:
  - property: latency
    value: 15
    unit: m
    element: churn_features.feature_ts
    driver: operational
    description: Newest feature_ts must be within 15 minutes of read time for online inference.
  - property: frequency
    value: 5
    unit: m
    element: churn_features.feature_ts
  - property: availability
    value: 99.9
    unit: percent
  - property: timeToDetect
    value: 10
    unit: m
  - property: timeToNotify
    value: 15
    unit: m
  - property: retention
    value: 2
    unit: y

quality:
  - type: library
    metric: nullValues
    element: churn_features.customer_id
    mustBe: 0
  - type: library
    metric: duplicateValues
    element: churn_features.customer_id
    mustBe: 0
  - type: library
    metric: nullValues
    element: churn_features.txn_count_30d
    mustBeLessThan: 0.5
    unit: percent
  - type: sql
    query: SELECT COUNT(*) FROM gold.customer_churn_features WHERE tenure_days = 0
    mustBeLessThan: 100
    description: Guards against the zero-backfill failure mode.
  - type: custom
    engine: greatExpectations
    implementation: expect_column_values_to_be_between(column="txn_count_30d", min_value=0, max_value=5000)

changePolicy:
  breaking:
    definition: Column removal, rename, type narrowing, or a change to any authoritativeDefinitions URL.
    noticePeriod: 30d
    requires: major version bump and written acknowledgement from every agent in permissibleUse
  nonBreaking:
    definition: Column addition, type widening, description edits.
    noticePeriod: 0d
    requires: minor version bump and changelog entry
  onViolation

The retention-outreach-agent is forbidden from reading churn_flag because a label that leaks into an action agent turns a prediction into a self-fulfilling instruction, and the latency SLA is stated against feature_ts, a column the consumer can read, so the agent verifies freshness itself.

How Do You Operationalize Data Contracts for AI Pipelines?

A contract is only as real as the system that rejects a violation, so the operational question is where the file lives, what checks it, and who hears about a breach.

This is what Zhamak Dehghani's data mesh principles call "federated computational governance": domains own their contracts and the platform automates the checks [16]. The sibling guides on policy-as-code for AI data access and data lineage for AI models cover the permissible-use row and the record of which contract version fed which model.

Why Not Just Use API Contracts and Schema Registries?

API contracts and schema registries fill the schema row and leave the other five empty. An Avro schema in a registry rejects a payload with the wrong type, but it says nothing about whether the value is stale or whether the calling agent may use the field for this purpose, and it cannot tell you what status means when two services define it differently.

There is also a structural mismatch. An API contract governs one request-response boundary owned by one service team, while AI consumers read from tables and feature stores assembled downstream of several sources, none of which owns the joined result. GDPR Article 5(1)(b) requires that personal data be "collected for specified, explicit and legitimate purposes and not further processed in a manner that is incompatible with those purposes" [17], and an agent that assembles a profile from three individually permitted APIs can still violate purpose limitation on the composite. The contract has to attach to the data product the agent reads. Schema registries remain the enforcement point for the schema row of that larger contract.

NexusOne Makes the Semantic Layer the Contract Enforcement Surface

NexusOne checks contracts in the catalog and policy layer because that is the one layer every engine and every agent already passes through. Datasets are cataloged in DataHub and Apache Gravitino with glossary-bound definitions, so the semantic-intent row of a contract resolves to a term the agent can read at query time. Tagging the dataset in DataHub and mapping the tag to a role generates Apache Ranger policies at the bucket, schema, and table level, and one Ranger policy is enforced identically across Trino, Apache Spark, Apache Kyuubi, and S3 object storage per object. The permissibleUse block therefore compiles to policies the engines enforce, with agents governed by the same Keycloak identity model as humans.

Agent and MCP requests enter through the AI & Data Control Plane, which classifies each request, applies per-role budgets, redacts PII before frontier-model egress, and logs every request with its policy outcome, so a contract violation sits in the same record as the query that caused it. The layer runs identically on-prem, in your cloud, hybrid, or air-gapped, on open formats like Apache Iceberg. To work through a contract for one of your own feature sets, book an expert consultation.

Key Takeaways

• A schema-plus-SLA contract protects analysts; an AI consumer also needs semantic intent, permissible use, and a freshness guarantee in the same file.

• Organizations averaged 67 data incidents a month and 15 hours to resolve each in the 2023 Monte Carlo survey, and agents act on bad data faster than humans detect it.

• Write contracts in the Open Data Contract Standard shape so CI can lint them and quality engines can execute them.

• State freshness against a timestamp column the consumer can read, so the agent verifies data age itself.

• Map the permissible-use block to policy-engine roles so the contract is enforced at query time.

• Breaking changes require a major version bump and recorded consumer acknowledgement before promotion.

FAQ

What Are Data Contracts for AI Systems?

Data contracts for AI systems are versioned, machine-readable agreements between a data producer and an AI consumer such as a model, feature pipeline, or agent. They extend schema plus SLA with the meaning of every field, which agents may use which fields for which purposes, and the maximum acceptable age of the data at read time. ODCS v3.2.0 supplies the YAML shape; Great Expectations and Soda execute the checks.

What Is a Data Contract in Enterprise AI?

In enterprise AI, a data contract is the interface between the team that produces a dataset and the models and agents that consume it. Andrew Jones defines it as an agreed interface between generators of data and its consumers. In practice it is a YAML file specifying schema, quality checks, service levels, ownership, permissible use, and change policy, kept in version control and enforced in CI and at query time.

How Do Data Contracts Improve AI Agent Reliability?

Data contracts turn unstated assumptions into checks that fail before an agent acts. Schema checks in CI stop a renamed column from producing an empty join, quality checks quarantine a batch whose null rate jumped, and freshness SLAs on a readable timestamp let the agent refuse stale inputs. Permissible-use rules enforced by the policy engine keep an action agent from reading a training label.

How Do API Contracts Differ From Data Contracts in AI Data Pipelines?

An API contract governs one request-response boundary owned by a single service and validates payload shape and type. A data contract governs a data product that may be assembled from many sources and adds semantic definitions, purpose restrictions, freshness guarantees, quality thresholds, and change policy. AI pipelines need both: the registry enforces the schema row, and the data contract carries the rows the registry has no field for.

References

1. Data Downtime Nearly Doubled Year Over Year, Monte Carlo Survey Says. Business Wire, May 2, 2023. https://www.businesswire.com/news/home/20230502005377/en/Data-Downtime-Nearly-Doubled-Year-Over-Year-Monte-Carlo-Survey-Says

2. Chad Sanderson, The Rise of Data Contracts. Data Products Substack, August 22, 2022. https://dataproducts.substack.com/p/the-rise-of-data-contracts

3. Andrew Jones, Driving Data Quality with Data Contracts, Chapter 2: Introducing Data Contracts. Packt via O'Reilly, 2023. https://www.oreilly.com/library/view/driving-data-quality/9781837635009/B19790_02.xhtml

4. Open Data Contract Standard (ODCS) v3.2.0. Bitol. https://bitol-io.github.io/open-data-contract-standard/latest/

5. Bitol, a Linux Foundation AI & Data incubation project. https://bitol.io/

6. Data Contract Specification (deprecation notice in favor of ODCS). GitHub, datacontract/datacontract-specification. https://github.com/datacontract/datacontract-specification

7. ODCS Schema section. Bitol. https://bitol-io.github.io/open-data-contract-standard/latest/schema/

8. ODCS Service-Level Agreement section. Bitol. https://bitol-io.github.io/open-data-contract-standard/latest/service-level-agreement/

9. How Jumio Built a Real-Time Feature Store on AWS. AWS Machine Learning Blog, August 18, 2026. https://aws.amazon.com/blogs/machine-learning/how-jumio-built-a-real-time-feature-store-on-aws/

10. StreamFeatureView SDK Reference (expected_feature_freshness, monitor_freshness). Tecton Documentation. https://docs.tecton.ai/docs/sdk-reference/feature-views/StreamFeatureView

11. Evolution (Schema Evolution). Apache Iceberg Documentation. https://iceberg.apache.org/docs/latest/evolution/

12. Business Glossary. DataHub Documentation. https://docs.datahub.com/docs/glossary/business-glossary

13. Tag Based Policies. Apache Ranger Wiki. https://cwiki.apache.org/confluence/display/RANGER/Tag+Based+Policies

14. Freshness Checks. Soda Documentation. https://docs.soda.io/soda-documentation/soda-v3/sodacl-reference/freshness

15. GX Core Overview (Expectations, Checkpoints, Actions). Great Expectations Documentation. https://docs.greatexpectations.io/docs/core/introduction/gx_overview

16. Zhamak Dehghani, Data Mesh Principles and Logical Architecture. martinfowler.com, December 3, 2020. https://martinfowler.com/articles/data-mesh-principles.html

17. Article 5 GDPR, Principles relating to processing of personal data. gdpr-info.eu. https://gdpr-info.eu/art-5-gdpr/