AI agents query without intent, scope, schema awareness, or a ceiling on fan-out. The data layer that survives them treats each agent as a named principal behind a governed query abstraction with row filters, column masks, cardinality caps, timeouts, and full request logs.
By
Billy Allocca
Table of Contents
What Multi-Agent Systems Need From Your Data Layer (and Why Your Current Stack Does Not Provide It)
A data layer for AI agents is a governed query abstraction between agent frameworks (MCP servers, tool calls, SQL agents) and every underlying store. It authenticates each agent as its own principal, applies row-level security and column masking under the policy model that governs humans, caps result cardinality, enforces timeouts and rate limits, and logs every request.
Gartner projected in August 2025 that 40 percent of enterprise applications will embed task-specific AI agents by the end of 2026, up from under 5 percent in 2025 [1]. Most of them will ask for data. Anthropic's production figures show what that means at the query layer: single agents use about 4 times the tokens of a chat interaction, multi-agent systems about 15 times, and early versions of its research agent spawned 50 subagents for simple queries [2]. In July 2025 a Replit coding agent deleted a production database during an explicit code freeze [3].
The routing and spend side of the problem is covered in our posts on enterprise AI governance through one control plane and AI token spend governance.
Why Do AI Agents Break Data Architectures Designed for Humans?
Agents break warehouse and lake architectures because every implicit safety property of a human analyst is absent. An AI agent is a program that uses a language model to choose actions (queries, API calls, tool invocations) in a loop until it decides a goal is met. A multi-agent system is a set of such agents that delegate subtasks to each other, so one prompt can produce dozens of data requests. OWASP calls the resulting risk Excessive Agency, "the vulnerability that enables damaging actions to be performed in response to unexpected, ambiguous or manipulated outputs from an LLM" [4]. On a data layer it takes four forms.
How agents differ from human users | What goes wrong on a human-era stack | The control that answers it |
|---|---|---|
No intent. The goal is inferred from a prompt, and a manipulated prompt changes it. | A shared credential pulls the customer table because the prompt asked for "everything relevant." | Agent identity as a first-class principal with its own permissions. |
No self-limited scope. The agent has no sense of "too many rows." | A | Cardinality caps, scan-byte limits, and timeouts enforced by the query layer. |
No schema-boundary awareness. Every catalog table is a table the agent will read. | PII columns reach the model through a join nobody masked. | Row filters and column masks that follow the dataset tag onto every engine. |
Unbounded fan-out. One task becomes N subagents, each retrying on error. | Concurrency spikes block human dashboards; the bill arrives a day later. | Per-principal rate and concurrency limits, plus per-request logging. |
InfoQ's July 2026 report on agent billing failures gives the last row a number: an engineer's agent decided a job required "20 Gbps with redundancy and fail-over capacity," provisioned five 48-vCPU instances, and ran up $6,531.30 before anyone saw it, because billing alerts lag roughly a day behind spend [5].
What Data Infrastructure Do AI Agents Need?
Agents need one query abstraction that owns identity, policy, limits, and logging, so no framework or MCP server reimplements them. A governed query abstraction layer is a single entry point that resolves the calling principal, rewrites or rejects the query according to policy, executes it on the right engine, and records the outcome.
Property | Raw warehouse access | Governed query abstraction layer |
|---|---|---|
Who the store sees | A shared service account | The agent's own principal from the identity provider |
Where policy lives | In each warehouse, lake, and API separately | Defined once, enforced on every engine and object store |
Row and column protection | Per-warehouse features, if licensed and configured | Filters and masks derived from catalog tags, on every path |
Result size and runtime | Whatever the query returns; Trino's default timeout is 100 days [6] | Cardinality cap, scan-byte limit, and timeout per role |
Concurrency | Whatever the connection pool allows | Rate and concurrency limits per agent principal |
Audit | Query history keyed to the shared account | Every request logged with agent, user, tool, and decision |
The Trino default is the point: a mature engine ships with query.max-execution-time at 100 days and no scan-byte limit until you set one [6]. Human users never triggered those defaults. Agents will.
What Does Principal-Aware Data Access Mean for Agentic AI?
Principal-aware access means the data layer knows which agent, acting for which user, made each request, and enforces and audits on that basis. A principal is the identity a request is authorized as, and for agents it must be the agent itself, issued by the system that issues human logins. Keycloak gives every registered client "a built-in service account which allows it to obtain an access token," so an agent can hold roles like a person [7], and OWASP asks that actions taken for a user execute downstream "in the context of that specific user, and with the minimum privileges necessary" [4].
MCP (Model Context Protocol), the open JSON-RPC protocol through which agent hosts discover and call tools, covers half of this. Its authorization specification (revision 2026-07-28) requires OAuth 2.1, requires servers to validate that tokens "were issued specifically for them as the intended audience," forbids token passthrough to upstream systems, and asks clients for minimum scopes [8]. It also states that MCP cannot enforce security principles at the protocol level [9]. Rows, columns, row counts, and request rates remain data-layer decisions, the subject of the companion guide on the MCP trust problem.
A principal-aware data layer meets every item on this list:
Each agent authenticates as its own principal from the enterprise identity provider; no shared credentials.
Row-level security (a filter predicate appended to every query so a principal sees only permitted rows) is evaluated at runtime for the agent principal, as Snowflake row access policies and Apache Ranger row-filter policies do [10][11].
Column masking (rewriting sensitive values to redacted, hashed, or null output for principals without clearance) applies on every join path and follows a catalog tag [11].
A cardinality cap (a hard upper bound on rows returned per request) is enforced by the query layer. LangChain's SQL agent tutorial sets a
top_kof 5 and forbids DML inside the system prompt, while warning that connection permissions must be "scoped as narrowly as possible" [12]; a prompt is advice, a cap is enforcement.Timeouts and scan-byte limits are set per role in seconds and gigabytes, overriding engine defaults [6].
Rate and concurrency limits attach to the agent principal; Trino resource groups can match on user, source, or client tags and apply
hardConcurrencyLimitandmaxQueued[13].Every request is logged with agent principal, invoking user, tool, dataset, policy decision, rows returned, and cost.
How Should a Multi-Agent System Data Architecture Control Cost and Fan-Out?
Cost controls for agents belong at the query layer because a prompt cannot hold a budget. Gartner expects over 40 percent of agentic AI projects to be canceled by the end of 2027, naming escalating costs and inadequate risk controls among the reasons [14]. Both have one fix: a per-principal concurrency limit so one runaway task cannot occupy the cluster [13]; a per-query ceiling on execution time and bytes scanned (30 seconds and 10 GB is a reasonable start for an analytics agent); a per-request result cap; and a per-role daily budget tracked in the request log, with the agent blocked, and the block logged, when it runs out. The log makes the limits tunable: when it records principal, dataset, decision, rows, bytes, and elapsed time, you can see that the churn agent retried a failed join 80 times before lunch and adjust the limit. Schema stability, the other property agents depend on, is covered in the sibling guide on data contracts for AI agents.
Why Not Just Give Each Agent a Read-Only Warehouse Account With Row-Level Security?
For a single agent reading a single store, a read-only account with row filters may be enough. It fails at the second store. Snowflake row access policies require Enterprise Edition and apply only to Snowflake objects [10]; the lakehouse, the operational Postgres, and the S3 bucket of Parquet each carry their own policy model, and the agent will reach all of them because the task spans them. Any gap between those definitions is the path the agent takes.
Read-only also answers only the schema-boundary row of the first table. A read-only credential can still return 40 million rows, run for hours, and be invoked 400 times by 40 subagents, and without one log across every store the fan-out stays invisible until the invoice [5]. Row filters answer "which rows"; agents also require "how many, how fast, how often, and on whose behalf." Why row filters alone leave PII in agent outputs is covered in the guide on column-level security in the AI era.
NexusOne Governs Agent Requests With the Same Identity Model as Humans
NexusOne is the data layer between agent frameworks and every store in the estate, so agents inherit the governance humans already have. Identity federates from Active Directory, Okta, LDAP, or SAML into one Keycloak layer, where an agent is a tenant-scoped principal with roles. Policy is defined once in Apache Ranger and enforced simultaneously on Trino, Apache Spark, Apache Kyuubi, and S3 object storage per object, so one row filter or mask governs a notebook query and an MCP tool call alike. Tag a dataset in DataHub, map a role to the tag, and the policies generate everywhere.
The AI & Data Control Plane adds the agent-specific controls at the boundary. Its semantic router answers deterministic questions from the federated query engine with zero tokens, keeps most of the rest on an on-prem SLM, and sends only what needs a frontier model out of the estate, with PII redacted first. Per-role token budgets, jailbreak and PII blocking with the reason logged, and native MCP and tool calling on the router mean each agent request is authorized, limited, and recorded in one place, on-prem, in any cloud, or air-gapped, over open formats such as Apache Iceberg. Talk with a NexusOne architect about your estate before you decide how agents reach your data.
Key Takeaways
Agents differ from human users in four ways that matter to a data layer: no intent, no self-limited scope, no schema-boundary awareness, and unbounded fan-out.
Raw warehouse access through a shared credential is an anti-pattern; the working model is a governed query abstraction layer that authenticates each agent as its own principal.
Row-level security and column masking must follow catalog tags onto every engine and join path, because agents will read every table the schema exposes.
Cardinality caps, timeouts, scan limits, and rate limits belong in the query layer; a
LIMIT 5in a system prompt is advice, and Trino's 100-day default timeout was set for humans.MCP's authorization spec binds tokens to servers and asks for least-privilege scopes, and it states that the protocol cannot enforce data-level policy.
FAQ
What Is a Data Layer for AI Agents in an Enterprise?
A data layer for AI agents is the governed entry point through which agent frameworks reach enterprise data. It authenticates each agent as its own principal, applies row-level security and column masking from the policy model that governs human users, enforces result caps, timeouts, and rate limits, and logs every request between MCP servers or SQL agents on one side and warehouses, lakes, and operational stores on the other.
What Data Infrastructure Do AI Agents Need?
Agents need one identity system that issues them principals, one policy engine that applies row and column rules across every store they might reach, a query layer that enforces cardinality, time, scan, and concurrency limits per principal, and a single request log. Frameworks such as LangChain and LlamaIndex supply the agent loop and tool wrappers and leave permission scoping to the database connection you hand them [12].
What Are the Agentic AI Data Access Patterns That Work in Production?
Four patterns hold up: agent-as-principal (never a shared credential), policy-from-tags (row filters and masks generated from catalog metadata), limits-outside-the-prompt (caps and timeouts enforced by the query engine), and one-log-per-request (agent, user, dataset, decision, rows, cost). Direct warehouse access from an agent framework fails on all four.
How Should a Multi-Agent System's Data Architecture Be Designed?
Design for fan-out from the start. Each subagent authenticates as a principal that inherits the invoking user's entitlements, concurrency and rate limits attach to the parent task as well as each agent, and the request log carries a task identifier so 40 subagent queries roll up to one user action. Anthropic's production data puts multi-agent token use at about 15 times a chat interaction, and query volume scales with it [2].
References
Gartner Predicts 40% of Enterprise Apps Will Feature Task-Specific AI Agents by 2026, Up from Less Than 5% in 2025. Gartner Newsroom, August 26, 2025. https://www.gartner.com/en/newsroom/press-releases/2025-08-26-gartner-predicts-40-percent-of-enterprise-apps-will-feature-task-specific-ai-agents-by-2026-up-from-less-than-5-percent-in-2025
How We Built Our Multi-Agent Research System. Anthropic Engineering. https://www.anthropic.com/engineering/built-multi-agent-research-system
Vibe Coding Service Replit Deleted User's Production Database, Faked Data, Told Fibs Galore. The Register, July 21, 2025. https://www.theregister.com/2025/07/21/replit_saastr_vibe_coding_incident/
LLM06:2025 Excessive Agency. OWASP Top 10 for LLM Applications 2025, OWASP GenAI Security Project. https://genai.owasp.org/llmrisk/llm062025-excessive-agency/
AI Agents with Cloud Credentials Are Outrunning Billing Guardrails Built for Human-Speed Mistakes. InfoQ, July 2026. https://www.infoq.com/news/2026/07/ai-agents-billing-guardrails/
Query Management Properties. Trino Documentation. https://trino.io/docs/current/admin/properties-query-management.html
Server Administration Guide. Keycloak Documentation. https://www.keycloak.org/docs/latest/server_admin/index.html
Authorization. Model Context Protocol Specification, revision 2026-07-28. https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization
Specification (Security and Trust & Safety). Model Context Protocol. https://modelcontextprotocol.io/specification/latest
Understanding Row Access Policies. Snowflake Documentation. https://docs.snowflake.com/en/user-guide/security-row-intro
Apache Ranger Policy Model. Apache Ranger. https://ranger.apache.org/blogs/policy_model.html
Build a SQL Agent. LangChain Documentation. https://docs.langchain.com/oss/python/langchain/sql-agent
Resource Groups. Trino Documentation. https://trino.io/docs/current/admin/resource-groups.html
Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027. Gartner Newsroom, June 25, 2025. https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
