Implementing Identity-Aware AI Data Agents with AWS Lake Formation

In the evolving landscape of data management, businesses are increasingly looking to leverage AI to facilitate user interactions with complex datasets. A recent development involves the creation of identity-aware AI data agents that enable business users to pose questions about lakehouse data in natural language. This innovation is particularly significant as organizations have already established governance policies to control access to various datasets. The primary challenge lies in ensuring that these agents respect those governance rules without necessitating a complete overhaul of the application code.

When a user submits a query, the AI data agent is responsible for mapping that query to the appropriate data and constructing a corresponding database query. The agent operates under its own AWS Identity and Access Management (IAM) role, which means that AWS Lake Formation recognizes the agent's credentials rather than the individual making the request. This situation presents two less-than-ideal options for organizations: they can either restrict the tool's access, which limits self-service analytics, or they can rebuild access controls directly into the application code, effectively moving governance out of the data layer.

To address these challenges, a new approach is proposed: identity-aware AI data agents that propagate each user's identity throughout the entire process—from the user, through the agent, and into the tool. This ensures that Lake Formation can evaluate the user's access rights accurately. As a result, application code does not need to make any authorization decisions, existing governance policies remain intact, and AWS CloudTrail can effectively track which individual accessed the data.

The foundation of this approach relies on several well-documented components, including OAuth 2.0 token delegation, AWS IAM Identity Center, Lake Formation, and Lambda. A key innovation is the introduction of a foundation model (FM) that sits within the identity propagation chain. This FM acts as the agent's brain, determining which tools to engage and how to structure the arguments. Importantly, the user's identity token must traverse the HTTP transport layer directly, bypassing the FM's reasoning layer entirely. This article elaborates on the three configurations within the Amazon Bedrock AgentCore that facilitate this process.

For developers building agents on frameworks such as Strands or LangGraph, the insights provided here offer a deployable pattern using Amazon Bedrock AgentCore. Security engineers will find the identity transport and audit properties particularly relevant, while data platform owners will appreciate how existing Lake Formation grants can extend seamlessly to AI workloads without necessitating any alterations.

Enhancing Data Access with User Identity

The proposed pattern allows data agents to query Lake Formation-governed data with per-user access controls, comprehensive CloudTrail audit trails, and no modifications to existing governance policies. Lake Formation evaluates grants based on the actual user, ensuring that only the rows and columns to which the user is entitled are returned. Furthermore, CloudTrail maintains a record of the assumed role, clearly identifying the human user behind the query.

The architecture's design emphasizes the importance of maintaining user identity throughout the querying process. By treating the agent as an OAuth delegated actor, the user's token travels alongside the agent's HTTP transport, yet does not enter the model's context. The token exchange that generates query-time credentials occurs server-side within Lambda, ensuring that it remains scoped to a single invocation.

The Practical Implications

To effectively implement this pattern, organizations need to configure four key components: the request header allowlist on the AgentCore Runtime, the metadata configuration for the AgentCore gateway, and the propagation of headers within Lambda. This configuration guarantees that the user's identity is preserved throughout the process, allowing for accurate governance without compromising security. The result is a clean separation where the user's identity is maintained end-to-end, the model remains unaware of the user's token, and the data layer enforces access controls as if the user had queried the data directly.

With the implementation of identity-aware AI data agents, businesses can harness the power of AI to facilitate data access while ensuring compliance with governance policies. This approach not only streamlines the querying process but also enhances security and audit capabilities, making it a significant advancement in the field of AI design and data management.