Automata Network - 2.0 (@AutomataNetwork): The 5 levels of agentic trust

The 5 levels of agentic trust Our own feelings about contemporary AI agents are warm, but nuanced. In this article we describe our mental model for the concept of agentic trust, where it currently falls short, and what the end goal should be. 🖼️ https://pbs.twimg.com/media/Gh50gh3a8AArhUs.png Level 1: Opaque AI agents ‘Security’ through obscurity We believe that AI agents as they exist today are a) a very useful tool for the world and b) no better than centralized applications with a trust-me problem elaborated in this earlier post (https://blog.ata.network/verifiability-as-the-missing-piece-for-ai-agents-in-web3-504839dca893). The implications are threefold; i) for the end user, who delegates decisions without any uniform way to verify the veracity of the actions undertaken by the AI agent nor the agent itself; for intermediaries, who offer AI services to users (e.g. https://x.com/elizaos_ai , a framework for AI agents), and seeks to ensure the legitimacy and integrity of the model; and developers, who face the uphill task of building openly auditable AI systems without exposing its inner workings. Level 2: Verifiable AI agents AI agent does what it is expected to Broadly defined, the question is: how do we help users trust AI agents? We start by verifying its components. Readers who are frequent readers of this blog might already be familiar with hardware-based cryptographic gadgets like TEEs (Trusted Execution Environments) to prove the execution integrity of the AI model that ran within the secure enclave. If we want to have the ability to compare different levels of agentic trust, we ought to make moves towards trust minimization. That the AI agent operates in an expected way without any dependencies is a start. Techniques such as zero-knowledge proofs (ZKML) are another way of adding guarantees to AI agents, but it is not obvious at this point if the overheads introduced make it a feasible option to what is already a computationally intensive process. Level 3: Authentic AI agents AI agents are verified against its claims If the AI agent the end user interacts with is consistent with the LLM model, that’s a kind of verifiability: Level 3 provides cryptographic proofs to verify that each component of the agentic framework is what it claims to be. This establishes trust in the authenticity of the AI agent. If at Level 2 we expect a cryptographic hash of the program executed by the TEE, along with the TEE attestation, Level 3 would necessitate tools such as attestable builds (https://community.aws/content/2lZdD8iDfAGbQpWK8g8fGvQ1kp8/creating-attestable-builds-with-aws-nitro-enclaves?lang=en) (reproducible build compiled within a TEE) to attribute the AI agent to its referenced model. Verification is often limited if parts of the AI system are proprietary or not fully accessible i.e. LLM models are kept private by the organization that developed them, or when training data is not disclosed. The core of agentic trust, this should be non-negotiable for any AI workflow and what we believe the industry should strive towards in earnest. Level 4: Real-world AI agents Integrity of data sources are validated AI agents handle different information contexts on the fly and require verifiable data sources to respond and make high-quality decisions. The limitations of frontier model training are well espoused with “rubbish-in-rubbish-out” data systems that throttle the progress of AI with LLMs trained on huge swaths of junk content scraped from the internet. One idea that might help avoid degradation of AI agents is to make sure the model gives more weight to data that can prove its trail from its point of generation, known as data provenance. This can be achieved (https://pom.ata.network/) with verifiable machines that can validate the origin of data. Level 5: Autonomous AI agents Provable ‘deletion’ of humans Our simple one-sentence definition of autonomy: Autonomy is the ability to make decisions without the possibility of external override. The human is persona non grata when it comes to decisions undertaken by the AI agent. Additional proofs are required to ensure that intervention is not possible at the model level. Open AI frameworks that are subject to public oversight benefit from having its correctness demonstrably proven; this ensures that, for example, no backdoors exist, nor any harmful bias embedded within the training data. The majority of trust efforts have gone to allow different parts of the agentic framework to trust each other, such as the relationship between the LLM and the AI agent, but this is importantly different from our goal of assuring the end user. Accountability is crucial if we are to have AIs that can act truly agentically and in accordance with human intent: that is, alignment. This also means removing (implicitly) trusted parties from the threat model i.e. any entity involved in creating the AI model. At the same time, the authenticity of data fed into the AI model should no longer be a constraint.