What is an AI-first agent?
And what will happen to the world when they arrive?
Using Claude Code is a truly magical experience. But it’s still so frustrating in all sorts of ways. I can close my eyes and pretend, superficially, that I’m co-working with a senior engineer. But it just doesn’t feel the same. For example, I can’t easily communicate with it over Slack. It won’t go off and proactively gather requirements from the relevant stakeholders in the company. It often doesn’t have the context of various social and cultural dynamics motivating specific technical choices. A lot of this knowledge is some combination of tacit, embodied and normative. So it’s hard to remember ahead of time all the context that it may be missing. I could go on and on listing out other such sources of friction. I’m singling out CC because I use it the most. But all “agentic coding” tools I’ve seen suffer from these frictions.
There’s a deeper pattern underneath these frictions. Our community is confused about what it means for something to be an “agent”. When everything is an “agent”, nothing is. This and subsequent posts will explore a very specific sort of agent. One that maintains state across time, continuously learns from its environment and presents itself almost as a “coworker” in social environments. This is in contrast with other notions of “agent” that are either rigid workflows implemented with LLMs or enhanced information retrieval tools. Admittedly, we didn’t have models good or cheap enough for this confusion to matter until now. But we’ve started to get there with Gemini 3, Opus 4.5, GPT-5.1-Thinking, etc.
Before exploring the implications of such agents, we need to define them. That’s what this essay does. Subsequent posts will dig into the harder questions.
Amusingly, as is often the case, my response to this confusion is to provide my own definition. An AI-first agent is an AI-first product that also satisfies the following criteria.
Each agent instance has a clear “boundary” that separates what’s “inside” the agent and “outside”.
Each agent instance has the capacity to skilfully request extensions to its boundaries based on its ongoing internal processes, so as to create more value in its context. Conversely, reductions to its boundaries causally lead to an immediate shift in its ongoing internal processes, so as to continue creating the most value that it can.
Each agent instance can participate in social dynamics to change its behavior for the pursuit of value creation. For example, feedback from an agent instance’s “manager” might lead to immediate changes to the deepest levels of its 4P stack, like its name. But feedback from a fellow employee or another agent may not.
Each agent instance has an approximation of the 4P stack in its memory that allows it to better navigate its environment. Please refer to Professor John Vervaeke’s work for more context on this. This post contains a brief explanation of his work. More specifically:
Propositional knowledge - it has a collection of propositions that may not be in any of the underlying LLM’s weights, but that nevertheless condition them to better fit within the contexts they find themselves in.
Procedural knowledge - it has a collection of “skills” that it can deploy based on what it finds relevant within a given context.
Perspectival knowledge - it maintains a repertoire of “episodes” that inform what perspectives in its environment it should find relevant.
Participatory knowledge - it provides the agent with a coherent identity, overarching hierarchy of values, and a description of the agent-arena relationship that it’s participating in (that is, how the agent understands itself in relation to its environment and the other agents within it).
Notice that these criteria address the frustrations I opened with. The inability to communicate over Slack or gather requirements stems from the absence of persistent boundaries and social participation. The missing context about tacit social dynamics reflects an impoverished 4P stack.
My previous post shows that Claude Code isn’t yet an AI-first product. Therefore, it’s necessarily not an AI-first agent either. But it’s worth imagining what a version of CC might look like if it satisfied some of the criteria specified above. The most conspicuous gap is the absence of a stable identity. Without one, the agent cannot have clear boundaries because there is no persistent self to bound. It cannot participate in social dynamics because it lacks the continuity required for others to hold it accountable over time. And perhaps most importantly, it cannot develop participatory knowledge. A coherent identity, a stable hierarchy of values, and an understanding of one’s relationship to the arena one is participating in all require something that persists across interactions. Today’s CC has some memory via features like CLAUDE.md. But by default, it doesn’t continuously learn from each interaction. It cannot refine its sense of self over time.
Such an identity would start with something like a Google/Microsoft account: a name, email address, and a trusted description of its human owner. Possessing such a stable identity would also make it easier to deterministically enforce various rights and sanctions upon the agent. A stable identity would also make it easier to potentially give it access to a payment method for participating in the broader economy, and the ability to provision other identities (e.g. GitHub account, GCP account, etc).
Stable identities would make it easier to afford clear boundaries of “inside” and “outside” for the agent. For example, all of its private Drive files, emails, GitHub repositories, etc would be “inside” the agent’s boundary. It’d then use existing social environments like email, Slack, etc. to communicate with other human or AI agents that are “outside” of it. The agent’s governors and human collaborators would be able to reuse security semantics from each of these products. Ditto for its AI collaborators. One could also imagine a central dashboard for the agent’s owners to examine its current tasks, resources, thoughts and goals.
Developing software with such an agent within a broader social milieu would increasingly resemble developing with a human coworker. But one that possesses a lossy copy of all human knowledge. Its owner would give it specific permissions to the GitHub repo and shared Drive, and then ask it to go off and create value within the company.
Different “roles” of humans and agents would have different levels of social privileges to request/demand changes to its observable behavior via natural language instructions. For example, the agent’s owner would have the right to change its name, change its tasks, etc. But perhaps a human collaborator would only have the right to “request” it to join a specific piece of work. This would afford its human collaborators the ability to build cultural technologies to better govern its behavior/alignment.
I’ve described what might seem like a distant future. But each of these capabilities sits upon a spectrum, and we’re closer to viable versions than you might think. One can imagine substantially stronger and weaker versions. It’s not clear exactly when such a system will hit the market. But I’d anticipate some rudimentary MVP certainly within a year or two. I myself see concrete paths to bringing such an MVP to market in the next year with modest capital. Therefore, I know that many others do so too.
The arrival of AI-first agents will have profound implications for the world even if they’re not “AGI”. The next few posts in this series will map out some key open questions that arise. Most of these questions defy obvious answers or solutions. I invite you on a journey to sit in these questions with me.
What is the relationship between such AI-first agents and a company’s culture?
How can such AI-first agents be used to afford the cultural development of organizations?
How can we invite the social sciences into AI? What would it mean to develop a sociology of AI employees interacting with humans?
What would the governance of such agents look like?
Rather than thinking about an “end state”, what are some key thresholds that we should seek to monitor? Coding agents are a leading indicator of what’s happening. What’s happening to coding agents will soon take place for the rest of knowledge work.
What would it mean for an agent to develop genuine procedural and perspectival knowledge through experience? Today’s agents have pre-loaded skills. But they don’t acquire new skills through practice. They don’t accumulate episodes that inform what they find salient in novel situations. What changes when they can?
 | A guest post by
|