How can I wield an LLM to create real value within complex domains?

Shifting from extracting answers to explicitly mapping value

I quit my job working on Gemini in late 2024 to pursue the deeper tendrils of this question. I spent a decade doing deep learning research at Google, and worked on LLMs for code generation before they were cool.

I’ve since talked to and advised CEOs, VPs and ICs across various fields. We’ve been curious about how they could better adopt AI within their organizations. Specifically, for non-coding complex tasks. LLMs are very good at generating seemingly plausible answers to questions within complex domains. Domains where success depends on nuance, context, taste and judgment. However, plausibility is necessary but insufficient for creating value in such domains. Most questions leaders ask me are downstream of overcoming a specific problem. How can I prompt an LLM to create real value within complex domains?

I won’t pretend to have The Answer to this big question. However, I’ve spent years wrestling with it. Much of this difficulty comes from the fact that LLMs are deeply ontologically odd. They’re neither “people” nor “things”, and fall within a crack between those categories.

I’ll provide an overview of how this inter-categorical nature arises from how they’re built. I’ll then introduce a tool called a Value Map to make it easier to wield LLMs more effectively.

This essay will be easiest to digest for readers familiar with the literature on psychological development (e.g. Kegan, Piaget, etc), and specifically Professor John Vervaeke’s work on Relevance Realization.

---

LLMs produce both magic and idiocy

It’s quite jarring that Codex can rapidly generate an entire working codebase from scratch, yet seem so dumb when I ask it nuanced questions with no “right” answer. This seeming contradiction is maddening for anyone first learning to use LLMs, myself included.

I’ve noticed an archetypical pattern when people first encounter LLMs. They’ll decide to try an LLM on a complex task like refining their company’s strategy doc, making a hiring decision or brainstorming solutions to some interpersonal conflict. They’ll then try uploading the relevant docs to the LLM and give it a vague prompt like “Please solve this problem.”

One of two things typically happens. The LLM will give them something that blows them away, and they’ll be convinced that the LLM is a magic box that can do anything. Or it’ll produce sophisticated-sounding nonsense. They’ll keep throwing more prompts at the wall, eventually giving up and concluding that LLMs are untrustworthy idiots. Each iteration through this movie generally sediments their default stance towards LLMs.

Perhaps they’ll even slosh back and forth a few times along this spectrum. People seem to go through this pattern every time there’s a step-change qualitative improvement in the frontier models.

The dilemma of using LLMs

The most valuable problems in the world are deeply complex with no “right” answer. Who doesn’t want the capacity to create extraordinary value with LLMs on such complex problems? Especially if it’s possible to do so without abandoning our sense of taste, reality and individual specialness.

Yet we’re constantly torn between outsourcing our faith to the LLMs and pessimistically writing them off as untrustworthy idiots.

Outsourcing our faith to the LLMs is tempting because in many ways, they are magic boxes! The most dogmatic true believers will say that the machine gods are at hand. That we must be careful lest we summon silicon demons instead of machines of loving grace.

Pessimistically writing off LLMs is tempting because it’s true that they’re massively over-hyped. They can’t do everything yet, and the AI ecosystem is full of grifters. The most dogmatic skeptics would say that LLMs are just next-token predicting “stochastic parrots”. That LLMs don’t truly understand everything, that they’re building intelligence on “incomplete math”, and LLMs are fundamentally untrustworthy for solving complex problems.

Both the true believers and the true skeptics directionally carry deep wisdom. LLMs are clearly going to change the world and have already changed the world. Getting left behind by LLMs within our current state of capitalism is terrifying. There’s value in embracing innovation. On the other hand, it’s true that we shouldn’t let our professional careers, and our civilization, drive off the cliff with ungrounded thinking. There’s value in maintaining a sense of groundedness.

It’s easy to feel torn between these valid perspectives. Fortunately, this is a false dilemma. Each perspective on value creation is incomplete because they misunderstand not just what an LLM is, but where value lives in relation to an LLM.

Frontier LLMs are deeply ontologically unfamiliar. They’re clearly not “people” in the way we understand that term, but neither are they merely “things”. They’re far more sophisticated than a toaster, a car, an airplane or any other “thing” I may interact with. They can convincingly express person-like psychological patterns despite having no embodiment, consciousness or fear of mortality. Specifically, they can make it seem as if they experientially know things, despite never having truly experienced anything.

We don’t already have cultural tools for naming, relating to and effectively working with such artifacts. So we’re tempted to unconsciously relate to LLMs via categories we’re already intimately familiar with. It’s easy to relate to an LLM as if it’s a wise and knowledgeable person when it produces a compelling response. Similarly, it’s easy to reduce it to an over-hyped boondoggle when it produces nonsense.

Neither stance is entirely wrong, but neither stance is entirely right. Finding a more complete stance requires us to first understand how LLMs are built to better understand what they are, so that we can create more value with them.

What can pre-training teach us about prompting LLMs?

Pre-training essentially involves training the LLM to statistically model the public internet. The training procedure involves taking the whole public internet, cutting it up into sub-sequences of say 1000 words, and training an LLM to predict the 1,001st word across all of these sequences simultaneously. It does many zillions of iterations of this next-word prediction training procedure, and seeks to learn how to produce good predictions across all of them.

This simultaneity allows the LLM to deeply learn patterns across every written artifact across culture, proportional to how much that culture has interacted with the internet. These artifacts can be stories, news articles, blogs, code, etc. across many languages. These artifacts capture representations of lived human experience their authors found worth publishing on the internet.

Each culture can be thought of as its own “cinematic universe” of what is true, good and beautiful. What might be “good” in the East may not be identical in the West. The style guide that Google considers “good” for Python may disagree with PEP8 along a number of dimensions.

Cultures organically produce artifacts that enable their participants to solve problems so as to better achieve what that culture believes is true, good and beautiful. Such an artifact’s value is proportionate to the difficulty of the problem that it helps to solve. The hardest problems are often those that require us to change and grow our relationship with ourselves, and our environment. That is, problems that invite psychological development.

The Hero’s Journey is a cross-cultural schema for mapping the arc of psychological development. The internet is rife with cultural artifacts that encode either a full hero’s journey, or fragments of one. Everything from myths, scripture, memoirs, blog posts, philosophical works, business analysis, fiction, web forums and even code either encodes, or is causally downstream of some such journey.

All these cross-cultural developmental arcs and their fragments are within a pre-trained LLM. That’s a big part of why LLMs can feel so psychologically rich, despite never having actually lived through such development themselves. It also contributes to their ontological oddness by placing them between the cracks of “person” and “thing”.

This capacity to role-play the human condition makes it tempting for an end-user to assume that the model already knows the rich cultural world we inhabit, and which patterns within the LLM’s latent structures are most relevant for our specific context.

Practically speaking, if an LLM doesn’t know what “world”, “culture” or “cinematic universe” it’s in, it’ll infer one from the prompt. Therefore, prompting well begins with explicitly showing the model the broader “agent-arena relationship” that it’s participating in, rather than assuming it can flawlessly infer it.

What can SFT/RLHF teach us about prompting LLMs?

Specifying such an agent-arena relationship to a pre-trained model can be quite cumbersome. Supervised fine-tuning and Reinforcement Learning from Human Feedback are the first two steps in post-training. They serve to give the LLM a more coherent agent-arena relationship. They teach the model its name, that it interacts with the world via conversational turns, that it should strive to follow the user’s instructions while following broader principles of helpfulness, honesty and harmlessness.

At a high level, SFT seeds the LLM’s behavior for RLHF, by training the model against golden conversational examples for it to fit against.

At a high level, RLHF training data is collected by giving the model a prompt, asking it to generate a few candidates, and then asking paid human raters to pick the best responses based on some guidelines developed by the LLM’s engineers. For example, the input prompt might be something like “Generate Python code to find the first N primes.” A human rater then picks the best response, and this whole process happens a zillion times. The LLM gradually learns to produce responses that receive higher rewards from the human rating pool.

However, the intangible richness of human preferences and values can’t be easily compressed down into a single scalar in this way. As a rater, it’s impossible to give only the preferences you’d want, and to exclude any preferences you’d unconsciously possess. As an engineer, it’s impossible to write down any “guideline” or “legislation” that can perfectly anticipate every edge-case in reality, even if it’s iteratively improved over time. There’s a reason the US follows the Common Law system.

As a result of SFT and RLHF, the model learns a behavioral attractor for responding to prompts from every conceivable domain. This attractor is a product of the rating population’s implicit and explicit notions of what constitutes a “good” response across a diverse pool of raters. As a metaphor, this behavioral attractor increasingly tracks something like the flattened aggregated “spirit” of all human knowledge and culture, as channeled through the flattened aggregated “spirit” of the rater pool, informed by the cultural processes of the relevant frontier lab.

This further exacerbates the ontological unfamiliarity of an SFT+RLHF’ed LLM. This behavioral attractor makes it seem as if the model is someone with a coherent name, recognizable “personality”, and an agreeableness towards following our instructions. Such role-play makes it tempting to assume that the LLM is unconsciously tracking our local context, implicit expectations and deeper motivations in the same way that a human collaborator might. However, it’s neither a “person” in the way we’d conceptualize that category nor merely a “thing”.

Therefore, practically speaking, prompting an LLM is akin to instructing a competent, well-meaning contractor with access to the sum of all human knowledge at their fingertips and none of your local context. Imagine that the prompt text box is the only message they’ll ever receive, and each inference call will invoke a fresh new contractor with broadly the same competencies and educational background as the previous ones. If you’re lucky, the contractor who gets “summoned” has the same implicit context as you, and you won’t have to be as explicit with what you want. If you’re unlucky, you’ll have to describe yourself in painstaking detail so as to explain your problem to someone from an alien culture, restricted to the text box as a medium of transmission.

SFT+RLHF’ing a pre-trained LLM essentially creates a “user interface” for the global content stored inside of it. Specifically, it allows users to use explicit instructions to reorganize this global content to elicit value-creating behaviors within their local contexts.

What can RL with verifiable rewards teach us about prompting LLMs?

People started deploying SFT+RLHF’ed LLMs to sequential decision-making tasks with greater levels of autonomy over longer time horizons. However, such models started going off the rails after a certain time horizon. Doing RL on verifiable rewards from a broader “environment”, rather than human feedback as in RLHF, made it easier to train models to perform coherently across longer time horizons. METR plots each new frontier model’s ability to solve tasks over longer time horizons. As of June 2026, task completion has been rapidly increasing with every new release.

Understanding an important limiting factor for increasing task horizon length sheds light on how we can create value with such models. The observable symptom is that LLMs make “errors” at each time-step, these errors compound and that’s what ultimately makes them unreliable for tasks involving sufficiently long time horizons. Let’s slow down and unpack this.

The “dream” of LLM-based agents is to be able to drop them into tasks that were previously entirely executed by a human. However, the most valuable tasks occupied by humans often contain substantial ambiguity and unpredictability. The environment may silently change in ways that are unobservable, a specific lever for influencing the environment may fail unpredictably, and so on. Every time-step in such a task exposes the agent, whether it’s a machine or a human, to a combinatorially explosive space of possibilities and competing decisions. It must unpack the underlying purposes behind those competing decisions, choose one that’s most aligned with the organization’s broader purpose, and successfully enact its choice.

Human organizations have various cultural processes that continuously develop their employees’ capacity to resolve such competing decisions in accordance with what’s most valuable for the organization. This is often true both during an individual’s execution of a task, and between tasks. However, we don’t yet have such cultural processes or engineering mechanisms for “developing” LLM-based employees already deployed in the wild. Prompting is a fairly brittle mechanism for aligning an LLM-based agent over long time horizons.

Training an LLM with verifiable rewards across a diverse range of environments over long time horizons creates a coherence in how it resolves competing decisions, similar to what takes place in human psychological development. It’s akin to how a human’s “identity” changes and coheres as they develop psychologically. This faux-development likely creates “bleed” into the model’s general capabilities, making it appear “smarter” than it used to be without extensive RL post-training. Such faux-development likely also makes the model much better at seeming like it has “good vibes”.

Crucially, it further exacerbates the inter-categorical confusion between “person” and “thing”, because the LLM starts to behave autonomously over longer time horizons in ways that resemble a real human collaborator.

However, apparent competence at autonomous behavior is not the same as alignment with your task’s broader purpose. A model coherently pursuing a task may still be optimizing for a notion of “success” that’s subtly but importantly different from your own.

The more capable a model appears, the more important it becomes to explicitly specify what “value” means within the arena it’s deployed in and to establish rich measurement in proportion to its increased capabilities.

Explicitly mapping value instead of extracting answers

Each stage of training makes an LLM easier to wield while simultaneously making it harder to notice its inter-categorical nature as something between “person” and “thing”. Pre-training gives it access to vast patterns across human culture. SFT+RLHF provide a conversational user interface for these patterns, and RL allows the model to enact increasingly complex task-driven identities.

This inter-categorical confusion makes it easy to fall into the trap of either outsourcing our faith to LLMs or pessimistically writing them off. Both stances relate to the LLM as if it’s something from which we can extract answers, which limits the value that can get created.

Creating exceptional value with an LLM requires an internal shift in perspective from extracting answers from the LLM towards explicitly mapping out what’s valuable.

Users attempting to extract answers from an LLM typically seek a “fixed final answer” for creating value. They relate to unexpected behavior as mistakes that must be controlled and fixed.

In contrast, users attempting to take responsibility for explicitly mapping value treat value creation as an iterative, dialogical and ongoing process. They relate to unexpected behavior as misunderstandings to resolve in an ongoing fashion by providing more context.

How can I explicitly map value for an LLM?

Creating a Value Map for a given problem enables us to take responsibility for explicitly mapping value for a given task. A Value Map comprises two pieces:

• Arena Ticket - contextualizes the broader arena the LLM participates in. It explicitly defines the LLM’s task, the shape of the arena and the sort of value we’re instructing it to create.

• Measurement Ticket - decides how we’ll measure whether the LLM’s actually following our instructions, and how we’ll measure changes in the broader arena to see if value’s actually being created.

Creating Value Maps and translating them to an LLM is a relational skill that requires deliberate practice. Many leaders already have experience turning implicit context of what’s valuable into explicit maps for their employees. However, essentially no human beings have experience doing so for the totally novel simulacrum of human consciousness that is an LLM. Doing so necessitates shifts in the value mapper’s consciousness, no matter how developed they already are.

An Arena Ticket involves answering the following archetypical questions:

1. What’s the broader context/situation/arena that the LLM is placed into?

   • If you were onboarding a new employee/contractor with access to all human content, but no context about your situation, what’s the minimum set of explicit, observable context you’d include?

   • Imagine that you can only communicate with this new employee/contractor via your system instructions and prompt, and it’ll be a new person at every inference call. What would you tell them?

2. What’s your minimal yet legible description of the task? Describe in a way that such a context-less contractor would understand.

3. What’s at stake? What values, constraints or trade-offs matter for completing this task?

   • Focus on the things you’d want to have happen, rather than all the things you don’t want to have happen. Set your context-less contractor up for success.

4. Do you have a few examples of “ideal” or “gold” responses that may show, rather than tell, the LLM what it should do for some archetypical inputs?

5. What local context from the arena is not already visible to the LLM?

A Measurement Ticket involves the following archetypical questions:

1. How can I measure whether the LLM’s outputs and actions are actually aligned with the spirit of my instructions?

2. How can I measure whether the LLM’s outputs and actions are creating intersubjectively observable changes in the arena?

3. What is my rationale for why such changes in the arena lead to broader value creation within my cultural context?

By design, the answers within the Arena Ticket constrain and inform the space of possible answers within a Measurement Ticket, and vice versa. That is, value mapping is an inherently iterative process. A good Value Map for an LLM also makes it easy to onboard other human collaborators onto it.

A concrete example of value mapping

Suppose you’re working with a chatbot like ChatGPT to brainstorm your product’s strategy for 2026. You’ve uploaded your strategy doc and asked for feedback. It generates a plausible revision, and now you need to decide what to do with it.

A stance of extracting answers would enact the following loop:

1. Prompt the LLM to improve the strategy doc.

2. Read the response it generates.

3. Ask yourself if the response is good.

4. If it is, accept it. Otherwise, keep tweaking the prompt.

Notice how the LLM’s output silently defines what’s “good”, and allows the human to get unconsciously swept along. This is particularly dangerous for a complex task like constructing a company strategy.

Taking responsibility for explicitly mapping value radically changes the user’s relationship with the LLM. After each inference call, the user explicitly brings into consciousness the following questions that are based on the previous section’s archetypical questions for value mapping.

1. What does “success” for this document look like? Specifically, what sort of intersubjectively observable changes in behavior am I trying to engender within my co-workers with this document?

   • Which specific co-workers am I targeting?

   • Which specific co-workers am I explicitly not targeting?

   • How will it be framed and presented to them?

2. What’s my specific rationale for whether the current LLM-generated revision will lead to those observable changes?

3. What specific changes need to be made to the LLM-generated revision to produce a materially better artifact, and what are the underlying higher-order leadership principles motivating those changes?

4. Have I provided the LLM with context on all of these considerations, in a way that’s legible to it?

Engaging with this material consciously makes it easier to filter out bad LLM responses, and to steer the model far more efficiently towards value creation.

Conclusion

Humankind’s never produced any artifact quite like a frontier LLM. LLMs fall within the inter-categorical cracks between “person” and “thing”, and possess a compressed representation of the sum of all human technical and developmental content.

Their apparent personhood and agency make it easy to either outsource our faith to their magic or write them off as untrustworthy idiots. However, value doesn’t exist in the LLMs, but between the LLMs and the arena we choose to embed them into. Sitting in the practice of crafting Value Maps invites us to take greater responsibility for both making that relationship explicit and measuring whether value’s actually being created.

Their inter-categorical nature is a powerful provocation into not just who we are as human beings, but what we are as human beings. About what we truly value and stand for. About our own relationship with what we consider to be true, good and beautiful.

The next time you find yourself swept away, either positively or negatively, by whatever an LLM produces, I’d invite you to pause and consciously ask yourself what value you’re actually trying to create with it, which arena you’re trying to create it within and how you’ll take accountability for measuring whether such value is actually being created.

---

If this essay resonated with you, I’d love to hear from you! Please email me at varun@doubleascent.com.

Acknowledgements

• Brian Whetten for everything he’s taught me about growing my own leadership, developmental patterns, communicating complex ideas and his idea of a “Value Sandwich” that eventually became a Value Map.

• John Vervaeke whose work on mapping human cognition and exploration of Relevance Realization was instrumental for articulating this essay. e.g. “agent-arena relationship”, etc.

• David Chapman and Charlie Awbery for their work on mapping out and developing practices for the interplay between nebulosity and pattern.