Why Human Judgment Matters More Than Data in Enterprise AI
The right context starts with a clear purpose.
LLMs have broad general-purpose knowledge. Without task-specific context, they tend to produce a reasonable answer for the typical case, not necessarily yours. That's the average.
It covers roughly 80% of a person's work, with little effort, and that's already useful. But I'm more interested in reaching 120%, using AI to produce work better than we could achieve alone.
Getting there takes human purpose and judgment. People give the model direction, either during the work or by building their expertise into the system beforehand. I call this shifting away from the average.
Three popular beliefs share the same assumption: give AI enough information, and it will know what matters.
- Second brains: "Give the AI all my notes and it will understand me."
- LLM Wikis: "Let the AI organize everything in advance, and the knowledge will be ready for any question."
- "RAG is dead": "Coding agents find what they need on their own, so companies don't need retrieval engineering anymore."
The mistake is assuming that collecting, organizing, or finding information is enough to achieve the result you need. The best GenAI use cases start with a clear goal and build human judgment into the system to achieve it.
Human Judgment Shifts the Answer Away From the Average
Human purpose decides which direction the answer should move.
Picture the answers an LLM could give spread across a page. That breadth is what makes LLMs useful, and it's also why their default is the average. With a broad request, the answer tends to land near the middle, where it fits most situations. The answer worth finding might sit somewhere else: toward an exception, a contradiction, or a detail that changes the conclusion.
Human purpose sets the direction. Context helps shift the answer.
Moving away from the middle isn't the goal by itself. Purpose tells the model which way to go.
Once you've chosen a model, the most direct way to guide it is through what goes in. That's why so much comes down to context.
Sometimes the point of context is not to teach the model something new. It is to stop the model from giving the answer it would normally give.
Human judgment shapes AI work in five ways:
- Meaning. People define what data, terms, and business metrics mean.
- Selection. People decide which information the model receives, when, and in what order.
- Rules. People establish the policies, constraints, and exceptions the agent must follow.
- Standards. People define what counts as a correct result and how the system's work should be evaluated.
- Direction. People question assumptions, correct mistakes, and redirect the model as the work develops.
Building Human Judgment Into Enterprise AI
For tasks that require specialized knowledge, an AI agent needs access to that expertise to work reliably.
BMW's Mobile Data Recorder (MDR) project shows how.
BMW's test cars record thousands of sensor signals. For years, only a small group of specialists knew how to query and interpret the data.
The model could generate queries, but it needed BMW's expertise to understand what the data meant.
According to Microsoft's Marco Casalaina, the BMW team spent about six months working across departments to build a data dictionary. They defined what sensor signals meant, which thresholds mattered, and which terminology engineers used.
The data dictionary was part of a broader system combining domain knowledge, modernized data infrastructure, and specialized AI agents.
With that system in place, an agent could turn a question like "find hard-braking events in wet weather last week" into the right analysis. According to Microsoft, early internal results indicate that analysis can be up to 12 times faster.
The same idea works beyond BMW:
Semantic layers. People define what revenue means and how each metric is calculated, so the model interprets the data the same way the business does.
Evaluation. An agent might complete a pricing spreadsheet and still get the prices wrong. An expert explains what went wrong, and the team uses that reasoning to evaluate and improve the agent.
Reliability is part of quality. The goal is to make sure the agent gets it right consistently.
More Data and Better Search Still Need Judgment
A Second Brain Is RAG Without a Purpose
The idea: Put years of notes, journals, and documents into a folder, give an AI access to them, and it will know you well enough to write your articles, emails, and scripts.
What people get wrong: They assume a collection of personal notes is already a knowledge base designed for the work they want AI to do.
How information is organized, what gets stored, and how it's retrieved all shape the context the model receives. Without a clear purpose guiding those decisions, the AI combines whatever looks relevant into a generic answer.
It has years of personal data, but no reliable way to tell what actually matters for the task. And much of what matters about a person was never written down.
Collecting knowledge and designing context for a task are different jobs.
Synthesis Needs a Direction
Some systems marketed as LLM Wikis are still folders of notes with a search box. Karpathy's LLM Wiki goes further. It uses ingest-time synthesis to build and update linked Markdown pages as new sources arrive, creating reusable knowledge before a future question is asked.
The idea: Let AI organize everything in advance, and the knowledge will be ready for any future question.
What people get wrong: They assume that organizing knowledge in advance makes it useful, whatever they're trying to achieve.
Without a clear purpose, synthesis favors the patterns common across sources. The result can be accurate and well organized and still pull you toward an average understanding of the topic. If you're trying to develop your own point of view, the exception, the contradiction, or the unusual detail is often exactly what matters, and it's what gets smoothed away.
Karpathy's own approach shows why human direction matters. He chooses the sources, shapes how the wiki is organized, and reviews how new information is integrated. His design also preserves the original documents and tracks contradictions.
But even a wiki that records every contradiction can't know in advance which one will matter to your next question. Recording a contradiction is not the same as knowing why it matters. That depends on what you're trying to understand.
The synthesis can be right. The judgment can still be wrong.
That's why going back to the original sources isn't wasted work. A new question might need evidence that a summary built for a different purpose left out.
The Limits of Lightweight Retrieval
The idea: Coding agents can find what they need on their own, so companies no longer need retrieval engineering.
What people get wrong: They assume finding relevant information is the whole job. An agent also needs the right context, rules, and controls to complete the task.
Code repositories give agents file paths, symbols, dependencies, and often tests to check their work. These structures make lightweight search effective for many coding tasks.
Simple search is enough for many tasks with clear goals and easy-to-use information, whether they're coding tasks or enterprise workflows.
But many of the most valuable enterprise GenAI use cases are far less tidy. They involve more information than people can realistically examine, spread across systems with conflicting definitions, changing versions, and outdated documents.
Consider an agent reviewing a supplier contract. Finding the relevant clauses is only the first step. It also needs to apply the right company policies, identify exceptions, and know when a decision requires human review.
The clause that looks most like the question isn't always the one that decides the case. Following the easiest match can pull the answer back toward a generic one.
That's where purpose shapes the design. People decide which information the agent should use, which rules apply, and how its results should be checked. Some of this requires retrieval engineering; other parts require workflow controls and evaluation.
Relevance is designed before it is retrieved.
An agent can discover evidence nobody expected. Purpose decides what that discovery means for the task.
RAG was never just about vector databases. It's about bringing the right external context to the model when the task needs it. Search can still be agentic, while the system controls which sources are used and how the evidence is applied.
Build established knowledge into the system, and retrieve the evidence each task needs.
As models get better at finding context, the value of engineering shifts toward defining what that context means.
Good Use Cases Start From the Gap
The gap between a generic answer and what the task demands reveals where human judgment matters.
If the model's first answer is good enough, take the 80% and move on. When it isn't, the question is whether better context, expert knowledge, or system design can close a gap that matters.
AI can also connect information across systems and documents at a scale that would be difficult for people to manage. That makes new analysis possible, but it raises questions that more data and better search cannot settle.
More sources mean more decisions about meaning.
A model can compare information across systems, but someone has to establish which definition is correct, which version applies, and which source the organization trusts. Once those decisions are made explicit, an agent can apply them across far more information than people could process themselves.
A model cannot recover knowledge the organization never captured.
Much of what experts know lives in their experience: why a threshold was chosen, which exceptions are acceptable, or when a record is no longer reliable. An agent can identify gaps and ask questions, but it cannot know facts that were never made available to it. BMW's six months of work went into making that expertise accessible.
Purpose determines the direction.
A generic answer can be useful without being right for the task. The model can examine different possibilities, but people must decide which questions are worth asking, which evidence matters, and what a good result should achieve.
Someone still has to own the decision.
When an incorrect result has serious consequences, producing an answer and acting on it are different responsibilities. People must decide what the agent is allowed to do, when it needs approval, and who is accountable for the outcome.
Better models can search, compare, and connect more information. They make human expertise usable across more work, but they don't remove the need for people to define what matters and take responsibility for decisions.
Everyone gets the same average. The advantage is the judgment you build around it.