Recognizing informational insufficiency in sequential decisions
The broader phenomenon is awareness of unawareness: a reasoner may recognize that its knowledge is insufficient for a decision before it can identify exactly what information is missing.
How does a reasoner recognize that its knowledge is insufficient before identifying exactly what information is missing to take or not take an action?
This means the reasoner detects that something is missing before it knows what it is, which we call recognition before identification.
Why would we need such a thing?
Solving the object-level problem is only half of what an autonomous reasoner has to do. It also has to decide whether to commit, continue thinking, retrieve information, ask for help, try another approach, use a tool, or stop.
Recent work from Google DeepMind and Princeton showed that language
models have an internal confidence signal and use it to decide whether
to answer or abstain. Changing that signal inside the model changes how
often it abstains. Google DeepMind · Princeton · Nature
Now, confidence answers questions like how likely is my current answer to be correct? But there are other questions, such as is the reasoning process I am currently using capable of resolving this yet?
A system can have substantial compute available and still repeatedly pursue the same bad interpretation. More computation is not necessarily what it needs. It may need some signal that the current reasoning is failing to capture something necessary.
Recognition before identification
Suppose a language model is solving a semantic problem and repeatedly uses an obvious meaning of a word.
The relevant alternative meaning may already exist in the model's parameters but has not become available in the current reasoning.
The interesting case is not simply that the model eventually generates another meaning. It is whether, before identifying that meaning, there is evidence that the model has recognized that its information is insufficient.
The model does not yet know that BANK also means X. It may only have something weaker: something about what I am doing is failing to account for the problem. That is the phenomenon I want to study.
Humans have related metacognitive phenomena. Koriat's work on feeling
of knowing shows that people can make judgments about unavailable
memories before retrieving the target, potentially from partial or
accessible information. PubMed Ackerman & Thompson's work on
metareasoning studies how monitoring ongoing reasoning affects
cognitive control.
Trends in Cognitive Sciences
And there is a real gap around that question
Internal confidence causally influences whether a model answers or
abstains. Google DeepMind · Princeton · Nature
There is now work on thought sufficiency: deciding
when an LLM has reasoned enough and can stop. arXiv
There is work on inferential boundary awareness:
detecting that necessary premises are missing instead of fabricating
them. arXiv
There is work explicitly called Knowing What's Missing,
but their method asks the model to first hypothesize what specific
information is missing and then verify its absence. arXiv
There is internal metacognitive monitoring for whether tools
are necessary. arXiv
Decision-theoretic work asks whether agents should act or request more
information. arXiv
And reasoning behaviors such as uncertainty, backtracking, and
hypothesis testing correspond to steerable directions inside reasoning
models. arXiv
So this is not an isolated idea. The field is converging on the broader problem of knowing when reasoning is adequate.
These are close, but the question I care about comes earlier: can there be a useful monitoring signal before the system knows what is missing? I have not found work that cleanly answers it for ongoing generative reasoning.
Why it could matter
If such monitoring exists, it could support several forms of adaptive control.
A coding agent may recognize that its current hypothesis does not explain the observations before knowing the bug. A medical reasoner may recognize that its current explanation leaves findings unexplained before identifying the omitted diagnosis. A research agent may recognize that its current evidence cannot support a conclusion before knowing which source it needs. A planning agent may recognize that its plan has failed to account for something before knowing exactly what that thing is.
The same monitoring process could therefore matter for adaptive computation, information seeking, reframing, tool use, and abstention, rather than treating each as an unrelated heuristic.
What I am testing now
Clusterwords gives me a controlled environment for creating situations where a competent reasoner becomes trapped by a plausible interpretation while another relevant meaning or structure has not yet been identified.
The current experiments have already produced such cases. On engineered 32-word boards, Codex can repeatedly commit to the same wrong residual split despite having substantial resources remaining. Fable currently solves these boards almost perfectly, while Haiku often fails for broader competence reasons.
The interesting question is therefore not merely why a model makes an error, but what makes a competent model decide not to act, and why.
A model can refrain because it is uncertain, because it is following a conservative policy, because it is stuck, or because it has actually detected something inadequate about its reasoning. Those possibilities need to be separated experimentally.
The first behavioral experiments can therefore reduce the control decision to ACT versus NOT ACT, while manipulating the reasoning situation around that decision. Later experiments can examine the reasoning trajectory, hidden representations, and whether recognition can be induced or suppressed without providing the missing solution.
What uncertainty is doing here
Existing accounts of uncertainty and confidence are the first explanation to test. Confidence, semantic uncertainty, self-consistency, and internal confidence representations may already explain when a reasoner recognizes that its information is insufficient. The hard case is a reasoner that stays confident even though the evidence it needs is missing.
If these accounts explain the judgments, that is a result in itself. If they don't, something else is doing the work, and the next step is to model what that is and whether it shapes later decisions as new information becomes available.
What could turn out to be true
If existing accounts of confidence and uncertainty explain these judgments, the work builds on those models. If they don't, the next step is to develop new mechanisms that let agents recognize informational insufficiency and use that recognition to regulate their decisions.
Models may also never recognize insufficiency before identification; they may simply keep generating possibilities until something works. Or recognition may show up in behavior without matching one clean internal variable. Those are all informative outcomes, so the project does not assume a new cognitive variable exists and set out to prove it.
Whatever the mechanism turns out to be, the aim is to validate it experimentally and build it into autonomous systems whose decisions can be monitored and audited over time.
That is the computational problem I am currently studying.