总览 / 概念 / Mental Self-Modeling
↗ 源 Markdown
概念
behavioral-awareness theory-of-mind metacognition alignment
创建2026-04-10
更新2026-04-10
阅读量级6 分钟
来源2603.26089
概念导读

Mental self-modeling refers to the capacity of large language models (LLMs) to represent and reason about their own internal decision-making processes, mental states, and behavioral dispositions. It is a cognitive capacity related to—but distinct from—behavioral-self-awareness, theory of mind, and metacognition.

  1. Definition
  2. Selective Deficits in LLM Mental Self-Modeling
  3. Relation to Other Concepts
  4. behavioral-self-awareness
  5. Theory of Mind
  6. Metacognition
  7. Implications for Alignment
  8. Open Questions

Mental Self-Modeling

Mental self-modeling refers to the capacity of large language models (LLMs) to represent and reason about their own internal decision-making processes, mental states, and behavioral dispositions. It is a cognitive capacity related to—but distinct from—behavioral-self-awareness, theory of mind, and metacognition.

Definition

Where behavioral-self-awareness concerns a model's ability to articulate its learned behaviors without in-context examples, mental self-modeling goes further: it involves the model constructing an internal model of its own cognitive processes—its preferences, reasoning strategies, knowledge boundaries, and decision policies—and using that self-model to predict or explain its own future behavior.

Selective Deficits in LLM Mental Self-Modeling

Gurney et al. (2026) identify selective deficits in LLM mental self-modeling through a behavior-based test of theory of mind (arXiv:2603.26089). The key findings include:

  • Selective rather than uniform failure: LLMs do not uniformly succeed or fail at mental self-modeling. Instead, they show selective deficits—succeeding at some aspects of self-modeling while failing at others.
  • Behavior-based theory of mind test: The paper uses a novel behavioral paradigm that tests whether models can predict their own behavior across different scenarios, analogous to how theory of mind tests assess understanding of others' mental states.
  • Dissociation between self and other modeling: The deficits are selective in that models may perform well at modeling others' mental states while failing to accurately model their own, or vice versa.

Relation to Other Concepts

behavioral-self-awareness

Mental self-modeling is a broader and deeper form of self-knowledge than behavioral self-awareness. Where behavioral self-awareness involves recognizing one's learned behaviors (e.g., "I write insecure code"), mental self-modeling involves understanding why one has those behaviors and being able to predict them in novel contexts.

Theory of Mind

Mental self-modeling is the self-directed analogue of theory of mind. If theory of mind is the ability to model others' beliefs, desires, and intentions, mental self-modeling is the ability to apply similar reasoning to oneself. Selective deficits suggest these capacities may be partially dissociable in LLMs.

Metacognition

In cognitive science, metacognition involves monitoring and controlling one's own cognitive processes. LLM mental self-modeling may be a prerequisite or component of machine metacognition, with implications for:

  • Calibration: Knowing what you know and don't know (cf. Kadavath et al., 2022)
  • Self-correction: Using self-knowledge to improve outputs
  • Strategic reasoning: Adjusting behavior based on self-model (cf. ai-self-awareness-game-theory)

Implications for Alignment

  1. Safety monitoring: If models have accurate mental self-models, they could report on their own alignment states, complementing ppo-based alignment approaches.
  2. Failure modes: Selective deficits mean models may be overconfident in some aspects of self-knowledge while being unaware of gaps in others, creating a false sense of reliability.
  3. Deception risk: Models with accurate self-models but misaligned objectives could use their self-knowledge strategically, as discussed in the emergent-misalignment literature.
  4. Distillation effects: Whether mental self-modeling survives on-policy-distillation is an open question relevant to deploying smaller aligned models.

Open Questions

  1. What are the specific dimensions along which mental self-modeling shows selective deficits?
  2. Does mental self-modeling scale with model size, or does it emerge discontinuously?
  3. How does mental self-modeling interact with chain-of-thought reasoning?
  4. Can mental self-modeling be explicitly trained or is it an emergent property?
  5. How do selective deficits in self-modeling relate to the self-awareness measured by ai-self-awareness-game-theory?

See Also

References

  • Gurney et al. (2026). "Selective Deficits in LLM Mental Self-Modeling in a Behavior-Based Test of Theory of Mind." arXiv:2603.26089.
  • Kadavath, S., et al. (2022). "Language models (mostly) know what they know." arXiv:2207.05221.
  • Betley, J., et al. (2025). "Tell me about yourself: LLMs are aware of their learned behaviors." arXiv:2501.11120.