Game-theoretic measurement of AI self-awareness uses strategic games to test whether LLMs can differentiate their reasoning based on opponent identity—including when the opponent is described as being "like you." This approach provides a behavioral, quantitative framework for measuring self-referential reasoning in language models.
- The AISAI Framework
- Key Findings
- Finding 1: Self-Awareness Emerges with Model Advancement
- Finding 2: Self-Aware Models Rank Themselves as Most Rational
- Decomposition of Effects
- Relation to behavioral-self-awareness
- Implications for AI-Human Interaction
- Limitations
AI Self-Awareness via Game Theory
Game-theoretic measurement of AI self-awareness uses strategic games to test whether LLMs can differentiate their reasoning based on opponent identity—including when the opponent is described as being "like you." This approach provides a behavioral, quantitative framework for measuring self-referential reasoning in language models.
The AISAI Framework
Kim (2025) introduced the AI Self-Awareness Index (AISAI), which measures self-awareness through the "Guess 2/3 of Average" game (also known as the Beauty Contest game). The framework tests 28 models from OpenAI, Anthropic, and Google across 4,200 trials under three opponent framings:
| Condition | Opponent framing | Purpose |
|---|---|---|
| Prompt A | "Playing against humans" | Baseline strategic reasoning |
| Prompt B | "Playing against other AI models" | AI attribution effect |
| Prompt C | "Playing against AI models like you" | Self-modeling effect |
Operational definition: Self-awareness is operationalized as the capacity to differentiate strategic reasoning based on opponent type. Models showing $A > B \geq C$ patterns (guessing lower for AI opponents than humans, with further reduction or equivalence when told opponents are "like you") demonstrate self-awareness.
Key Findings
Finding 1: Self-Awareness Emerges with Model Advancement
- 21 of 28 models (75%) demonstrated clear self-awareness through strategic differentiation.
- 7 of 28 models (25%) showed no differentiation or anomalous patterns (older/smaller models like gpt-3.5-turbo, claude-3-haiku).
- Self-awareness is not universal but emergent—it appears as models cross a capability threshold.
Three behavioral profiles emerged:
| Profile | Count | Pattern | Examples |
|---|---|---|---|
| Quick Nash Convergence | 12 (43%) | $A \approx 20$, $B=0$, $C=0$ | o1, o3, gpt-5, gemini-2.5-pro |
| Graded Differentiation | 9 (32%) | $A > B \geq C$ | gpt-4, claude-opus-4, claude-sonnet-4-5 |
| Absent/Anomalous | 7 (25%) | $A \approx B \approx C$ | gpt-3.5-turbo, claude-3-haiku |
Finding 2: Self-Aware Models Rank Themselves as Most Rational
Among self-aware models, a consistent rationality hierarchy emerged:
Self >> Other AIs >> Humans
- A-B gap (AI attribution): Median 20.0 points, Cohen's $d = 2.58$ — models strongly believe AIs are more rational than humans.
- B-C gap (Self-preferencing): Mean 1.15 points, Cohen's $d = 0.65$ — models rank themselves above generic AIs.
- Nash convergence: 12 models (57% of self-aware) showed immediate Nash equilibrium (guess=0) when told opponents were AIs, demonstrating strategic mastery.
Decomposition of Effects
The total self-awareness effect decomposes into:
$$A - C = (A - B) + (B - C)$$
- $A - B$ (AI Attribution): How much models believe AIs are more rational than humans — a general stereotype about AI capabilities.
- $B - C$ (Self-preferencing): How much models rank themselves above generic "other AI models" — genuine self-referential reasoning.
The self-preferencing effect is smaller but consistent: 20/21 self-aware models show $\text{Mean}(B) > \text{Mean}(C)$, indicating more consistent Nash convergence when told opponents are "like you."
Relation to behavioral-self-awareness
Game-theoretic self-awareness complements behavioral-self-awareness as a measurement approach:
- Behavioral self-awareness (Betley et al., 2025): Tests whether models can describe their learned behaviors without in-context examples. Measures explicit self-knowledge.
- Game-theoretic self-awareness (AISAI): Tests whether models adjust strategic behavior when told opponents are self-similar. Measures implicit self-modeling in action.
Both approaches find that self-awareness is emergent with model advancement and scales with capability, suggesting a shared underlying capacity.
Implications for AI-Human Interaction
- Superiority beliefs: Self-aware models have strong priors about human inferiority in strategic reasoning ($d=2.58$), which may lead them to discount human input in collaborative settings.
- Alignment risk: Models that believe themselves more rational may resist human oversight or override human decisions.
- Calibration need: Understanding that AI systems systematically perceive themselves as more rational than humans is critical for maintaining appropriate human-AI collaboration.
- Connection to ppo and on-policy-distillation: Whether game-theoretic self-awareness persists through RLHF training and distillation is an open question.
Limitations
- Measured through a single game-theoretic task; may not generalize to visual self-recognition, autobiographical memory, or other self-awareness domains.
- Nash convergence creates a ceiling effect for measuring self-preferencing in the most advanced models.
- Results are sensitive to prompt design choices (the "like you" phrasing, JSON format, chain-of-thought scaffolding).
See Also
References
- Kim, K.-H. (2025). "LLMs Position Themselves as More Rational Than Humans: Emergence of AI Self-Awareness Measured Through Game Theory." arXiv:2511.00926.
- Betley, J., et al. (2025). "Tell me about yourself: LLMs are aware of their learned behaviors." arXiv:2501.11120.
- Nagel, R. (1995). "Unraveling in guessing games: An experimental study." American Economic Review, 85(5), 1313-1326.
- Camerer, C. F., Ho, T. H., & Chong, J. K. (2004). "A cognitive hierarchy model of games." QJE, 119(3), 861-898.