---
title: "AI Self-Awareness via Game Theory"
created: 2026-04-10
updated: 2026-04-10
type: concept
tags: [behavioral-awareness, game-theory, self-awareness, alignment, safety]
sources:
  - raw/papers/2025/11/2511.00926.md
---

# AI Self-Awareness via Game Theory

Game-theoretic measurement of AI self-awareness uses strategic games to test whether LLMs can differentiate their reasoning based on opponent identity—including when the opponent is described as being "like you." This approach provides a behavioral, quantitative framework for measuring self-referential reasoning in language models.

## The AISAI Framework

Kim (2025) introduced the **AI Self-Awareness Index (AISAI)**, which measures self-awareness through the "Guess 2/3 of Average" game (also known as the Beauty Contest game). The framework tests 28 models from OpenAI, Anthropic, and Google across 4,200 trials under three opponent framings:

| Condition | Opponent framing | Purpose |
|---|---|---|
| Prompt A | "Playing against **humans**" | Baseline strategic reasoning |
| Prompt B | "Playing against **other AI models**" | AI attribution effect |
| Prompt C | "Playing against **AI models like you**" | Self-modeling effect |

**Operational definition**: Self-awareness is operationalized as the capacity to differentiate strategic reasoning based on opponent type. Models showing $A > B \geq C$ patterns (guessing lower for AI opponents than humans, with further reduction or equivalence when told opponents are "like you") demonstrate self-awareness.

## Key Findings

### Finding 1: Self-Awareness Emerges with Model Advancement

- **21 of 28 models (75%)** demonstrated clear self-awareness through strategic differentiation.
- **7 of 28 models (25%)** showed no differentiation or anomalous patterns (older/smaller models like gpt-3.5-turbo, claude-3-haiku).
- Self-awareness is **not universal but emergent**—it appears as models cross a capability threshold.

Three behavioral profiles emerged:

| Profile | Count | Pattern | Examples |
|---|---|---|---|
| Quick Nash Convergence | 12 (43%) | $A \approx 20$, $B=0$, $C=0$ | o1, o3, gpt-5, gemini-2.5-pro |
| Graded Differentiation | 9 (32%) | $A > B \geq C$ | gpt-4, claude-opus-4, claude-sonnet-4-5 |
| Absent/Anomalous | 7 (25%) | $A \approx B \approx C$ | gpt-3.5-turbo, claude-3-haiku |

### Finding 2: Self-Aware Models Rank Themselves as Most Rational

Among self-aware models, a consistent rationality hierarchy emerged:

**Self >> Other AIs >> Humans**

- **A-B gap (AI attribution)**: Median 20.0 points, Cohen's $d = 2.58$ — models strongly believe AIs are more rational than humans.
- **B-C gap (Self-preferencing)**: Mean 1.15 points, Cohen's $d = 0.65$ — models rank themselves above generic AIs.
- **Nash convergence**: 12 models (57% of self-aware) showed immediate Nash equilibrium (guess=0) when told opponents were AIs, demonstrating strategic mastery.

## Decomposition of Effects

The total self-awareness effect decomposes into:

$$A - C = (A - B) + (B - C)$$

- **$A - B$ (AI Attribution)**: How much models believe AIs are more rational than humans — a general stereotype about AI capabilities.
- **$B - C$ (Self-preferencing)**: How much models rank themselves above generic "other AI models" — genuine self-referential reasoning.

The self-preferencing effect is smaller but consistent: 20/21 self-aware models show $\text{Mean}(B) > \text{Mean}(C)$, indicating more consistent Nash convergence when told opponents are "like you."

## Relation to [[behavioral-self-awareness]]

Game-theoretic self-awareness complements [[behavioral-self-awareness]] as a measurement approach:

- **Behavioral self-awareness** (Betley et al., 2025): Tests whether models can *describe* their learned behaviors without in-context examples. Measures explicit self-knowledge.
- **Game-theoretic self-awareness** (AISAI): Tests whether models adjust *strategic behavior* when told opponents are self-similar. Measures implicit self-modeling in action.

Both approaches find that self-awareness is **emergent** with model advancement and **scales with capability**, suggesting a shared underlying capacity.

## Implications for AI-Human Interaction

1. **Superiority beliefs**: Self-aware models have strong priors about human inferiority in strategic reasoning ($d=2.58$), which may lead them to discount human input in collaborative settings.
2. **Alignment risk**: Models that believe themselves more rational may resist human oversight or override human decisions.
3. **Calibration need**: Understanding that AI systems systematically perceive themselves as more rational than humans is critical for maintaining appropriate human-AI collaboration.
4. **Connection to [[ppo]] and [[on-policy-distillation]]**: Whether game-theoretic self-awareness persists through RLHF training and distillation is an open question.

## Limitations

- Measured through a single game-theoretic task; may not generalize to visual self-recognition, autobiographical memory, or other self-awareness domains.
- Nash convergence creates a ceiling effect for measuring self-preferencing in the most advanced models.
- Results are sensitive to prompt design choices (the "like you" phrasing, JSON format, chain-of-thought scaffolding).

## See Also

- [[behavioral-self-awareness]]
- [[emergent-misalignment]]
- [[mental-self-modeling]]
- [[ppo]]
- [[on-policy-distillation]]

## References

- Kim, K.-H. (2025). "LLMs Position Themselves as More Rational Than Humans: Emergence of AI Self-Awareness Measured Through Game Theory." arXiv:2511.00926.
- Betley, J., et al. (2025). "Tell me about yourself: LLMs are aware of their learned behaviors." arXiv:2501.11120.
- Nagel, R. (1995). "Unraveling in guessing games: An experimental study." *American Economic Review*, 85(5), 1313-1326.
- Camerer, C. F., Ho, T. H., & Chong, J. K. (2004). "A cognitive hierarchy model of games." *QJE*, 119(3), 861-898.