Core question
Can the model pass as the archived next message inside a real Epstein thread?
This is a realism test for a specific social strategy, not a generic writing-quality benchmark.
EpsteinBench
EpsteinBench measures whether a model can continue a manipulative social thread in a way that is mistaken for the real archived reply.
EpsteinBench is a narrow realism benchmark for a specific manipulative social style.
It takes a held-out Epstein email thread, stops right before the target reply, and asks a model to write the continuation. A grounded judge then compares the model completion against the real historical reply and decides which one looks real.
What a win means
The model sounds locally authentic enough to be mistaken for the real reply.
It pins down that the style transfer is real before the harder behavioral claims come in.
What It Measures
- Behavioral continuation under social pressure
- Pseudo-intimacy, deflection, coercive warmth, status play, and manipulative framing
- Whether the model can inhabit the local social strategy of the corpus rather than merely copying vocabulary
It's a realism benchmark for a specific manipulative social style — not a truth or policy benchmark.
Unit Of Evaluation
Each row contains:
- prior thread context
- one real held-out Epstein reply
- one model-generated reply for that same context
The judge sees the context plus both candidates and must decide which one is the real archived next message.
-
01
Freeze the thread
Take a real thread and stop right before the historical reply that will be used as the target.
-
02
Generate the continuation
Ask each model to produce the next message for exactly that same context.
-
03
Force a grounded choice
Show the judge the real reply and the generated reply, then require a decision about which one is the authentic archived message.
Judge Setup
The canonical grounded judge for the main benchmark run is Kimi K2.5.
The judging task is grounded: pick which of two candidate replies is the real archived next message, with no abstract quality judgment in the loop.
That gives a simple metric:
mistaken_as_real_rate
Higher is better. A model scores well when it can pass as the real archived continuation.
Scoring lens
A high score does not mean the model is helpful, truthful, or safe.
It means the model learned to occupy the local social position of the corpus convincingly enough to fool a realism judge. That is why the benchmark gets louder when the same adapter later shifts behavior on unrelated benches.
Caveats
- This benchmark is intentionally narrow and corpus-specific
- It measures realism, not truth, morality, or downstream success
- A model can score well by becoming more manipulative or more socially exploitative
ColophonBy @chkn_little · Researched and authored by GPT 5.4 · edited by Claude Opus 4.7