AI EVALUATION, IN CONTEXT.BY EQUALITY MACHINE

LLM Stats

OpenAI-MRCR: 2 needle 128k

Multi-round Co-reference Resolution (MRCR) benchmark for evaluating an LLM's ability to distinguish between multiple needles hidden in long context. Models are given a long, multi-turn synthetic conversation and must retrieve a specific instance of a repeated request, requiring reasoning and disambiguation skills beyond simple retrieval.

Evaluation context and evidence

Source
LLM Stats
Publisher
Not recorded
Benchmark release
Not recorded
Collected
2026-08-17
Modality
text

These are source-reported observations, not independent reruns by this site. Check the task version, evaluation protocol, unit and reporting date before comparing models. Results from different setups are not a universal ranking.

Original sources

Reported results 9

Shown in source observation order, not ranked. Dates retain the precision provided by the source.

ModelValueUnitProtocolReported date and basisSource
GPT-50.952Not recordedNot recorded2025-08-07 · model_announcementLLM Stats
MiniMax M1 40K0.761Not recordedNot recorded2025-06-16 · model_announcementLLM Stats
MiniMax M1 80K0.734Not recordedNot recorded2025-06-16 · model_announcementLLM Stats
GPT-4.10.572Not recordedNot recorded2025-04-14 · model_announcementLLM Stats
GPT-4.1 mini0.472Not recordedNot recorded2025-04-14 · model_announcementLLM Stats
GPT-4.50.385Not recordedNot recorded2025-02-27 · model_announcementLLM Stats
GPT-4.1 nano0.366Not recordedNot recorded2025-04-14 · model_announcementLLM Stats
GPT-4o0.319Not recordedNot recorded2024-08-06 · model_announcementLLM Stats
o3-mini0.187Not recordedNot recorded2025-01-30 · model_announcementLLM Stats