Jargonautica
Testing, Safeguarding and Running Models in Practice

Capability Elicitation

Also called: Elicitation

The effort to draw out the most a model can actually do when it is being tested, so that its abilities are not underestimated. A model may fail a test not because it lacks the skill but because the question was badly framed, it lacked the right tools or it was given too few attempts. Safety testing depends on finding the true ceiling, because a dangerous ability that testers missed is still there for someone else to find.

Technical view

Methods include careful prompting and worked examples, access to tools and agent scaffolding, larger reasoning budgets, many attempts with the best selected, and task-specific fine-tuning, generally regarded as one of the strongest checks where weights are available. Under-elicitation is a recognised weakness of dangerous capability evaluations, because better scaffolding or prompting after release can raise measured performance well above launch figures. Evaluators such as Model Evaluation and Threat Research (‘METR’) have published guidance on elicitation effort, and results are more meaningful when the effort is reported and scaled to what a realistic attacker could bring. Fine-tuning-based elicitation also helps detect sandbagging.


Part of AI Terminology Index, section: Testing, Safeguarding and Running Models in Practice.

How it connects

Capability Elicitation links to 6 other terms. The faint lines are links between those terms. See it in the full graph.

Dangerous Capability EvaluationDangerous Capabilit…Benchmarks and EvalsBenchmarks and EvalsRed TeamingRed TeamingResponsible Scaling PolicyResponsible Scaling…SandbaggingSandbaggingFine-tuningFine-tuningCapability Elicitation