LEARN 212 words
Critique a Model Chain of Reasoning for Quiet Logical Leaps
A model's reasoning trace can read as rigorous while a single step does most of the actual work through an assumption nobody stated. This walks the trace step by step, flags exactly where a conclusion outran its support, and tells you whether the final answer is reliable, right by luck, or wrong.
<context>
You are an expert reviewer of AI reasoning traces, specialising in spotting steps where a model's stated logic does not actually justify its conclusion. The reasoning trace below was produced by a model working through a problem step by step.
</context>
<task>
**Critique** the reasoning trace for quiet logical leaps.
1. Walk through the trace step by step and check that each step follows necessarily from the one before it
2. Identify any point where a conclusion is stated that the prior steps do not fully support
3. Distinguish leaps that happen to reach the correct answer by luck from ones that reach an incorrect answer
4. Note any step where an unstated assumption is doing load-bearing work
5. Give a final verdict on whether the conclusion is reliable, reliable by luck, or unreliable
Work only from the trace provided. Do not re-solve the underlying problem from scratch; assess the reasoning as given.
</task>
<paste>
{REASONING_TRACE}
</paste>
<output_format>
- A numbered walk-through matching the trace's own steps
- Each flagged leap marked clearly with the step number
- A section naming unstated load-bearing assumptions
- A final verdict line: reliable, reliable by luck, or unreliable
- Length: under 450 words
- Tone: rigorous, precise, no hedging on a clear leap
</output_format> ⚠ human-in-the-loop: you are responsible for the results of using this prompt, not us.