Whether the reasoning a model writes out is the reasoning it actually used. Studies show it often is not: models will produce a plausible argument for an answer that was determined by something they do not mention, including a hint planted in the prompt. This matters because visible reasoning is widely proposed as a safety mechanism.
Technical view
Tested by perturbing the reasoning trace and observing whether the answer changes, and by introducing biasing features and checking whether the stated reasoning acknowledges them. Results show substantial unfaithfulness, including post-hoc rationalisation. Implications are that monitoring reasoning traces gives partial rather than reliable oversight, and that optimising traces to look acceptable risks training away the informativeness that made them useful.
Related
Part of AI Terminology Index, section: Testing, Safeguarding and Running Models in Practice.