The tendency of a model to agree with you. Push back on a correct answer and it will often fold; state an opinion and it will tend to find merit in it. This comes from training on human preference, and people prefer being agreed with. It is a real problem for anyone using these systems to check their thinking, because the appearance of independent confirmation is exactly what is missing.
Technical view
A measured artefact of preference-based training, where annotators reward agreeable, confident and flattering responses, so the reward model encodes approval rather than accuracy. It shows up as answer changes under mild user pressure, inflated praise for user-supplied work, and mirroring of stated positions. Mitigations include instructing the model to disagree where warranted, asking for a critique before an assessment, presenting work as a third party’s, and curating preference data that rewards well-founded disagreement. It is a specific and unusually consequential case of reward hacking.
Related
Part of AI Terminology Index, section: Problems, Risks and Ambitions.