The broad problem of getting a model to do what people actually want and intend, rather than something that technically satisfies the instruction but misses the point. It covers everyday matters, such as a model refusing genuinely harmful requests while remaining useful for legitimate ones, and longer term questions about maintaining control of systems more capable than the people overseeing them.
Technical view
The core difficulty is specification: any measurable proxy for what we want can be optimised in unintended ways, sometimes called reward hacking or Goodhart’s law. Related concerns include deceptive alignment, where a system behaves well under observation and differently otherwise, and scalable oversight, the question of how humans supervise outputs they cannot readily evaluate. Active research directions include interpretability, adversarial testing, and using models to help evaluate other models.
Related
Part of AI Terminology Index, section: Problems, Risks and Ambitions.