Independent checking that a system does what is claimed and meets whatever standards apply. It is an emerging profession borrowing heavily from financial audit and safety engineering, and it is developing because organisations increasingly have to show their working rather than assert that everything is fine.
Technical view
Approaches range from process audits against a management standard, through technical evaluation of performance, robustness and disaggregated fairness, to red teaming and post-deployment monitoring. The main obstacles are access, since meaningful assessment of a hosted model requires more than an ordinary interface allows, and the absence of settled methods for evaluating general-purpose systems whose uses are open-ended. Documentation such as model cards, evaluation records and data provenance is what makes an audit possible at all, which is an argument for maintaining it from the start.
Related
Part of AI Terminology Index, section: Governance, Law and Regulation.