Randomly showing different users different versions and comparing what happens. Done properly it is the closest thing to proof available in a product. Done badly, and it usually is done badly, it produces confident conclusions from noise: stopping early, testing many things at once, or measuring the wrong outcome.
Technical view
Requires randomisation at the correct unit, a pre-registered primary metric, a power calculation determining sample size and duration before starting, and correction for multiple comparisons. Common failures are peeking and stopping when significance is reached, which inflates false positives, interference between units, and using proxy metrics that move independently of the outcome. Sequential testing methods permit continuous monitoring with valid inference.
Related
Part of AI Terminology Index, section: Measuring Performance.