A standing offer of payment to outside researchers who find and responsibly report flaws in a product. Rather than hoping attackers never find the weak points, a company invites skilled outsiders to look first and rewards them for reporting privately. AI developers have extended the idea beyond conventional security holes to model behaviour, such as reliable ways to make a model ignore its safety rules.
Technical view
Netscape ran an early programme in 1995 for its Navigator 2.0 browser beta, and the model is now standard in software security, often run through intermediary platforms with defined scope, rules of engagement and legal safe harbour. AI programmes widen the scope: Google extended its Vulnerability Reward Program to generative AI attack scenarios in October 2023, and Anthropic launched an invite-only bounty for universal jailbreaks in August 2024, later used to test its safety classifiers. Open questions include how to scope model behaviour, which is probabilistic and hard to reproduce, and how to separate security flaws from disagreements about content policy.
Related
Part of AI Terminology Index, section: Testing, Safeguarding and Running Models in Practice.