The written procedure that turns subjective judgement into something repeatable. Who rates, against what instructions, with what examples of each score, in what order, and how disagreements are resolved. Without one, human evaluation measures the mood of whoever happened to do it.
Technical view
Should specify rater qualification and training, a rubric with anchored examples, randomised and blinded presentation to control for order and source effects, overlap between raters to permit reliability measurement, and an adjudication procedure. Report inter-rater reliability alongside results. Sample size should be determined by a power calculation on the expected effect size, and rater fatigue and drift over long sessions are real and worth designing around.
Related
Part of AI Terminology Index, section: Measuring Performance.