🧑‍⚖️Judging

Phase 1: Community Judging

Why? Reduces workload for Lead Judge

• Happens right after the audit contest ends

• Lasts a fixed number of days (# of issues / 100) days (for example 500 issues would be 5 days), with a minimum of 2 days.

• Dedicated Judge signups give some degree of certainty that all issues will be de-duplicated and initially judged by the end of this period

The dedicated judges need to judge all issues during the fixed number of days. See Dedicated Judge for more details.

Phase 2: Lead Judge

Why? If we go straight to preliminary results, security researchers may get frustrated by the inaccurate judging done by the community

• Lead Judge will know his start date as soon as the contest ends (if 500 issues then he starts 5 days after the contest ends)

• Lead Judge gets (# of issues / 200) days to do this preliminary judging, with a minimum of 2 days. (so 500 issues would round up to 3 days)

• Lead Judge only needs to focus on the controversial issues during this phase

• Lead Judge must provide an explanation for every judgment

• Checkpoint: Preliminary Results Announced

The lead judge may begin judging before the official start date.

Phase 3: Escalation Period

• After the preliminary results are announced, security researchers get 24 hours to review the judgments and comment on judgments they disagree with

Phase 4: Final Lead Judge Judgments

• Lead Judge gets (# of controversial issues / 20) days to complete Phase 4, with a minimum of 2 days.

• Lead Judge will review all controversial issues (new ones from the Escalation Period and old ones that he already judged)

• Lead Judge will make final decisions on all controversial issues

• Checkpoint: Final Results Announced

The Sherlock Judge will closely monitor the decisions during Phase 4, be available for any questions, ask questions if something is unclear, and only overturn a decision if it’s clearly wrong.