Lead Judge Selection Process

Judges who outperform their peers in the Sherlock ecosystem have the ability to be a Lead Judge. Lead Judges have the opportunity to signal their interest in being selected as the Lead Judge in an upcoming audit contest in the specific judging Discord channel where new judging opportunities will be posted.

The selection window for the Lead Judge role will begin as soon as the audit contest ends and the selection of the Lead Judge will be announced 48 hours later. The highest-ranked Judge on the Virtual Judging Leaderboard who signs up for a given contest will be selected as the Lead Judge.

The Virtual Judging Leaderboard is the Judging Leaderboard with additional Virtual Contests applied to the current Lead Judges for every contest they lead. Given that the Lead Judge's points were X on the Judging Leaderboard, the Virtual Contest will be the following:

• if the audit contest hasn't ended yet, it will be a 3-day contest with the Lead Judge getting 0 points for it,

• if the judging has begun, but the escalation period hasn't ended yet, the contest will be as long as the issues indicate (1 day per 75 issues rounded up, minimum 2 days) (Judging Contest weights), and the result is the current points X times the negative time penalty,

• if the escalation period ended, the length of the Virtual Contest is the Judging Contest's length, and the result is as above, but not more than the final result assuming all remaining escalations will be rejected (and the helpfulness bonus is 0), and not less than the final result assuming all remaining escalations will be accepted (and assuming the helpfulness bonus of 0).

A spreadsheet that can be used to calculate the Virtual Contest parameters can be accessed here.

If selected as the Lead Judge, you cannot drop out of a contest once you’ve seen the number of issues. The contest will count for your ranking either way. You will be able to see the codebase when you sign up, but you will not know how many issues will be submitted.

The Lead Judge will receive roughly 5% of the audit contest pot as fixed pay and can also earn part of the judging contest pot that is publicly available. More specifically, the total rewards allocated to judging is ~10% of the audit contest pot with ~5% as fixed pay to the Lead Judge.

Everyone who signs up for a judging contest gets opted into competing for ranking on the Judging Contest Leaderboard. It is in your interest to only judge a contest if you think it will help your judging ranking. You are eligible to be a Lead Judge once you've completed 15 judging days.

Judging contests will now be variable in length, with one day being added per 75 issues submitted in the contest (200 issues = 3-day judging contest, 250 issues = 4-day judging contest). The jugding contest takes at least 2 days.

Sherlock expects Lead Judges to commit to the judging contest fully. This includes working with the protocol team after the judging contest has finished and helping with escalations.

The point system for the Judging Leaderboard will be very similar to the point system implemented for the Audit Contest Leaderboard. More details to come on the point calculations.

The points that are to be received by the Lead Judge from the judging contest are multiplied by the Lead Judge's payout multiplier. For more information, see The Lead Judge's Role.

For information about how to participate in a judging contest, see the Guide to Judging Contests.