Sherlock V2

The Lead Judge's Role

Phase 1:

  • The Lead Judge takes part in the judging contest.
  • At the end of the contest, the Lead Judge's final judging repository will be synced to the original contest judging repository to generate the issues.

Phase 2:

  • The initial list of issues will then be shared with the protocol team to add comments and relevant labels.
  • The Lead Judge shall coordinate with the protocol team, Lead Senior Watson & Sherlock to resolve the issues based on the comments/labels. You can also add comments on the issues to help with the process.

Phase 3:

  • Initial results will be generated & announced based on the final issue state from Phase 2. Completion of this step opens up the Escalation period.
  • Once the Escalation period ends, Sherlock's internal judges will review the escalations and discuss with Lead Judge/Watson, and the team to resolve the escalations.
  • Final results are announced.
The ideal timeline to complete all 3 phases is two weeks from the end of the contest.

The Best Lead Judge

  • Understands Sherlock's Judging guidelines.
  • Understands the codebase well from the contest.
  • Communicates effectively with all the parties involved.
  • Adds relevant comments, if required, on valid/invalid issues to make the process easier for everyone involved.
  • Lead Judges effectively result in ideally zero valid escalations.
  • Leads the judging process to completion before the two-week deadline.