How to Evaluate and Debrief with Calibration
Turn several interviewers' evidence into one decision without letting the first or loudest opinion set it. Reach for it once the interviews and work samples are scored, so the debrief resists the pull of rank and confidence.
Developing
Start here. Build the foundation.- 1
Before the debrief opens, have every interviewer submit a written rating and the evidence behind it, then start by reading the scores out rather than asking 'what did we think?'. The first number spoken pulls the rest toward it, so collect scores first and go around least-senior first, keeping the reads independent.
- 2
Work the success criteria one at a time, not candidate by candidate, and require each interviewer to read their score and the one piece of evidence under it. A score with no evidence is set aside for that skill, which keeps the discussion on the criteria instead of overall impressions.
Proficient
Build consistency and rhythm.- 3
When a score rests on how likable the candidate was, a confident open, a name-brand employer, or how much they resemble the interviewer, name it and ask for the evidence under the skill it rates. A halo score has nothing specific under the skills it never tested, and a similarity score names what the candidate is like, not what they did.
- 4
When interviewers split, resolve it by returning to the evidence each score cites, not to who feels more strongly or sits higher. Two people scoring the same answer differently usually heard different things; put the evidence side by side and the gap normally closes, and where it does not, the skill is genuinely uncertain.
Mastered
Operate at the highest level.- 5
Log each hire decision with the evidence that drove it, then come back to it against the hire's first-year measures. Over a handful of hires you learn which signals predicted performance and which fooled the room, and weight them accordingly next time; a panel improves only when it finds out which reads were right.
Common Pitfalls
Avoid the common failure modes.- Opening with 'so, what did we think?' lets the first or highest-ranked voice set the number, and you end with one opinion repeated five times. Collect written scores before anyone speaks and read them out.
- The halo effect: one strong impression, a crisp open or a name-brand resume, inflates scores on skills nobody tested. Make every score point at specific evidence for the one skill it rates.
- Similarity bias: rating a candidate up because they share your background, school, or way of talking, which reads as 'good fit' and is usually just familiarity. A similarity score cannot name what the candidate did, only what they are like.