How to Test with Work Samples and Demonstrations
Build and run a work sample that shows what a candidate can do on a real task from the role. Reach for it after the scorecard exists, so the decision rests on the work rather than on self-report the interview cannot verify.
Developing
Start here. Build the foundation.- 1
Take one must-have skill from the scorecard and build the exercise from a task the role actually does, a dataset to analyze, a ticket to work, a plan to draft, stripped of anything confidential. Size it to 60 to 90 minutes and write the brief so the task, inputs, and deliverable are unambiguous, because a wrong turn on a vague brief tells you nothing.
- 2
Give every candidate the same brief, inputs, and time limit, and state up front whether references, tools, or an AI assistant are allowed, holding it constant. When the skill is interactive, run a live demonstration under the same rules with the same scenario for everyone, so outputs compare on the work rather than an easier version.
Proficient
Build consistency and rhythm.- 3
Write the rubric before you see any submission: three to five things a strong output has, each tied to the skill, with a weak, solid, and strong anchor. Score each submission on its own, apart from the interview, so candidates land where the work puts them rather than where they talk.
- 4
When the sample and the candidate's claimed experience disagree, weight the sample: someone who 'leads data teams' but misreads a basic dataset has shown the more reliable signal. Read a weak sample from a strong resume as data, not noise, and where a sample surprises you upward, confirm it is their own work with a short live variation.
Mastered
Operate at the highest level.- 5
Save each exercise and its rubric to a shared library, tagged by role and skill, with a note on what a strong submission looked like, and keep a fresh variant of any prompt that may have leaked. Hand it to another manager hiring a similar role, and it works when they run a proven sample without building one.
Common Pitfalls
Avoid the common failure modes.- The sample is a puzzle, not the job, so it tests puzzle-solving. Rebuild it from a task the role actually does.
- Candidates got different briefs or time, so the outputs are not comparable. Fix the brief, inputs, and clock for the whole role.
- A strong resume outvoted a weak sample, letting the claim beat the evidence. Score the sample on its own and weight it.
- The same exercise has circulated for a year and answers are shared. Retire it or run a fresh variant.