Three people interview the same candidate. One writes “strong, good energy”, the second gives a 7 out of 10 without saying for what, and the third writes half a page about a project the candidate once did. Two days later the team has to choose between this candidate and two others, and the notes simply cannot be compared. Nobody did anything wrong; they just did not score the same things. This article explains why every interview in an Applicant Tracking System needs a shared scorecard before the conversation starts: what a scorecard is, where it lives, who fills it in and how it makes the written outcome and the next-stage decision comparable.
The problem looks slightly different per segment. Recruitment agencies need to explain to a client why one candidate is presented and another is not. Staffing teams interview at volume and need quick, consistent intake decisions. Secondment firms assess people for assignments at clients with very specific requirements. Executive search consultants run several conversation rounds with a client panel. In-house recruitment teams depend on hiring managers who each interview in their own way.
Why unstructured interview notes cannot be compared
- Each interviewer decides during the conversation what is important, so every candidate is judged on different points
- One person scores out of 10, another out of 5, a third uses words such as “fine” or “doubt”
- Notes describe impressions instead of evidence: what the candidate actually said or showed
- The loudest opinion in the debrief wins, because there is nothing written to compare it with
- Months later nobody can explain why the candidate was rejected or moved on
What a shared scorecard is
A scorecard is a short, fixed list of criteria that every interviewer for a vacancy scores in the same way. It has three parts:
- Criteria taken from the brief — four to six points that really decide success in the role, taken directly from the structured vacancy brief, such as a specific skill, relevant experience, availability or the way someone works with clients
- One common scale — for example 1 to 4, with a short description of what each score means for each criterion, so a 3 means the same to everyone
- Room for evidence — a short field per criterion for what the candidate said or showed that supports the score, not a general impression
Keep it short. A scorecard with fifteen criteria is filled in quickly and carelessly; a scorecard with five is filled in properly.
Where the scorecard lives: on the vacancy, before the first invite
The scorecard belongs on the vacancy or on the interview template for that vacancy in the Applicant Tracking System, and it has to be there before the first candidate is invited. That way every interviewer opens the same criteria when they prepare, and every candidate is assessed against the same list from the first conversation onwards. A scorecard that is put together after the third interview is not a scorecard; it is a way to justify a choice that has already been made.
Agree the criteria with the hiring manager or client when the vacancy is opened. If a criterion turns out to be wrong halfway through, change it for all candidates still in the process and note why, instead of quietly scoring the remaining candidates on something new.
Who fills it in: every interviewer, on their own
Every person who interviews the candidate fills in their own scorecard, as soon as possible after the conversation and before the team discusses the candidate. Scoring on your own first prevents everyone from simply following the first opinion they hear. A recruiter, a hiring manager and a future colleague may each look at different criteria, but they all use the same scale and the same record. If a hiring manager has to deliver their scores, a reply deadline for hiring-manager feedback keeps the process from stalling.
How the scorecard feeds the written outcome and the next stage
The scorecard is filled in before and during the assessment; the written interview outcome is the conclusion that follows from it. Because every interviewer scored the same criteria on the same scale, the outcome can say clearly where the candidate is strong, where there are doubts and what still needs to be checked. That conclusion can then be held against the written entry criteria of the next pipeline stage: does this candidate meet them, yes or no, and on which evidence.
It also makes candidates comparable with each other. When three candidates for the same vacancy were scored on the same five criteria, the team compares evidence per criterion instead of overall feelings.
Common mistakes to avoid
- The scorecard is invented during the interview — the interviewer decides on the spot what to look at, so the first candidates were assessed on different points than the last
- Different scales — one vacancy uses 1 to 5, another 1 to 10 and a third yes or no, so reports across vacancies say nothing
- Only an empty free-text field — a single open box invites the same impressions as before, just in a different place
- Scores without evidence — a number with no explanation cannot be checked or discussed
- Scoring after the debrief — when scores are filled in after the discussion, they reflect the group opinion, not each interviewer
- Criteria that have nothing to do with the role — vague points such as “click” or “fit” without a description open the door to bias
Measure it with your own data
Count from your own records: what share of vacancies had a scorecard before the first interview, what share of interviews has a completed scorecard from every interviewer, how many scores include evidence, and how long it takes between the interview and the completed scorecard. Report the number of interviews behind each figure. Do not rely on general claims about better hires; your own counts show whether interviews are really being assessed in the same way.
A practical start
- Pick four to six criteria from the vacancy brief together with the hiring manager or client
- Choose one scale for all vacancies and describe what each score means
- Add the scorecard to the vacancy or interview template before the first invite
- Ask every interviewer to score on their own, with evidence, before the debrief
- Write the interview outcome from the scorecards and check it against the entry criteria of the next stage
- Review once a month which criteria are rarely used or always score the same, and adjust them
Comparable interviews start before the conversation
A good interview decision is not made in the debrief, but in the preparation: agreeing what you are looking for and how you will score it. With a shared scorecard on the vacancy, every conversation adds evidence to the same picture. Explore the product or book a demo — we show how vacancies, interviews and the pipeline are set up in BoldATS for your segment.