Nearly every hiring process in a small company runs on two artefacts: a document the candidate wrote about themselves, and a conversation they had with you. Both feel like evidence. Neither is.
This is not an argument that hiring is impossible, or that the people doing it are careless. It is an argument that the two things we lean on most were never designed to answer the question we put to them.
A CV is a claim
The CV is a marketing document. That is not a criticism — it is what the format is for. A candidate who has been through a redundancy, a reorganisation, or a stretch of contract work is doing the sensible thing by presenting that history in its best light. The problem sits with the reader, not the writer: we treat the CV as a record when it is a set of assertions arranged in order of confidence.
What a CV can tell you is limited but real. It can tell you where someone worked, and roughly when. It can tell you the vocabulary they reach for when describing their own work, which is a weak signal of exposure. It cannot tell you whether they designed the system or joined the team that inherited it. It cannot tell you whether the migration finished, or was abandoned, or still runs every night with a manual step nobody has documented. Those are precisely the details that separate someone competent from someone plausible, and precisely the details a two-page summary cannot carry.
It is also a writing test administered without a rubric. Two candidates can describe identical work, one tersely and one with a great many adjectives, and the terser one will read as the less senior of the two. A candidate who has been coached on CV writing will beat a better candidate who has not.
The interview rewards rehearsal
Thirty minutes of interview is a narrow sample taken under unusual conditions. The candidate sits opposite people they want something from, answering questions they can largely predict, in a format they have practised.
Anyone who has interviewed for a few years knows the genre. There is the question about a difficult colleague. There is the question about a project that failed, where the expected answer includes the phrase “and that is when I learned”. There is the exercise everyone describes as a conversation. A candidate who has done several of these in the last month has an advantage that has nothing to do with engineering.
Rehearsal is not dishonesty. Preparing is rational, and the ability to explain your work clearly is a real skill, worth having in people who will sit in front of customers or write design documents. But fluency under interview conditions is a specific skill, unevenly distributed, and only loosely correlated with the ability to do difficult work. Some excellent engineers are slow, deliberate talkers who go quiet under a spotlight and write beautifully when given an hour alone. Some fluent interviewees have not built anything hard in years.
What the interview reliably measures is whether someone can talk about work in the presence of strangers. We then behave as though it measured the work.
The notes do not survive a second reader
Ask two people who sat in the same interview to write up their impressions and you will usually get two different candidates. One writes “strong on design, a bit vague on testing”. The other writes “good communicator, some gaps”. Neither note records what was actually said. Both are summaries of a feeling, translated after the fact into the vocabulary of whoever is holding the pen.
This matters more than it first appears. A decision supported by adjectives cannot be re-examined. When a founder asks why one candidate progressed and another did not, there is nothing to point at. When a colleague who was not in the room wants to compare two people assessed a month apart, there is nothing to compare. When a rejected candidate asks for feedback, the honest answer is that you formed an impression and the impression was not favourable.
An interview record made of adjectives cannot be audited. It can only be re-argued.
The uncomfortable implication is not that the decision was wrong. It is that if it were wrong, you would have no way of finding out.
What would count as evidence
Take the question seriously for a moment. If you genuinely wanted to know how someone works — before committing a salary to the answer — what would you look at?
- Something they made, under conditions you setNot a portfolio the candidate chose, which tells you what they are proud of, but a task drawn from the actual work of the role. The closer the task is to a real week, the more the output means.
- The order they did it inDid they read the existing code before changing it? Did they check the assumption the brief was built on, or accept it? Did they write the test first, or last, or not at all? Sequence is often more informative than the finished artefact, because the finished artefact is what a careful person produces either way.
- What they did when they got stuckThe moment of confusion is where working style becomes visible. Some people narrow the problem. Some widen it and lose an hour. Some ask for help at the right time and some never ask at all. A record of that moment is worth more than any answer to “tell me about a time when”.
- How they used the tools in front of themDocumentation, a debugger, a colleague, an assistant. If the role involves these, the screen should involve these. Watching someone reach for a tool and use it well tells you something a closed-book exercise cannot.
- What they said about their own work afterwardsTwo paragraphs on the trade-off they made, and what they would do differently, are hard to fake and easy to assess. They also give you the candidate's own account of the work, which is a useful thing to hold next to your reading of it.
None of this is exotic. Each item is closer to the work than a conversation about the work, and each produces something a second person can look at.
What an evidence-based screen has to satisfy
For a screen to deserve the name, it needs at least four things. It is worth being strict about them, because each one is easy to skip and each one is load-bearing.
- One task, fixed before anyone is screenedIf the task drifts between candidates you are comparing different exams and calling it a comparison. A task can be mediocre and still be fair. A task that changes is neither.
- A rubric written in advance that is specific enough to be wrong“Good problem solving” is not a criterion, because nobody can disagree with it. “Notices the off-by-one before writing the fix, and says so” is a criterion. It can be met, missed, or argued about.
- A record detailed enough for a second readerThe test is simple: can someone who was not there reach their own conclusion from what you kept? If the answer is no, what you have is a memory with a score attached.
- A stated confidence, and honesty about what would raise itSome sessions produce a clear answer. Some produce a partial one, because the candidate ran out of time or the task never touched the skill in question. A process that reports every candidate at the same level of certainty is not measuring certainty. It is manufacturing it.
The point of all this is not to abolish the interview or throw away the CV. Both carry information and neither is going anywhere. The point is to stop treating them as the evidence when they are the claim, and to put something behind them that a person who was not in the room could check.
The screen described above is the one we run: a single task, a rubric fixed in advance, a recorded session, and a report that quotes it.
See how we do this →