How to run a technical screen a non-engineer can defend

In a small company, the technical screen usually belongs to whoever has the least spare time. Sometimes that is the technical founder between two release cycles. Often it is an operations lead, a generalist, or a first recruiter who took the role because nobody else would.

The awkward part is not running the screen. It is being the person who has to say yes or no at the end, to a candidate whose work you cannot fully evaluate, on the strength of notes you did not write and would not know how to challenge.

That is a solvable problem, but only if you fix the process before the first candidate and accept that your job is not to become an engineer for the afternoon. Your job is to run a process whose output you can read.

Fix the task and the rubric before the role goes live

Two things need to exist before anyone applies: the task, and the standard you will judge it against. Both should be written down, and both should be agreed with whoever on the team does have technical judgement — even if that person has twenty minutes a week to give you.

The task should come from the work. Not a puzzle, not a problem from an interview practice site unless the job genuinely consists of puzzles. Take something the team actually did last quarter, strip out the confidential parts, and keep the shape of it: a vague requirement, an existing codebase with a wart in it, a decision with a real trade-off and no obviously correct answer.

It should be short enough to finish and long enough that finishing badly is possible. If every candidate who attempts the task produces more or less the same thing, the task is not discriminating between anything, and the rest of the process is decoration.

The rubric is the harder half. Write criteria as observable behaviour, and write them before you have a candidate in mind, because the temptation to fit the standard to the person in front of you arrives fast and feels like insight.

“Would I work with them” is not a criterion. It is a conclusion you will reach anyway, and writing it down only disguises it as a measurement. Neither is “good culture fit”, which in practice means the candidate reminded the interviewer of themselves.

Give each criterion a weight, and write one line explaining the weight. If you cannot explain to someone else why communication is worth more than speed for this role, you do not have a rubric. You have a preference with a number after it.

Record the session, not the impression

The single most useful thing a non-engineer can insist on is a record. Not notes. A record.

What you want is the sequence: the brief as the candidate received it, the messages they sent, the work they produced, the order of it, and when they stopped. With that, a technical colleague can form an independent view in twenty minutes, and you can have a conversation about a specific moment instead of a general feeling.

Screen recording is one option, and it is intrusive enough that many good candidates will decline. A transcript, the final artefact, and timestamps are usually enough, and they are far easier to review, search, and store responsibly.

Two practical rules. First, get written consent before you record anything, and say plainly what is kept, who sees it, and for how long. Consent collected after the fact is not consent; it is a request for forgiveness. Second, keep the record for every candidate, including the ones you rejected in ten minutes. A process that only documents its successes cannot be checked, and the candidates you rejected are the ones most likely to ask why.

Reading how a candidate uses the assistant

If the role involves an AI assistant — and increasingly it does — then the screen should include one. What you are looking at is not whether the candidate used it. It is how.

• What they ask itA narrow question — what this function returns, why this test is failing, what the idiomatic way to do something is in this library — suggests someone directing a tool. A single broad instruction to build the whole feature suggests someone hoping the tool will do the thinking. Both tell you something. Only one is a working style you can hire into a small team.
• What they do with the answerDo they read it before pasting? Do they run it? Do they check the edge case the assistant did not mention? Output that goes straight into the deliverable without being executed is visible in the record, and it is one of the clearest signals in the whole session.
• Whether they catch it being wrongAssistants are wrong with confidence, and the wrongness is usually subtle: an off-by-one, a library that does not exist, a plausible-looking function signature that is not the real one. A candidate who stops, doubts, and tests is doing the job. A candidate who accepts is doing a different job.
• What they say about it afterwardsA short note at the end — the assistant suggested one approach, I went with another because of the retry behaviour — tells you more about judgement than any answer to a direct question about judgement.

None of this requires you to read the code. It requires you to read a transcript, which is a skill you already have and use every day.

What the report has to say

Eventually you have to write something down, or read something someone else wrote, and decide. A report you can act on has a recognisable shape.

• The brief, in plain languageOne paragraph on what the candidate was asked to do, written for someone who will never open the repository.
• Each criterion, with its evidenceThe score, a quotation from the session that supports it, and one sentence on what that quotation shows. If the quotation does not support the score, the score is wrong, and that is a useful thing to notice before you make an offer rather than after.
• The counter-evidenceWhere did they struggle? What was missing? A report with no weaknesses in it is not a report. It is a reference.
• A confidence statement, and what would raise itConfident on the debugging criterion, not confident on design because the task never forced a design decision — that is a sentence you can act on. A percentage with no argument behind it is not.
• A recommendation, labelled as oneThe report should say what the evidence supports. It should not pretend to be the decision. You know the role, the team, the budget, and the person you already have in the seat next to this one.
• The record, attachedEvery claim above has to be checkable. If it cannot be checked, it is only a more confident version of the interview notes you were trying to replace.

Saying no with a straight face

The test of a process is not the candidate you hired. It is the candidate you rejected, six weeks later, when someone asks why.

If you can open the record, point to the moment a criterion was missed, and read out what actually happened, you are fine. You do not need to be an engineer to do that. You need the record to exist, and the criterion to have been written before you met the person.

There will still be judgement calls. A candidate who is strong on three criteria and weak on the fourth is a decision, not a calculation, and no process will make that disappear. What a good process does is make sure the decision is yours, informed, and explainable — rather than a feeling that hardened into a fact because nobody wrote anything down.