Coding interview questions that actually tell you something

A good interview question separates engineers who will do the job well from engineers who will not. That is the entire specification, and it rules out most of the questions in common use — not because they are too hard, but because they sort candidates along an axis that has little to do with the work.

The test for a question

Before adopting a question, check it against three things.

Does it discriminate? If nearly everyone passes or nearly everyone fails, you have spent 45 minutes learning nothing. Questions drift towards both extremes over time — an easy one stops distinguishing as your candidate pool improves, a hard one becomes the reason you reject people you should have hired.

Is the signal about ability or about exposure? Many questions have a single insight. If a candidate has seen the insight before, they solve it in four minutes; if not, they may not find it in forty. You have measured whether they read a particular blog post.

Does it survive being answered badly? The most useful questions still produce information from a candidate who does not finish. A question where partial work is uninterpretable throws away your data on everyone in the middle — which is most people, and precisely the group where your decision is hardest.

Four question shapes that work

1. Debug something that is broken

Give them a small program — 100 to 200 lines — with a real bug, and a failing case. Ask them to find and fix it.

This is the closest common interview format to actual daily work, and it is dramatically under-used. It tests reading unfamiliar code, forming and discarding hypotheses, using tools deliberately, and knowing when to stop theorising and add a print statement.

It also degrades gracefully: everyone makes progress, so you can compare how people narrowed the search space even when nobody finishes. And it is nearly impossible to prepare for in the memorisation sense, because the bug is yours.

Pick a bug with a plausible cause and a non-obvious location — an off-by-one in a boundary case, a mutation of a shared object, a swallowed exception. Avoid anything that hinges on a language-specific gotcha, which is exposure again.

2. Extend an existing codebase

Provide a small working program and ask for a feature. The signal is in whether they read enough to follow existing conventions, whether they find the right place to make the change, and whether the result looks like it belongs.

This is the only common format that tests the single most frequent engineering activity — modifying code that already exists — and it tends to surprise people who interview well on blank-file problems.

3. A problem with no clean answer

Deliberately underspecify. “Here is a log format. Write something that tells us which requests are slow.” There are ten reasonable interpretations, and what you are watching for is whether they notice that.

Strong candidates ask what “slow” means, whether the file fits in memory, whether this runs once or continuously. Weaker ones pick an interpretation silently and are then surprised when it was the wrong one. This is directly predictive: the expensive mistakes in real projects are almost never algorithmic, they are building the wrong thing confidently.

4. Something from your actual product

Take a real problem your team solved in the last year, strip it to something achievable in half an hour, and remove the domain knowledge requirement.

The advantages compound. It is obviously relevant, so candidates engage with it rather than enduring it. You already know the interesting trade-offs, so you can have a real conversation about their approach. And it cannot be memorised from a question bank.

Three that reliably do not work

Competitive-programming puzzles

Dynamic programming problems, clever graph traversals, anything with a known trick. The defence is that they measure raw problem-solving. What they actually measure, with high fidelity, is recent practice on this specific corpus.

That practice is a full-time activity for some weeks. It is available to new graduates and to people between jobs, and much less available to senior engineers with families and current employment — a group you probably want to hire. The correlation with job performance is weak; the correlation with time spent grinding is enormous.

Trivia

“What is the difference between X and Y?” Either they know it or they do not, and either way the answer takes eight seconds and tells you nothing about what they would do with the knowledge. Everything in this category is a search away on the job.

The single-insight puzzle

Any question where the outcome hinges on one non-obvious realisation produces a binary result. The candidate who saw it looks brilliant, the one who did not looks lost, and the difference between them is frequently luck. Worse, these questions are seductive for interviewers, because watching someone find the insight feels like watching intelligence happen.

Calibrate before you use it

A question is not ready until you have run it on people whose ability you already know. Give it to three engineers on your team, one strong, one average, one recently hired. If they all breeze through, it is too easy. If your strongest engineer struggles, it is too hard, or unclear, and you will be rejecting good candidates on the strength of your own ambiguous wording.

Time it too. A question that takes your team ten minutes will take a nervous candidate on unfamiliar ground thirty, reliably.

Score against the question, not against a memory

Before the first interview, write down what a weak, adequate and strong answer looks like. Concretely — which observations you expect, what “finished” means, what a good response to a hint is.

Without this, assessments drift towards the last candidate you saw, and towards how much you enjoyed the conversation. Both are noise, and the second is where most hiring bias enters.

Then keep the artefact. The code produced in the session is your evidence, and it should be reviewable by someone who was not there — which is a reason to run the interview in a shared editor whose contents persist rather than over a screen share that leaves nothing behind.

One thing worth more than the question

After they finish, ask them to review their own code. What would you change with more time? What breaks if the input is a thousand times bigger? Where would you add a test first?

The self-review is frequently the most informative five minutes of the whole session. It works on unfinished attempts. It cannot be prepared for. And it maps almost exactly onto how the person will behave in code review, which is where you will spend far more time with them than they will ever spend writing algorithms.