Coding interviews in the AI world
The interview should test for the job the candidate will actually do
Are you testing the wrong thing?
The interview process is supposed to predict job performance. When the conditions of the interview bear no resemblance to the conditions of the job, you’re not predicting anything. You’re running a different experiment and hoping the results transfer.
They don’t.
What we actually test when we ban AI
The stated rationale for AI-free coding interviews is reasonable on its surface. We want to see how candidates think. We want to know if they actually understand the problem. We don’t want someone to paste a prompt and copy the solution.
The concern is legitimate. The conclusion is wrong.
Here’s what an AI-free whiteboard session actually measures: the ability to hold syntax, algorithm structure, and edge cases in working memory simultaneously, under pressure, without tooling, in an artificial environment. That is a real cognitive skill. It just isn’t the one that determines whether someone will perform well on your team in 2026.
The cognitive load of software development has shifted. The job now requires a different kind of thinking: decomposing a problem into chunks small enough to prompt effectively, reading AI output critically before accepting it, knowing when the model is hallucinating and why. A candidate who can recite a BFS implementation from memory but blindly accepts a subtly wrong AI-generated solution is a worse hire than one who forgets the exact syntax but immediately spots the flaw in what the model produced.
We’ve been measuring recall. We should be measuring judgment.
The industry has figured this out
In October 2025, Meta began rolling out an AI-enabled coding interview that replaces one of the two traditional coding rounds at the onsite stage. The internal framing is revealing. The format was designed to be more representative of the actual developer environment and also to make LLM-based cheating less effective.
The problems Meta uses are designed with AI assistance in mind. They’re harder than a traditional coding question. The bar for what a candidate is expected to produce is higher precisely because they have help. Candidates work in a multi-file codebase they didn’t write and have to understand quickly. Prompt-and-paste fails immediately in that environment because understanding the existing architecture is the prerequisite for everything else.
CodeSignal launched AI-assisted coding assessments in May 2025 with a feature that matters more than the AI access itself: a full transcript of every candidate-AI interaction alongside a session replay. You’re not just seeing what the candidate produced. You’re watching how they think.
HackerRank moved in the same direction. Candidates work with AI tools in a controlled environment and interviewers get a detailed view of the problem-solving process, not just the output.
What an AI-free interview cannot show you
When I think about what I actually need to know about a candidate, four things matter that a no-AI interview cannot surface.
Prompt quality as a diagnostic. The way someone frames a problem for an AI is a direct readout of how they think about the problem. Vague prompts reveal vague thinking. A candidate who writes “fix my function” tells you something different from one who writes “this recursive function is hitting a stack overflow on inputs above n=1000, here’s the current implementation, what’s the likely cause.” The second candidate has already diagnosed the problem. They’re using the AI to confirm and implement. That’s engineering judgment.
Verification instinct. One E7 candidate at Meta watched Claude Sonnet repeatedly hallucinate on a maze problem. The question isn’t whether the AI was wrong. The question is whether the candidate caught it. Did they know what correct looked like before the model answered? Did they push back? A candidate who accepts wrong output without question is a risk that an AI-free interview will never expose, because you never gave them an AI to accept.
Task decomposition. Candidates who performed well in Meta’s AI-enabled format guided the AI incrementally rather than asking for wholesale solutions. One successful candidate described her approach: start with the core logic as a single function, review it, then build out from there. That instinct, to keep the scope small enough to verify at each step, is exactly how good engineers approach complex problems. It’s invisible in a no-AI environment because there’s nothing to decompose for.
Communication under ambiguity. Meta’s internal evaluation criteria for this round includes a phrase that functions as an answer key: “Should use AI, but need to show you understand the code. Explain the output. Test before using. Don’t prompt your way out of it.” That is a rubric for thinking, not for tool use. The candidate is being evaluated on whether they can narrate a reasoning process in real time, hold a conversation with an interviewer while working with an AI assistant, and remain the accountable decision-maker throughout. That skill matters on the job. AI-free interviews don’t test it.
The gaming problem is real but solvable
The obvious objection is that AI-assisted interviews are easier to game. A fast prompter with shallow understanding can look strong if the evaluation is just the output.
That’s true. But it’s a design problem, not a fatal flaw.
Interaction transcripts solve most of it. When you can see the full sequence of what a candidate asked for and how they responded to what they got, shallow prompters reveal themselves quickly. They ask for too much at once. They accept the first answer without testing it. They can’t explain the code when asked.
Multi-file codebases with staged checkpoints solve the rest. A problem that requires understanding existing architecture before making any change can’t be solved by pasting a description into a chat window. The AI doesn’t have the context. The candidate has to build it, which means they have to understand it.
The gaming risk in no-AI interviews is equally real and far less visible. A candidate who has memorized LeetCode patterns looks identical to one who genuinely understands algorithms. At least AI transcripts expose the reasoning process. A whiteboard session shows you the answer. A session replay shows you how the candidate thinks.
What to change before your next hire
None of this requires overhauling your entire process. Three things make the difference.
Redesign the problem before you change the rules. A standard LeetCode question with AI access is still a bad question. The problem needs to be complex enough that AI assistance is a navigation tool rather than a solution dispenser. Ambiguous requirements, existing codebases, staged checkpoints. Problems designed so that understanding is the prerequisite for prompting.
Make the AI interaction visible and gradable. If you’re using a platform, use one that captures transcripts. If you’re running your own interviews, ask candidates to narrate their prompts out loud and explain what the AI gave them before they act on it. That narration is the interview.
Keep a short no-AI segment with a clear purpose. Baseline fundamentals still matter. A candidate who can’t read a stack trace or reason about complexity without assistance is a real risk. A focused no-AI segment to test that floor is legitimate. Don’t treat that floor as the whole evaluation.
The prediction model you’re running
Every hiring process is a prediction model. The inputs are interview signals. The output is a forecast of job performance. When the inputs don’t reflect job conditions, the model is broken.
Running AI-free coding interviews for engineers who will spend their careers working alongside AI tools is like running driving tests with no steering wheel because you want to assess balance. The rationale sounds defensible. The instrument is wrong.
The industry has started correcting. The question is whether your process has.