Saffron Put Claude Code in the Interview. Grade the Prompts, Not the Diff.

If you still run a live LeetCode loop with a "no AI" policy, you are interviewing for a job that no longer exists at your company. Saffron, a Y Combinator P26 company backed by Afore Capital, just shipped a technical assessment that hands candidates a browser IDE, a real GitHub repo, and Claude Code, then scores what they prompt, what they accept, and what they rewrite. The signal moved. Most hiring loops have not.

What Saffron actually measures

Saffron drops a candidate into a sandboxed environment with a real codebase and any AI tooling they would use on the job. It captures every prompt, every diff, and every decision, then classifies each committed line as human-written, AI-generated, or AI-modified. The scoring is deterministic and structured: prompts, AI reliance percentage, tool calls, iteration depth.

The founders are Robert Chondro (MIT, Jane Street), Jerry Yao (Stanford, Jane Street), and Kazuma Choji (Harvey Mudd, published in NeurIPS and ICML). Their pitch is blunt. Interviews are testing the wrong thing. Candidates pass every round and then cannot ship a feature without AI doing the thinking for them, while the engineers who use AI to be genuinely 10x are getting filtered out by rounds that punish them for opening Cursor.

This is not a cheating-detection tool. CoderPad already sells that: keystroke playback that flags copy-paste from ChatGPT. Saffron is the opposite posture. It assumes the AI is on. The interview artifact is not the final PR. It is the prompt trajectory that produced it.

Why "did it compile" is now noise

interviewing.io ran the experiment everyone quotes: interviewers could not tell when candidates used ChatGPT mid-round to generate solutions. That was two years ago and the models have gotten better since. Canva ran its own version last year and found AI tools produced correct, well-documented solutions to their Computer Science Fundamentals questions in seconds, often on the first prompt with no clarification.

80%

If AI writes most of the code in production and one-shots your interview problems in the screen, a clean submission tells you nothing. It certainly does not tell you whether the person can supervise the code they just committed.

The three attribution metrics to adopt this quarter

Saffron surfaces these directly, but you can start tracking them in any process (async takehome, pair programming session, or a Loom review) before you buy new tooling. Name them out loud in your rubric.

1. AI Reliance %

The share of final committed lines that are AI-generated, AI-modified, or human-written. This is the headline number Saffron produces per candidate.

The trap is thinking lower is better. It is not. A senior IC who writes 100% of the code by hand in a modern repo is showing you they cannot delegate. A senior IC who accepts 95% raw AI output is showing you they cannot supervise. The range you want is role-dependent, but for staff-plus backend work, most strong candidates in Canva's pilot landed somewhere in the middle: heavy AI use, heavy editing, heavy rejection.

Set the target range before the interview. Write it in the rubric. If you cannot articulate what "good" AI reliance looks like for the role, you are not ready to run the loop.

2. Prompt Quality and Iteration Depth

Count the prompts. Read them. This is the metric that most closely maps to what Google announced for its 2026 SWE loop: a new "code comprehension" round with Gemini available, where interviewers explicitly evaluate "AI fluency, including prompt engineering, output validation, and debugging skills."

A candidate who fires one vague prompt ("write a function to parse this") and pastes the result is failing the round even if the code works. A candidate who asks three sharp clarifying prompts, pastes in the relevant file for context, rejects two suggestions, and iterates on a third is showing you the exact loop they will run on your codebase Monday morning.

The interview artifact worth grading is the prompt trajectory, not the final diff. </pull> Prompt archaeology is the new whiteboard. If you cannot see the prompts, you are grading the wrong artifact. ### 3. Human-Modification Ratio on AI Output Of the lines the AI generated, what percentage did the candidate touch before committing? This is the single cleanest proxy for "did they actually read the code." Canva's pilot found that candidates with minimal AI experience often struggled, not because they could not code, but because they lacked the judgment to identify when a suggestion was suboptimal. Their failure mode was accepting the diff. High AI reliance with a low human-modification ratio is the new "didn't finish the problem." It is the signal that separates an AI-native engineer from an AI-dependent one.

refolk prompt: Backend engineers in SF or NYC who ship AI-assisted PRs on GitHub and have written publicly about prompt engineering or agent workflows note: You get a ranked shortlist with recent commits, blog posts, and the specific projects where they used Claude Code, Cursor, or Aider in anger. slug: w8j17tz1s1

## The sourcing problem this creates

Here is the second-order effect nobody is talking about. Once you decide to hire for AI fluency, your existing sourcing pipeline is calibrated wrong. Boolean strings for "Python" and "distributed systems" surface the same 40,000 profiles they always did. None of those keywords tell you whether the person can drive Claude Code without getting driven by it.

The signal for AI fluency lives in unusual places: a candidate's blog post about their agent scaffolding, a public repo with a well-maintained CLAUDE.md file, a talk they gave at a local meetup, a thoughtful comment on a Simon Willison post. This is exactly why we built [Refolk](/): you describe the person in plain English ("staff engineer who has shipped agent workflows in production and written about prompt evaluation") and get a ranked shortlist across GitHub, LinkedIn, and the open web, rather than a keyword match that treats "AI" as a skill tag.

If you are going to run a Saffron-style loop, your top of funnel has to change to match. Sourcing for "engineers who pass an AI coding interview" is a different query than sourcing for "engineers with 5+ years Python." Refolk handles the former without you having to hand-build a new boolean every week.

## Why banning AI is now the riskier bet

The instinct for a lot of engineering managers is to double down. Lock the browser. Use CoderPad's keystroke playback. Fail anyone whose typing pattern looks pasted.

Canva ran the other experiment. They published the results in June 2025 under a title that reads like a manifesto: "Yes, you can use AI in our interviews. In fact, we insist." Simon Newton, their Head of Platforms, replaced the Computer Science Fundamentals screen with an AI-Assisted Coding competency after establishing that almost half of Canva's frontend and backend engineers are daily active users of an AI coding tool.

```stat
number: ~50%
label: Share of Canva's frontend and backend engineers who are daily active users of AI coding tools
note: Published in Canva's June 2025 engineering blog by Head of Platforms Simon Newton.
</stat>

If half your engineers use AI every day, the traditional round is not measuring on-job performance. It is measuring how well someone can pretend to work in 2019. Any EM still enforcing a no-AI policy is optimizing for a job that does not exist at their own company, and the strongest AI-native candidates will self-select out of the loop before final round.

The hiring-manager anxiety is real (65% of hiring managers say they worry about candidates using generative AI to cheat, and 20% of US professionals admit to secretly using AI in interviews). The answer is not detection. The answer is attribution.

## The interview format is about to swing back to async

There is a quiet consequence of Saffron-style attribution. To capture prompts, diffs, and reasoning at any useful depth, you need 60 to 90 minutes of process, a real repo, and time for the candidate to actually think. That does not fit into a 45-minute Zoom.

For five years the industry has been drifting toward live-only interviews because takehomes were "too easy to cheat on with ChatGPT." Attribution flips that. The takehome becomes more defensible than the live round, because you can now grade the process, not just the output. Expect the strongest hiring loops in 2026 to look like: recruiter screen that filters for AI attitude, async Saffron-style work sample with full attribution, then one senior IC conversation to walk through the prompt trajectory the candidate produced.

The recruiter screen matters more here than it looks. Canva reported that candidates who showed genuine curiosity about AI during the screen were the ones who performed well in the technical round. That is a free, upstream filter you can add to your loop tomorrow. Ask a candidate what they last used Claude Code or Cursor for. Listen for specificity. The good ones will not stop talking.

## What to do in the next 30 days

You do not need to buy Saffron this week to act on this. Three moves you can make now:

1. Add an "AI fluency" line to your interview rubric. Define what AI Reliance %, Prompt Quality, and Human-Modification Ratio look like for the role. If you cannot define them, do not run an AI-permitted round yet.
2. Rewrite one round of your loop to be AI-permitted, ambiguous, and repo-based. Steal Canva's design principle: problems that cannot be solved with a single prompt. Iterative thinking, requirement clarification, decision-making.
3. Recalibrate your sourcing to prioritize public evidence of AI fluency. This is a plain-English query, not a keyword search, and it is exactly what Refolk was built for. "Show me infra engineers who have shipped Claude Code or Cursor workflows in production and wrote about it in the last 12 months" returns a different shortlist than any boolean you have run this year.

The clean submission is dead. What remains is the prompt trajectory, the rejected suggestion, and the line the candidate rewrote because the model got it subtly wrong. Grade that. Hire from that. Source for that.

## FAQ

### Is Saffron just anti-cheating with extra steps?

No, and this is the important distinction. CoderPad's playback feature is anti-cheating: it exists to catch pasted AI code. Saffron assumes the AI is on and grades the process. The stated thesis is that companies are missing the engineers who use AI to be genuinely 10x, and the way to find them is to watch them work with AI, not to lock the browser.

### What is a reasonable AI Reliance target for a senior backend hire?

There is no industry benchmark yet, and role varies a lot. As a starting point, if a senior IC produces a working feature with 95%+ raw AI output and no rejections or edits, that is a red flag for supervision, not a green flag for productivity. Canva's strong performers used AI heavily but also rejected and rewrote suggestions frequently. Define the target range before the interview and put it in the rubric.

### How do I source candidates who will do well in an AI-permitted loop?

Look for public evidence of AI fluency: repos with a maintained CLAUDE.md, blog posts about prompt evaluation or agent scaffolding, talks at meetups, thoughtful comments in the LLM tooling community. Boolean strings against "AI" as a skill tag will not surface this. A plain-English query in Refolk ("engineers who shipped agent workflows in production and wrote about it") will.

### Will Google's 2026 interview change kill the LeetCode round entirely?

Not immediately, but the direction is clear. Google is adding a "code comprehension" round where candidates work with an existing codebase and Gemini is available, and interviewers evaluate prompt engineering and output validation. When Google, Canva, and a YC-backed startup all move in the same direction inside 18 months, the standard loop follows within two hiring cycles. Plan for it now.