The Repo-in-Lieu Standard: Waive, Shorten, or Assign the Technical Screen

Sometimes a candidate asks to skip your take-home and sends you to their public repos instead. This standard is how a recruiter and a hiring engineer decide, the same way every time, whether that code can replace the technical screen. It is for in-house recruiters, sourcers, and founders who hire engineers, often people they found through open-source work. It gives you three verdicts (Waive, Shorten, Assign), a coverage sheet that both graders fill in row by row, and a written equivalent route for candidates whose code is private.

What does the repo-in-lieu standard decide?

The repo-in-lieu standard decides one thing. Do specific public artifacts, each proven to be the candidate's own work, cover the rows of your existing screen rubric well enough to replace some or all of the screen? It does not decide whether to interview, what level the candidate is, or whether to hire.

That narrow scope is the point. Most teams that accept a GitHub portfolio instead of a take-home do it informally. An engineer skims the profile, likes what they see, and waives the exercise. Two candidates with similar repos then get different treatment depending on who did the skimming. This standard replaces the skim with a mapping exercise against a rubric you already use.

Terms used throughout:

• Screen rubric: the rows your standard technical exercise is graded on, such as problem decomposition, testing, or API design. The standard never invents new rows for repo candidates.
• Artifact: one commit URL or one pull request URL. A repo, a profile, or a star count is not an artifact.
• Authorship label: verified, partial, or unverified, given to each artifact by the check in this guide.
• Covered: a rubric row with at least one verified artifact that also passes the recency, scope, AI, and IP tests.
• Uncovered: a row with no qualifying artifact. Uncovered describes the evidence, not the engineer.

The three verdicts

The half-the-rows line for Shorten is my default, not a research finding. Choose your own line if you prefer, but write it into the policy before the first review. A threshold chosen after looking at a candidate is exactly the inconsistency this standard exists to prevent.

That figure is why a thin public footprint leads to Assign and never to a fail. A missing artifact for a rubric row tells you almost nothing about whether the candidate can do the work. It only tells you the public record does not show it.

When is public code enough to replace a technical screen?

Public code is enough when every rubric row it claims to cover is backed by an artifact that passes five tests: authorship, recency, scope match, AI assistance, and IP hygiene. If an artifact fails any one test, it drops out, and the row it was covering goes back to uncovered.

Authorship

Authorship proves that the candidate made the change, not a collaborator or a bot. It most often misleads through the "Verified" badge, covered in the next section. Every artifact gets a label (verified, partial, or unverified) before it can count toward any row.

Recency

Recency proves the skill is current. Write a recency window into the policy before reviewing. I use the same window the job description uses for "recent experience". This test can mislead in two directions. Old work can look strong but use a stack the candidate has not touched in years. Old commits can also look suspicious because they are unsigned, when signing simply was not in use at the time. Judge older commits by their dates, never by the badge alone.

Scope match

Scope match proves the artifact deals with the same kind of problem the screen tests: the same stack, scale, and shape of work. A beautiful command-line utility does not cover a row about designing a service under concurrent load. This test misleads when a repo is impressive as a whole but no single artifact maps to a specific row. If you cannot write the row name next to the URL, the artifact does not cover the row.

AI assistance

This test proves that a human, not a coding agent, did the design and judgement the row grades. No published guidance exists on discounting AI-assisted work in a hiring assessment, so the rule below is a policy choice:

• Exclude agent-authored artifacts outright.
• Mark human commits that carry an AI co-author trailer as partial.
• Move the rows those partial artifacts cover into the walkthrough.

This test misleads when the absence of a trailer is taken as proof that no AI was used. Whether a commit gets a trailer depends on the tool's settings at the time, so a missing trailer proves nothing.

IP hygiene

IP hygiene proves the code is the candidate's to show you. Basecamp's write-up on take-home tests puts it plainly: "it's rarely an option for candidates to submit code from a previous job with an application to a new job." This test misleads when a side project looks personal but relates to the employer's business. That is exactly where California's section 2870 stops protecting the employee: an employer cannot claim an invention made on the employee's own time and equipment "unless it relates to the employer's business."

How do you prove the candidate wrote the code?

Read authorship from the individual commits inside the candidate's pull requests and from who opened each PR. Do not rely on the "Verified" badge or the squashed commit on the default branch. Where that record is thin or contradictory, the live walkthrough is the final test.

No documented end-to-end procedure exists for confirming that a candidate wrote a given change. What follows is put together from how GitHub records each signal. Each layer is weaker than the one below it.

What the Verified badge proves

GitHub marks a commit or tag "Verified" when it carries a GPG, SSH, or S/MIME signature that GitHub can check cryptographically. That proves a signature exists. It does not prove who wrote the code. GitHub's documentation for the Copilot cloud agent says the agent's commits are authored by Copilot, with the human who started the task listed as co-author. It also says "the cloud agent's commits are signed, so they appear as 'Verified' on GitHub." A green badge on an agent commit is working exactly as designed.

Two more properties matter for older work:

• A stored verification record "can't be edited and will persist," even after keys are rotated or contributors leave.
• With vigilant mode on, every unsigned commit attributed to the user's email, including old ones, shows "Unverified".

An Unverified label on a commit made before the candidate started signing tells you about its date. It is not a red flag.

Squash merges and editable authors

When a PR is squash-merged, the author of the resulting commit can be set at merge time. A GitHub community discussion about Copilot PRs documents the workaround: "Click 'Change author' Select your GitHub user". So the author shown on the default branch is a choice made by whoever merged the PR. Open the PR, read its individual commits, and compare who opened it with who merged it.

Co-author trailers

A Co-authored-by line is just text in a commit message, and anyone can type one. It only counts toward a co-author's contributions if it uses the email associated with their account. Check that the email belongs to the candidate's account before giving it any weight. Even then, label the artifact partial, because co-authorship does not tell you which part the candidate wrote.

Agent and AI markers to scan for

• A bot author, such as a squash merge credited to copilot-swe-agent[bot].
• An Agent-Logs-Url trailer. GitHub says every cloud agent commit message links to the agent's session logs, and the attribution lint tool commit-check documents this trailer.
• Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>, which Copilot CLI adds automatically.
• AI co-author trailers added by editor settings. VS Code's git.addAICoAuthor setting has three modes (off, chatAndAgent, and all), and its default has changed between releases.

The repo-in-lieu procedure, step by step

The procedure takes the candidate's list of nominated work to a logged verdict in eight steps, shared between the recruiter and the hiring engineer. None of the timings here come from a published source. Treat them as estimates and replace them with your own after a few reviews.

Intake is the step teams skip, and it is the one that protects you. If you found the candidate through Refolk from their open-source work, the repos that surfaced them are already attached to their profile. The recruiter can propose a starting list and ask the candidate to confirm or add to it, instead of starting from a blank email. The IP attestation still comes from the candidate, in writing, every time.

Intake usually means about a day of waiting on the candidate (an unverified estimate). Sources disagree on when this review belongs. Basecamp gives its exercise only to late-stage candidates. Indeed recommends weighing portfolios alongside the challenge rather than instead of it. This standard is closest to Basecamp: run the review at the point where you would otherwise send the exercise, not earlier.

If you want more candidates whose commit history will pass the authorship and AI checks, you can filter for it when sourcing.

The engineer fills in the coverage sheet below, and the recruiter keeps it as the record of the verdict.

REPO-IN-LIEU COVERAGE SHEET
Candidate / role / level:
Recency window (set before review):
Stack and scale in scope:
IP attestation received (date):

ROW: <rubric row name>
Artifact URL (commit or PR, or "uncovered"):
Authorship label (verified / partial / unverified):
Evidence for label (PR opener, merger, individual commit authors, trailers):
AI or agent markers found:
Artifact date / inside window (yes / no):
Scope match (yes / no, one line why):
Status (covered / uncovered / walkthrough):
Recruiter score (1-5):
Engineer score (1-5):
Gap over 1 point resolved how:

VERDICT (Waive / Shorten / Assign):
Reasons:
Path offered / path taken:

How the repo-in-lieu standard goes wrong

The standard fails in eight known ways. Most of them produce a false Waive, where a candidate skips the screen on evidence that did not prove what the reviewer thought it proved. Each entry gives the failure, what it looks like when it misleads, and the check that catches it.

1. The Verified badge is treated as proof of authorship. False positive: a signed commit written by an agent shows "Verified" and counts toward a row. Check: compare the author with the committer, and look for a bot account or an Agent-Logs-Url trailer.
2. Squash-merge attribution is taken at face value. False positive: the squash commit on the default branch names the candidate, but the person who merged it set that author at merge time. Check: open the PR, read the individual commits, and compare who opened the PR with who merged it.
3. A Co-authored-by line is treated as evidence. False positive: a trailer names the candidate on a commit someone else wrote. Anyone can add the line, and the email may not belong to any account. Check: confirm the email belongs to the candidate's account. Even then, label the artifact partial.
4. A missing signature is read as a negative. False negative: older commits cannot be signed after the fact, and vigilant mode marks them Unverified. Check: judge by the commit dates, not by the badge alone.
5. A side project turns out to be employer IP. False positive: a "personal" repo whose subject overlaps the employer's business, so the section 2870 carve-out does not apply. The DTSA immunity covers disclosures to a government official or a lawyer to report a suspected violation of law, or in a sealed court filing. It gives no cover for showing code to a prospective employer. Check: compare the repo's subject with the employer's domain and get the IP attestation. If they overlap, stop reviewing and Assign.
6. The optional path creates adverse impact. The path is only open to people with public code, and over 80% of contributions are private. Check: monitor pass rates by path under UGESP section 4D, and keep the fallback route equivalent.
7. Rubric rows are quietly dropped. False positive: rows with no artifact are left off the sheet, so "every row covered" is true only because rows went missing. Check: count the rows on the sheet against the rubric. Every row needs a URL or the word "uncovered".
8. AI-polished repos pass as craft. One practitioner write-up warns that "With AI coding tools, anyone can generate a clean-looking repo in an afternoon, and reviewers know it." Check: read the commit history and run a live walkthrough. A repo whose whole history arrives in a few large commits gives you nothing to check authorship against, so move its rows to the walkthrough.

Failures 1, 2, and 8 come from the same mistake: trusting a record that someone other than the candidate could have shaped. Failure 7 is the most common in practice, because nobody notices a row that is missing. Failure 6 is the one that shows up a year later in an audit.

How do two graders reach the same verdict?

Two graders reach the same verdict through a fixed rubric, independent scoring, a small benchmark library, and repeated calibration. Expect weak agreement on the first few reviews and plan for it.

No published agreement target with a stated method exists for engineering take-homes, so the benchmarks below are borrowed from nearby settings. Treat them as rough reference points, not targets you are entitled to hit.

A few terms:

• Inter-rater reliability (IRR): how consistently different graders score the same case.
• Weighted kappa: an IRR measure that corrects for chance agreement and gives partial credit when graders are close.
• OSCE: a structured clinical skills exam used in medical education.

Three readings matter for this standard:

• Calibration takes repetitions. In the OSCE study, weighted kappa went from 0.49 to 0.57, then 0.66, then 0.82 across four blocks of 10 videos. Two graders reviewing repos together for the first time should expect to start near the low end.
• Shared evidence helps. Panel interviews averaged .74 against .44 for separate interviews by different interviewers. A repo review is closer to the panel case, because both graders look at the same artifacts. That is my inference, not a measured result, but it is a good argument for grading from one shared coverage sheet.
• The practical target is within 1 point. The working definition of agreement is scores within one point on a 1-5 scale, with agreement above 70% taken as a sign of a clear rubric. That comes from a vendor blog, so adopt it as your team's target, not as evidence that you have reached reliability.

Build a benchmark library before the second review. Practitioner guidance recommends "a collection of 3 - 5 anonymized submissions that exemplify clear pass, clear no, and borderline cases." For this standard, that means past coverage sheets with the candidate's identity removed: one clear Waive, one clear Assign, and one or more borderline Shorten cases. New graders score the library on their own, compare scores, and discuss every row where they differ by more than 1 point before they grade a live candidate.

How do you keep the optional path fair to candidates with no public code?

Offer the repo path to every candidate at the same stage, in the same words. Make the standard exercise and the walkthrough grade the same rows to the same bar. Track pass rates by path and group over time. The repo path is open only to people with public code, and that group is small and not representative of all engineers.

In Refolk's index of professional profiles, the 19 US matches work at employers that include the Python Software Foundation, Docker, Microsoft, Shopify, and LangChain, and one matched title is "Python Steering Council". The 5 German matches are at Docker, Brainlab, VECTOR Informatik, S&P Computersysteme, and mapeditor.org. These counts measure how people describe themselves in their headline, not actual open-source activity, so many active contributors will not appear in them. Even so, the direction is clear: the Waive path will be rare.

What UGESP asks of an optional path

UGESP (the Uniform Guidelines on Employee Selection Procedures, 29 CFR Part 1607) is the federal framework US agencies use to judge whether selection procedures have adverse impact. Four provisions matter here:

• Lesser adverse impact. Where two procedures are "substantially equally valid for a given purpose," the user "should use the procedure which has been demonstrated to have the lesser adverse impact."
• The four-fifths rule. A group's selection rate below four-fifths (80%) of the highest group's rate "will generally be regarded as evidence of adverse impact." The EEOC "is not bound" to base a determination on this rule.
• The bottom line. If the whole process shows no adverse impact, the agencies usually will not act on any single component, including "a separate procedure that is used as an alternative method of selection." That is enforcement discretion, not a safe harbour.
• Documentation. Procedures "should be completely and explicitly described." An unwritten waive habit fails this on its face.

UGESP gives no formula for an optional path. My working method is derived, not prescribed. For each group, compute the pass rate on each path (repo review, take-home) and for the whole process. Divide each group's rate by the highest group's rate. Separately, track who was offered the path at all. With path populations this small, the ratios will swing wildly from one hire to the next. Keep cumulative records and read the trend, not any single quarter.

The equivalent route, written down

For candidates whose work is private, the standard exercise is the route. It grades the same rubric rows the repo review maps to, so the bar for each row is the same whichever path the candidate takes. Never accept private employer code as a substitute, and never count the absence of public code against anyone. Send both options in one message, using the same wording for every candidate.

Hi <first name>,

Every candidate at this stage can choose one of two routes:

1. The standard exercise: <one-line description>, graded on <number> rubric rows.
2. A repo review: you nominate specific public commits or pull requests that you wrote, and a hiring engineer maps them to the same rubric rows. Any row your public work does not cover becomes a short live walkthrough. If fewer than half the rows are covered, the standard exercise applies.

Both routes are graded to the same bar, and choosing the exercise is never held against you. Most engineers' work sits in private repositories, so the exercise is the route most candidates take.

If you choose the repo review, please send:
- Links to specific commits or pull requests (not just repo names), with one line on what each shows.
- A short written confirmation that none of the nominated code belongs to a current or former employer, and that none of it relates to your current employer's business.

Please do not send code from a current or previous job, even if you still have access to it.

Thanks,
<your name>

Verification checklist and keeping the policy current

Before logging a verdict, the recruiter runs the checklist below. The policy itself needs a scheduled review, because several of the signals it depends on change with tool releases.

To keep the policy current:

• Re-check AI attribution defaults on every major tool release. Whether a commit carries an AI trailer depends on editor and agent settings that have changed between versions. Before each calibration session, look up the current default for VS Code's git.addAICoAuthor, the trailer format Copilot CLI writes, and how agent commits are credited. Then update the marker list in the AI scan.
• Re-read GitHub's signature verification documentation when you revise the policy. The meaning of "Verified" and how vigilant mode treats old commits are product behaviour, not law, and both can change.
• Grow the benchmark library. After each borderline Shorten, remove the candidate's identity from the coverage sheet and add it to the library. Re-run calibration whenever a new grader joins.
• Review path statistics cumulatively. Pull pass rates by path and group from the start of the policy, not just the last period, and record any change to thresholds alongside the date it took effect.
• Re-check the legal text at its source. Read UGESP in the eCFR directly rather than a summary, and have counsel confirm the IP and adverse-impact points in each jurisdiction you hire in.