# The Public-Work Interview Verdict: Interview, Screen, or Pass

*You will score an engineer's public artifacts across weighted dimensions and land on interview, phone-screen, or pass in about 45 minutes.*

- Canonical URL: https://www.refolk.ai/guides/public-work-interview-verdict
- Pillar: Engineering and open source
- Format: Framework
- Published: 2026-09-24
- Last reviewed: 2026-09-24
- Reading time: 16 min
- Keywords: evaluate engineer from github before interview, screening engineers from public work, decide who to interview without resume, grading developer portfolio to interview, assess candidate github contributions

## Key takeaways

- Public code is a work sample, and work-sample validity measured .54 in Schmidt and Hunter (1998) but was revised down to .33 by Roth et al. (2005) as samples spread beyond job-specific settings, so relevance-to-role is what preserves the signal.
- The cheapest artifact to read, the contribution graph, is the easiest to fake: backdating tools generate commits on 70% of days automatically, while a merged PR discussion thread cannot be scripted.
- AI-authorship detection cannot be a pass/fail gate. OpenAI's own classifier caught only 26% of AI text while flagging 9% of human writing, so the check feeds a phone-screen, never a rejection.
- A scored gate's payoff is a pass-rate shift, not raw speed: raising first-round pass rate from 34% to 54% cut recruiter workload from about 160 hours to 82.
- Supply scarcity should move the threshold, not just sourcing: in Refolk's index Rust engineers are about 4.8x rarer than Go in the US, so for scarce stacks the pass bar drops toward phone-screen.
- Job-search-timed repos are systematically lower use-value, with fewer stars and forks, so recency without corroboration from a second source is a weak signal, not a strong one.

You have more public work in front of you than interview slots to give. This guide is for engineering managers, technical founders, developer-relations leads, and technical sourcers who need to decide whether an engineer's public artifacts earn one of those slots, before any resume or take-home exists. It converts scattered signals - shipped code plus the context of the projects and companies it shipped in - into one of three defensible verdicts: interview now, phone-screen first, or pass, scored so that two reviewers land the same way.

The ranking pages that dominate this topic are portfolio checklists. They tell an engineer what to publish. They do not weigh the dimensions, set a threshold, or handle gamed contribution graphs and AI-generated code. This is the missing half: a scored verdict for the person spending the slot.

## Why score public work at all, and what it does and does not predict

Public code is a work sample, and work samples are among the better-validated selection methods, but their measured strength has fallen as they spread beyond specialist settings. That is the honest starting point, and it shapes everything downstream.

The strongest single number here is not about GitHub specifically. It is about the class of thing GitHub work belongs to. Schmidt and Hunter (1998) reported work-sample validity at r = .54. Roth, Bobko and McFarland (2005) revised the mean to r = .33. The mechanism matters: broader, less job-specific samples dilute the signal. So the artifact alone does not carry validity. Relevance to the role does.

| Method | Schmidt & Hunter 1998 (r) | Later revision (r) |
|---|---|---|
| Work sample test | .54 | .33 (Roth 2005) |
| Structured interview | .51 | .42 (Sackett 2022) |
| GMA | .51 | .31 (Sackett 2022) |
| Years of experience | .18 | - |

Read this table twice. Structured interviews and general mental ability sit at the top and hold their ranking even under the conservative Sackett et al. (2022) corrections. Years of experience predicts performance at r = .18 and plateaus after about five years - filtering primarily on experience eliminates high-ability candidates who would outperform experienced but less capable applicants. That is the case for reading code before reading a resume: the resume proxy you would otherwise lean on is the weakest column in the table.

One caution the literature is blunt about. There is no published validity figure for GitHub-specific signals against measured job performance. Blog claims of "40% better in reviews" are unsourced marketing. What is peer-reviewed is subtler and useful: a study of some 22,900 GitHub developers found open-source activity is significantly elevated during the job-search period, and that signaling projects are stargazed and forked less, pointing to lower direct use-value. Job-search-timed repos exist, and they are systematically shallower.

> **Note:** What the evidence supports
>
> Public code is a moderately predictive work sample whose value rises with job-relevance. It is not a validated proxy for performance, and it does not replace a structured interview. Score it to decide who gets the interview, not to skip it.

## Which public artifacts to read, and how each one lies

Not all artifacts carry equal weight, and the cheapest one to read is the one most likely to mislead you. The rule of thumb: the harder an artifact is to fake, the more it should move your verdict.

Here is what is retrievable per candidate and what each signal proves versus how it deceives.

| Artifact | What it proves | How it lies |
|---|---|---|
| Merged PRs and threads | Real workflow: CI, review, iteration | Rare; may be one-off or trivial |
| Contribution graph | Rough activity over time | Scriptable; gamed with backdating |
| Authored repos with READMEs | Ownership and communication | May be bootcamp clones or to-do apps |
| Stars and followers | Some traction at high counts | Vanity; popular can be shallow |
| Forks | Interest in a project | Forks without changes prove nothing |

Pull requests are the strongest artifact, and it is worth being specific about why. Open source forces CI, code reviews, and contribution guidelines that mirror real engineering workflows, and the PR discussion thread shows how someone debugged and iterated. A 30-comment review thread cannot be scripted. When you read a merged PR, use three tabs: the Conversation tab shows the description, timeline, comments, and reviews; the Commits tab shows how the branch changed over time; the Files changed tab shows the diff.

The contribution graph is the weakest artifact and the one recruiters over-trust. Many shortlist on the number of green squares and their distribution, but it is essential to look beyond them because they can be gamed. Depth beats daily commits. A single well-documented fix for a complex bug can carry far more weight than dozens of minor typo fixes.

> The cheapest artifact to read is the easiest to fake, so invert the instinct to glance at the graph first.

Forks and clones are the two traps that inflate a thin profile. Developers who only fork projects without meaningful contributions may lack the ability to create or improve code, and repositories filled with to-do apps, website clones, or algorithm practice often reflect bootcamp assignments rather than real-world problem-solving. High repo count is not productivity. Filter to authored code and merged PRs before you count anything.

## The scoring model: four dimensions, weighted

Score four dimensions - relevance, depth, recency, and corroboration - and weight relevance and depth above the other two, because those are the dimensions that preserve validity and resist gaming. No published, validated weighting scheme exists for a three-way engineering verdict, so this is a constructed model. I will say plainly where it rests on judgement rather than evidence.

Practitioner rubrics converge on the same qualitative dimensions even though none of them publishes weights: check what is behind the green squares, prize depth over daily commits, distinguish original code from forks, value meaningful PRs and community engagement, and note projects with 20+ stars showing real traction. Depth over volume is the recurring rule. What no source does is assign numbers. I do that here so two reviewers can reproduce a verdict, and I mark it as my construction.

Score each dimension 0 to 3, then multiply by its weight.

| Dimension | Weight | 0 (absent) | 3 (strong) |
|---|---|---|---|
| Relevance to role | 3x | Unrelated stack or domain | Same stack and problem domain |
| Contribution depth | 3x | Typo fixes, unchanged forks | Complex, well-documented merged fix |
| Recency | 1x | No activity in 12 months | Substantive work in last 90 days |
| Corroboration | 2x | Single unverifiable source | Two independent sources agree |

The weights are deliberate. Relevance carries the most weight because it is what kept work-sample validity from collapsing further than it did. Depth is tied with it because depth is the dimension gaming cannot manufacture. Recency is weighted lowest on purpose: job-search-timed repos are real, so recent activity without a second source is job-search theater, not evidence. Corroboration sits in the middle because agreement across sources - a merged PR plus a conference talk plus a published package - is the cheapest defense against every failure mode in this guide.

Maximum score is (3+3+1+2) x 3 = 27. Set thresholds:

- **Interview now:** 20 and above, with neither relevance nor depth scored 0.
- **Phone-screen first:** 11 to 19, or any high total where one core dimension is 0 or an AI flag fired.
- **Pass:** 10 and below.

> **Rule:** A zero in a core dimension caps the verdict
>
> No matter the total, if relevance or depth scores 0, the verdict cannot be "interview now." A candidate with a busy, recent, corroborated profile of trivial work is a phone-screen at best.

Treat the thresholds as a starting calibration, not a law. Run ten known cases through them first and adjust the cut points until the model agrees with judgements your team already trusts.

## Move the threshold with supply

Scarcity should move the pass bar, not just your sourcing effort. For a stack where qualified engineers are thin on the ground, a borderline score that would be a pass in an abundant market should shift toward phone-screen, because the alternative is an empty funnel.

This is where the market data earns its place in the model. In Refolk's index of professional profiles, Rust engineers are far rarer than Go engineers in the same market, and the gap widens across borders.

**4.8x - How much rarer Rust is than Go among US software engineers**

Refolk's index lists 2,698 US engineers with Go versus 566 with Rust.

| Skill | US software engineers | Share vs Go |
|---|---|---|
| Go | 2,698 | 1.00x |
| Rust | 566 | 0.21x |
| Go-to-Rust multiple | 4.8x | - |

The cross-border picture is starker still. Refolk's index shows 85 Germany-based software engineers listing Rust against 566 in the US, a 6.7x difference. If you are hiring Rust in Germany, the population you are drawing from is small enough that a strict pass bar will exhaust it before you fill a slot.

| Country | Engineers listing Rust | Top employer in sample |
|---|---|---|
| United States | 566 | Meta |
| Germany | 85 | Wolt |
| US-to-Germany ratio | 6.7x | - |

The practical rule: for an abundant stack, hold the interview bar at 20. For a scarce one, drop the phone-screen floor so more borderline candidates get a live conversation rather than a silent pass. You are not lowering your standard for the role. You are moving where public work alone can decide versus where a screen has to.

Finding enough profiles in a scarce stack to calibrate against is its own problem. This is the friction Refolk removes: instead of scraping repos and cross-referencing them by hand, you ask for the population in plain English and get the profiles back.

I ran this search: `Engineers in Germany with merged pull requests to popular Rust open-source projects in the last 12 months.` - [see the full result list](https://www.refolk.ai/s/p8vaa2ky10).

*Returns Germany-based engineers with recent merged Rust PRs, the exact population a scarce-stack verdict has to be calibrated against.*

## The procedure, step by step

Run the eight steps below in order. Budget about 45 minutes per candidate, most of it in step three, and write down every score with the artifact that produced it.

#### From handle to verdict

1. **Define the role bar and collect handles** - Fix the target stack and two or three must-have competencies, then gather GitHub, LinkedIn, and personal-site URLs. Every candidate should end with at least one verifiable public artifact link.
2. **Triage the profile for authenticity** - Confirm commits are authored not just forked, check signature and verified status, and ignore raw green-square counts. Flag the profile genuine, gamed, or thin.
3. **Read the two strongest PRs end to end** - Open the diff plus the Conversation tab. Read the description, timeline, comments, and reviews until you can name what they built and how they handled feedback.
4. **Score relevance to role** - Rate how close the artifact's domain and stack are to the open role, 0 to 3. Cite the repo behind the score.
5. **Score contribution depth** - Distinguish a complex, well-documented fix from typo fixes and forks, 0 to 3. Cite the specific PR.
6. **Score recency and corroboration** - Check for activity in the last 90 days and whether at least two independent sources agree, such as a merged PR plus a talk or a package. Record both scores.
7. **Run an AI-authorship sanity check** - Use a code-specific detector on flagship code as a flag, never a verdict, given documented error rates. Produce a flag, not a rejection.
8. **Combine and apply thresholds** - Sum the weighted scores and map to interview, phone-screen, or pass. Write the verdict so a second reviewer can reproduce it.

There is genuine disagreement about ordering. Some practitioners put authenticity triage first, as above; others start by reading PR content and treat authenticity as a later red-flag pass. One set says the first thing to look at is the candidate's activity level. I put triage first because it is cheap and it stops you from investing 15 minutes reading a spoofed history. If your candidates are pre-vetted, start with the PRs.

#### The verdict pipeline

1. **Collect handles** - Every candidate has one verifiable public link
2. **Authenticity triage** - Genuine, gamed, or thin
3. **Read strongest PRs** - Name what they built and how they reviewed
4. **Score four dimensions** - Relevance, depth, recency, corroboration
5. **AI-authorship flag** - A flag that routes, never a rejection
6. **Apply thresholds** - Interview, phone-screen, or pass

*Each stage discards or scores, so effort concentrates on candidates who survive triage.*

## Telling genuine work from gamed and AI-generated code

Two forms of fakery threaten this method: a gamed contribution history and AI-authored flagship code. Handle the first with verification and the second with a flag, never an accusation, because detector error rates make auto-rejection reckless.

Contribution-graph gaming is trivial and documented. Backdating tools create commits with backdated timestamps going back 365 days, randomly selecting days with 70% probability and one to five commits per active day. That is why a dense graph proves nothing on its own. Worse, commit identity itself can be spoofed: because activity is displayed across public and private repos, it is effectively impossible to authenticate commit veracity from the graph alone, and a threat actor can push a commit on behalf of a reputable user by spoofing username and email. The countermeasures are commit-signature verification and vigilant mode - confirm the verified badge on the commits behind a claim.

AI-authorship is the harder judgement, because the tools you would reach for are unreliable. Code-specific detectors consistently reach 90 to 95% accuracy on independent benchmarks, against 70 to 80% when general text detectors are applied to source code. But the error stories are cautionary. OpenAI shut down its own classifier because it correctly identified only 26% of AI-written text while incorrectly labeling 9% of human-written text as AI-generated. Turnitin claims under 1% false positives, yet a Washington Post study produced about 50% on a smaller sample. And one vendor found a competitor performed no better than chance at classifying known AI-generated code.

> **Watch out:** A detector flag is not a verdict
>
> At a 9% to 50% false-positive rate, auto-rejecting on a detector will lose strong human coders. Use the flag only to route the candidate to a phone-screen where you ask them to walk through the code live. Never put "AI-generated" in a written rejection.

The base-rate logic is what makes this a hard rule. If most of your applicants write their own code, a detector that flags 9% of humans will produce more false accusations than true catches. The check earns its place as an input to a phone-screen, not as a gate.

## How this goes wrong: eight failure modes

Most bad verdicts come from trusting a cheap signal. The eight failure modes below are the ones that reliably produce false positives, each with the check that catches it. This section is the most valuable part of the standard, because a model that overclaims is worse than no model.

| Failure mode | False positive | Check |
|---|---|---|
| Green-square gaming | Busy graph read as diligence | Open PRs, confirm signed commits |
| Fork inflation | High repo count read as output | Filter to authored, merged code |
| Identity spoofing | Reputable-looking history credited | Signature verification, vigilant mode |
| AI-detector over-trust | Strong human coder rejected | Treat detector as a flag only |
| Vanity metrics | Popular repo read as skill | Quality over popularity |
| Signaling bias | Job-search repos read as depth | Require corroboration, not just recency |
| No-GitHub penalty | Closed-source expert passed | Make public work one input, not the gate |
| Bootcamp-clone trap | Practice work scored as production | Distinguish clones from real problems |

Three of these deserve extra weight. **The no-GitHub penalty** is the one that quietly costs you the best hires. Not all candidates have or regularly use GitHub, so an absent profile is not absent skill - when there is no artifact, fall back to a screen, and note in the verdict that the evidence is incomplete. **Signaling bias** is the subtle one: the 22,900-developer study found that repos spun up near a job search draw fewer stars and forks, so recency is a weak signal unless a second source corroborates it. **Vanity metrics** catch experienced reviewers off guard - do not over-index on stars and followers, because popularity and depth are different measurements.

#### Reading a candidate on depth and relevance

Horizontal axis runs from Low relevance to role to High relevance to role. Vertical axis runs from Shallow contributions to Deep contributions.

| Quadrant | What it means |
| --- | --- |
| Off-topic but skilled | Phone-screen; probe transferable depth |
| Interview now | Deep work in your exact domain |
| Pass | Neither depth nor fit |
| On-topic but thin | Phone-screen; verify beyond the surface |

*The two highest-weighted dimensions decide most verdicts on their own.*

## What the gate buys you, and how to keep it honest

The payoff of a scored gate is a pass-rate shift downstream, not raw speed at the top, and that is exactly what a three-way verdict optimizes. Fewer wasted interviews is where the hours come back.

Screening is a real tax on the week. Recruiters typically spend 15 to 20 hours per week on candidate screening alone, McKinsey research puts the average at about 23 hours of screening per hire, and a single role requires 50-plus hours of internal team time across sourcing, review, interviews, and offers. Against that, the measured effect of a structured pre-interview gate is large: under a benchmark where recruiters examined 200 applications and stopped once 35 interview spots were filled, workload fell from about 160 hours to 82 because the gate raised the first-round pass rate from 34% to 54%. The lesson is not that screening got faster. It is that better people reached the interview, so fewer interviews were wasted.

**160 to 82 - Recruiter hours before and after a structured pre-interview gate**

The gate raised first-round pass rate from 34% to 54%, which roughly halved the work.

Keep the model honest with a short pre-flight before you call any verdict final.

#### Before you record the verdict

- [ ] Every score cites a specific repo or PR, not a general impression
- [ ] The strongest PR was read in the Conversation tab, not just the diff
- [ ] Flagship commits show a verified signature, not a spoofable name
- [ ] Any AI-detector flag routes to a phone-screen, never a rejection
- [ ] Recency without a second corroborating source was not scored as depth
- [ ] The pass bar was adjusted for how scarce the stack is in your market
- [ ] A candidate with no public work was screened, not silently passed

To keep the standard current, re-run your ten calibration cases whenever the market for a stack moves. Supply figures shift, so re-check the scarcity ratios in your own index before you lower a pass bar. And revisit your detector choice periodically: accuracy claims for code-specific tools change, and the only trustworthy test is whether a tool beats chance on code you already know the origin of. The model is a frame for one repeated judgement call. Its numbers are yours to calibrate; its discipline - cite the artifact, weight depth and relevance highest, and never let a flag become a verdict - is what makes two reviewers agree.

## Frequently asked questions

### How well does GitHub activity actually predict on-the-job performance?

There is no published validity figure for GitHub signals against measured job performance, so any blog claim of a specific percentage is unsourced. What is established is that public code is a form of work sample, and work-sample tests validated at r = .54 in Schmidt and Hunter (1998), revised to r = .33 by Roth et al. (2005). Treat public code as a moderately predictive sample whose value depends on how job-relevant it is, not as a proven proxy.

### Can I reject a candidate because an AI detector flags their code?

No. Detector error rates make an auto-reject reckless: OpenAI's own classifier caught only 26% of AI-written text while wrongly flagging 9% of human writing, and one study found a general detector's false-positive rate near 50% on a small sample. Code-specific detectors do better, around 90 to 95% accuracy, but still misfire. Use the flag to route the candidate to a phone-screen where you can ask them to walk through the code live.

### What if a strong engineer has no GitHub presence at all?

An absent profile is not evidence of absent skill. Many strong engineers work entirely in closed-source, so treat public work as one input, not the gate. When there is no public artifact, fall back to a short phone-screen or a take-home rather than passing outright, and note in the verdict that the score is based on incomplete evidence.

### How do I keep two reviewers from reaching different verdicts?

Score the same fixed dimensions, cite the specific PR or repo behind each score, and write the numbers down before combining them. The GitHub take-home process uses a PR scorecard and rubric precisely to limit interviewer bias. Reproducibility comes from citing artifacts, not from shared taste, so require a repo or PR link next to every score.

### Does a busy green contribution graph mean the candidate is diligent?

Not reliably. Backdating tools generate commits on about 70% of days with one to five commits each, so a dense graph can be pure script output. Read what is behind the squares by opening actual PRs and confirming signed or verified commits. A single well-documented fix for a complex bug carries more weight than dozens of automated green days.

### Should the pass bar be the same for every stack?

No. Move the threshold with supply. In Refolk's index Rust engineers are roughly 4.8x rarer than Go in the US, and Germany has about 6.7x fewer Rust engineers than the US. For scarce stacks, borderline candidates who would be a pass in an abundant market should shift toward phone-screen, or the funnel empties before you fill a slot.

---

*From the Refolk guide library. I revise these guides rather than replacing them, so the current version is always at https://www.refolk.ai/guides/public-work-interview-verdict*