# The Outside-Contributor PR Triage Read: Review, Ask for Proof, or Close

*Score any inbound outside PR on five public dimensions in under five minutes, land it on Review, Ask for Proof, or Close with a reason, and decide who earns bypass.*

- Canonical URL: https://www.refolk.ai/guides/outside-contributor-pr-triage-read
- Pillar: Engineering and open source
- Format: Framework
- Published: 2026-10-30
- Last reviewed: 2026-10-30
- Reading time: 21 min
- Keywords: how to triage AI generated pull requests, how to tell if a pull request is AI slop, first-time contributor pull request review policy, who should be on the pull request limit bypass list, signals of a bot or agent GitHub account opening PRs

## Key takeaways

- Score every outside pull request on five public dimensions (account history, cross-repo PR behavior, issue linkage, diff-to-claim fit, and a human question), 0 to 2 each, and never close on account age alone.
- In an LF Research survey of 234 contributors and maintainers, maintainers estimated that about 1 in 4 incoming pull requests is AI slop, rising to 31% in large projects.
- Trust built inside one repo is the easiest kind to farm: GitHub's Actions approval rule and pr-slop-stopper both exempt anyone with a single prior merge, and the xz attacker spent about six months on harmless patches before applying pressure.
- Merger diversity of 2 or more independent maintainers, counting only merges made by someone other than the author, is the best public stand-in for spotting a history made entirely of self-merges.
- GitHub's open-PR limit counts only open, non-draft PRs per repo from users without write access, so drafts and bursts across many repos must be checked by hand, and bypass should require merges reviewed by more than one maintainer.
- Refolk's index holds only 61 US profiles with open-source titles against 300 or more in developer relations, so the triage read should be written down well enough to survive whoever is on rotation.

A pull request just landed from an account nobody on the team recognizes, and someone has about five minutes to decide what happens to it. This guide is for the maintainer, engineering manager, or devrel lead making that call. It gives one scored read per PR across five public dimensions, lands each PR on Review, Ask for Proof, or Close with a reason, and sets the bar for the repo's trusted-contributor bypass list.

## What does the outside-PR triage read decide?

The read picks one of three outcomes for a pull request from an account you don't recognize. **Review** means a human reviewer is assigned now. **Ask for Proof** means the author must answer one specific question before review. **Close** means the PR is closed with a stated reason and a way back. A separate, slower decision sits on top: whether the author has earned a place on the bypass list that exempts them from the repo's open-PR cap.

The volume problem is real. In LF Research's survey of 234 contributors and maintainers, maintainers estimated that roughly 1 in 4 incoming pull requests is "AI slop" (low-quality AI-generated content). In large projects the share rises to 31%. The survey's exact cutoff for "large" is not confirmed. One secondary summary ties it to projects with more than 50 contributors. Coolify's maintainers were closing more than 120 low-quality AI-generated PRs by hand every month.

Policy has not caught up:

- Only 35% of projects have a policy permitting AI-assisted contributions.
- 32% have no policy and none planned.
- 53% of respondents disagree that the technology has improved trust in their communities.

This read is not an AI detector. It asks whether the author understands the change and stands behind it, and that question matters whether or not a model wrote the first draft. In a matched sample of 567 PRs, agent-authored PRs were accepted 83.8% of the time against 91.0% for human-written ones, with similar review times. Agent output is not junk by default. A read that closes everything that looks generated will close good work and drive off real first-time contributors.

Current public advice tends to be a policy template (disclose AI use, open an issue first) or a bot that prints an account-trust score. Neither tells the person doing the review how to weigh a 42-day-old account with two self-merged PRs against a clean, issue-linked diff. This read does, and the worked example below scores exactly that case.

## The five dimensions: what each proves and how it lies

Five public dimensions make up the read: account history, cross-repo PR behavior, issue linkage, diff-to-claim fit, and responsiveness to a human question. Score each one 0, 1, or 2. The first four are static and take under five minutes together. The fifth is asynchronous and is only scored once you have asked.

| Dimension | What it proves | How it lies |
|---|---|---|
| Account history | The account existed before this PR | Real newcomers score low; patient attackers score high |
| Cross-repo PR behavior | Other maintainers accepted this author's work | Self-merges and farmed typo fixes inflate it |
| Issue linkage | The change answers a known need | Author opens the issue minutes before the PR |
| Diff-to-claim fit | The diff does what it says, nothing else | Patches written to build credibility fit perfectly |
| Human question | The author understands this specific change | A fluent, generic agent answer passes a lazy question |

### Account history

Check the account's age, profile, and contribution calendar. Score it by age band:

- **0:** under 30 days
- **1:** 30 to 90 days
- **2:** over 90 days

The 30-day line matches anti-slop's default. Its docs note that raising the gate to 60 or 90 days gives stronger protection but may affect legitimate new GitHub users. Age proves only that the account was not created for this PR. It is wrong in both directions. A genuine first-timer fails it by definition. The JiaT75 account behind the xz backdoor was created January 26, 2021, and its first xz patch arrived October 29, 2021, so it would have passed every age gate in use today. This is the weakest dimension, and it can never close a PR on its own.

### Cross-repo PR behavior

Collect three things:

- **Merge rate across GitHub:** merged PRs as a share of merged plus closed-unmerged. Count only merges made by someone other than the author.
- **Merger diversity:** the number of distinct maintainers who have merged the author's PRs.
- **Burst check:** whether the author opened many PRs across unrelated repos in a short window.

Score **2** if the merge rate is at or above 30% and at least 2 maintainers who are independent of each other have merged the author's work. Score **1** if only one of those holds. Score **0** if neither holds, or if you see a burst. The 30% line matches anti-slop's default global merge ratio and the example config for contributor-report. The diversity of 2 matches contributor-report's example. No public tool I found publishes a threshold for histories made only of self-merges. Merger diversity is the nearest proxy, which is why self-merges must be left out.

GitHub's open-PR cap does not help here. It applies per repository, and GitHub itself says the cap does nothing when someone opens pull requests across hundreds of repositories at once. This check stays manual. It is also the slowest part of the read, because you have to work out whether the maintainers who merged someone's work are actually independent of them. [Refolk](/) brings back a person's public footprint across GitHub, LinkedIn, and the open web in one request, so you can see employer and org overlap without opening a dozen tabs.

### Issue linkage

- **2:** the PR links an issue that a maintainer has triaged
- **1:** the PR links an issue that nobody has triaged, or one the author opened just before the PR
- **0:** no linked issue for a non-trivial change

A triaged issue shows that someone on the team agreed the problem exists. Nobody has published a measured effect of issue-first rules on drive-by PR volume, so treat this as a signal, not a gate. Linkage is easy to fake by opening an issue minutes before the PR. Check the issue's timestamp and who acknowledged it.

### Diff-to-claim fit

Read the description first, then the diff.

- **2:** the diff does exactly what the description says, with no unrelated files
- **1:** the core change matches, but there is drift: formatting churn, stray files, or claims slightly beyond what the code does
- **0:** the description claims behavior the diff does not implement, or the diff touches substantial unrelated files

A good fit shows that someone produced a coherent change. It does not show intent. The xz credibility-building patches were clean and did what they said. That is why a perfect fit should always come with a look at who is pushing for the merge.

### Responsiveness to a human question

- **2:** the answer addresses the specific line or failing case you asked about, and it is correct
- **1:** fluent but generic, restating the description or answering a nearby question
- **0:** no reply within the stated window, or a wrong answer

This is the only dimension that tests understanding directly. It fails when the question is lazy. A broad question like "can you explain the change?" lets a generic answer score as a pass.

#### History versus diff fit

Horizontal axis runs from Thin or self-merged history to Independent cross-repo history. Vertical axis runs from Poor diff-to-claim fit to Tight diff-to-claim fit.

| Quadrant | What it means |
| --- | --- |
| Likely newcomer | Ask for Proof, then Review on a specific answer |
| Known-good shape | Review, and confirm a human actually comments |
| Drive-by noise | Close with a stated reason and a way to reopen |
| Credible author, odd diff | Ask for Proof and check who is pushing the merge |

*A thin history with a tight diff is where genuine newcomers land, and those PRs should almost never be closed.*

## Published trust thresholds you can borrow

Four public tools or rules publish thresholds for some mix of account age, merge rate, merger diversity, and earlier merges in the same repo. None publishes a threshold for histories made only of self-merges or for repos created per day. Use their defaults to calibrate your own judgement, not as verdicts.

| Tool | Account age | Merge gates | Prior repo merge |
|---|---|---|---|
| anti-slop | 30 days default | 30% global merge ratio | Configurable |
| contributor-report | 0 default, 30 in example | 30% rate, diversity 2 (example) | Not applicable |
| pr-slop-stopper | -20 to +15 points | None published | Exempt from scoring |
| GitHub Actions approval | "New to GitHub", undefined | None | Exempt after any merge |

Some context on each:

- **anti-slop** ships 34 rules and runs them all at once.
- **contributor-report** defaults every threshold to 0, so it does nothing until you configure it.
- **pr-slop-stopper** scores from -100 to +100. It warns at -10 and closes at -40.
- **pr-slop-gate** labels at 0.5, comments at 0.65, and closes at 0.85. Auto-close is off by default.

Two of the four exempt anyone who already has a single merge in the repo. That is the most important line in the table. Trust built inside one repo is the weakest kind and the easiest to farm, and the bypass section below comes back to it.

> **Rule:** Never close on account history alone
>
> A low age score must be paired with a second zero, on cross-repo behavior, issue linkage, or diff fit, before it can contribute to a Close.

### How the read turns scores into an outcome

These cut lines are my proposal. They have not been measured. Add up the four static dimensions for a total from 0 to 8.

- **Review:** static total 6 to 8, and diff fit is 2.
- **Ask for Proof:** static total 3 to 5. Also use this when diff fit is 1, or when the static total is high but unfamiliar accounts are pushing for the merge.
- **Close:** static total 0 to 2, or diff fit of 0 alongside another 0. An account-history 0 counts toward a Close only if another dimension is also 0.

After an Ask, a specific answer (2) moves the PR to Review. A generic answer (1) gets one sharper follow-up, and the PR closes if the second answer is still generic. No reply within the window (0) means Close, with an invitation to reopen.

**Worked example.** A 42-day-old account with two self-merged PRs opens a clean diff linked to a triaged issue:

- Account history scores 1.
- Cross-repo scores 0, because self-merges count toward neither the merge rate nor diversity.
- Issue linkage scores 2.
- Diff fit scores 2.

The static total is 5, which means Ask for Proof, not Close. If the author answers the line-level question specifically, the PR goes to Review. This is the path a real first-time contributor should take.

**Triage read log line**

```
PR: #[number]   Author: @[handle]   Read by: @[maintainer]
Disclosure: declared / undeclared / unknowable
Account history: [0-2]   (age band: <30 / 30-90 / >90 days)
Cross-repo: [0-2]   (merge rate excl. self-merges: [x]%   independent mergers: [n]   burst: yes/no)
Issue linkage: [0-2]   (issue #[n], triaged by @[maintainer] / untriaged / none)
Diff-to-claim fit: [0-2]   (unrelated files: yes/no)
Static total: [0-8]
Human question: asked yes/no   answer: [0-2] / pending / lapsed
Commenters pushing for merge checked: yes/no
Open drafts by author in this repo: [n]
Outcome: Review / Ask for Proof / Close
Reason: [one sentence the author will see]
Bypass candidate: no / watch / yes   (merges reviewed by: @[a], @[b])
```

*Paste as a maintainer-only comment or into the triage tracker; fill every field even when the outcome is obvious.*

## How to run the read in five minutes

Run the cheap checks that ignore the content first, then rate the diff, then ask your question within the five-minute window. The time budgets below are proposals and have not been measured. Treat them as a pace guide, not a promise.

#### The five-minute outside-PR read

1. **Check intake and disclosure** - Record whether AI use is declared, undeclared, or impossible to know. Context only, about 30 seconds.
2. **Band the account history** - Put the account in a band (under 30, 30 to 90, or over 90 days) from its profile and calendar, about 60 seconds.
3. **Measure cross-repo PR behavior** - Get the merge ratio excluding self-merges, count independent mergers, and check for bursts across repos, about 60 seconds.
4. **Confirm issue linkage** - Record whether the linked issue is triaged, untriaged, or missing, and who acknowledged it, about 30 seconds.
5. **Rate diff-to-claim fit** - Compare description to diff and flag unrelated files or claimed behavior that is missing, about 90 seconds.
6. **Ask the human question** - Within five minutes, post one question only someone who understands this change can answer, with a stated reply window.
7. **Land, log, and decide bypass** - Record the outcome with scores and a reason the author can see, and consider bypass only after repeated merges reviewed by more than one maintainer.

Sources disagree on the order. pr-slop-stopper skips scoring anyone with a prior merge in the repo, which skips steps 2 and 3. anti-slop runs its 34 checks at once. The Good Egg scorer ignores content entirely. I keep the checks that ignore content first because they are fast, but step 5 always runs. A strong history never excuses a diff that doesn't match its description.

Step 6 behaves differently depending on the band. For a PR in the Review band, the assigned reviewer's first comment does the job. For the Ask band, the question is the whole outcome. For the Close band, skip it and write the reason.

> **Note:** Disclosure is context, not a score
>
> Among projects with policies, 76% mandate AI disclosure, but the Linux kernel's Assisted-by trailer is already being removed from its documentation. Record disclosure, then score the work.

The question has to be one that only understanding can answer. Pick a specific line and an input, state, or failing case that the description does not mention. If the answer is in the PR text, an agent can find it.

**The Ask for Proof comment**

```
Thanks for the PR. Before I assign a reviewer, one question about the change itself:

In [file]:[line], what happens when [specific input or state the description does not cover]? If you can, point to the test that covers it, or describe the failure you would expect without your change.

I'll check back in [N] days. If I haven't heard from you by then I'll close this to keep the queue clear, and you're welcome to reopen with the answer.
```

*Replace the bracketed parts with a real line and a case absent from the description; keep it to one question.*

## Who belongs on the pull request limit bypass list?

Only authors with repeated merges in this repo, reviewed by more than one maintainer, including some substantive diffs. Someone whose only credential is one accepted typo fix does not qualify. The bypass list exempts trusted contributors from GitHub's open-PR limit without giving them full contributor access.

### How the limit works

- It is opt-in, and I found no documented default value.
- It applies only to users without write access.
- Only open, non-draft pull requests count.
- PRs opened by Copilot or another AI agent count toward the limit.
- Bypass exempts someone from the limit but does not grant write access.
- A separate setting can disable pull requests entirely or restrict them to collaborators.
- A third-party PR reports that the limit has no REST API. Check this before you script anything against it.
- GitHub describes two features as planned: a setting that counts drafts, and automatic bypass based on trust signals such as account age, number of merged PRs in the repository, and organization status.

Those planned automatic-bypass signals are the cheapest ones to fake. Account age can be waited out, and merges in one repo can be farmed. If automatic bypass ships, set it to require independent reviewers, not just a merge count.

Track record is a legitimate signal, which is exactly why it attracts gaming. In one study, agent PR acceptance was 37% when the repository had never seen the agent before, rising to 70% after 20 or more prior merges. Maintainers already lean on history. The job is to test whether that history is independent.

#### Trust evidence, cheapest to fake first

1. **Account age and profile** - Anyone can wait out a 30-day gate or fill in a bio
2. **One merge in this repo** - A single typo fix clears Actions approval and pr-slop-stopper
3. **Cross-repo merge rate** - Inflated by self-merges unless you count only merges by others
4. **Independent merger diversity** - Needs two or more unrelated maintainers to accept the work
5. **A specific answer on this diff** - Only someone who understands the change can give it

*The deeper the layer, the more it costs an attacker or an agent to fake, so weight the deeper layers more heavily for bypass.*

> **Watch out:** One merged typo is not trust
>
> GitHub's own docs warn that a malicious user could get past first-time-contributor approval by getting a simple typo or other innocuous change accepted. Don't let the same move earn bypass.

Before adding someone, look at where their merges came from. If you want to calibrate what a real multi-maintainer history looks like in a large project, pull a reference set of people who have one:

Ask me this: `Engineers in the US who have had pull requests merged into Kubernetes by at least two different maintainers in the last year` - [run the search](https://www.refolk.ai/start?q=Engineers%20in%20the%20US%20who%20have%20had%20pull%20requests%20merged%20into%20Kubernetes%20by%20at%20least%20two%20different%20maintainers%20in%20the%20last%20year).

*Returns named engineers with their GitHub and professional profiles, which gives you a baseline for what independent merger diversity looks like in practice.*

## How the read goes wrong: failure modes and false positives

The read fails in seven known ways. Most of them either punish a genuine newcomer or let a patient attacker through. Each one below comes with the check that catches it.

1. **Account age punishes genuine first-timers.** A sparse new account can trip enough default checks to be closed with no AI involved. First-timers are not the problem: in one study of security-related AI PRs, first-time contributors had higher odds of being merged (odds ratio 4.27). That sample is narrow, so don't generalize the number, but don't assume the opposite either. *Check:* never close on age alone. Require a second negative signal.
2. **Merger diversity gives a false negative.** Good Egg scores Guillermo Rauch MEDIUM on Next.js, his own company's repo, because he has zero merged PRs there. Owners and staff often have thin PR histories in their own repos. *Check:* look at merges in other repos by maintainers who are independent of each other.
3. **The bypass list becomes a backdoor.** One merged typo turns off Actions approval permanently. The xz attacker spent about six months on harmless patches, from October 2021 until the first pressure email on April 22, 2022. *Check:* require merges reviewed by more than one maintainer and some substantive diffs before bypass.
4. **"Independent" reviewers are sock puppets.** Approvals or pressure comments come from new accounts with little footprint. In the xz case, the accounts named Jigar Kumar and Dennis Ens pushed for the maintainer to hand over control, and Jigar Kumar "is never seen again." *Check:* score the commenters, not just the author.
5. **The human question gets an agent's answer.** A fluent, generic reply is the false positive here. *Check:* ask about a specific line or a failing case that the description does not mention, and score a generic answer as 1, not 2.
6. **Drafts slip past the cap.** Drafts don't count toward the open-PR limit, so an author can keep work in flight without ever reaching it. *Check:* sort the author's open drafts and log the count.
7. **"Review" turns into an unreviewed merge.** In a hand inspection of merged agentic PRs, most showed no feedback loop at all. *Check:* Review must mean a named human actually commented before merge.

**79.1% - Merged agentic PRs with no observable feedback loop**

Of 364 merged agentic PRs inspected by hand in the AIDev study, 288 were merged without any observable back-and-forth between reviewer and author.

Failure modes 3 and 4 share a lesson. Both xz signals sat outside the diff. The patches fit their descriptions, and the account was old enough. What gave the attack away was who was pushing for the merge and how little history those accounts had.

> Score the people pushing for the merge, not only the person who wrote it.

## Who sits in the triage chair?

Mostly developer relations staff and senior engineers, not people with open-source job titles. In Refolk's index of professional profiles, open-source titles barely exist as a hiring category. That makes the read's main risk inconsistency between reviewers, so write it down.

| Title family (US) | Total profiles | Exact-title share of 25 sample | Noise observed |
|---|---|---|---|
| Developer relations | 300 | 84% (21 of 25) | 1 off-target title |
| Open-source titles | 61 | 16% (4 of 25) | Mostly founders, CTOs, senior SWEs |
| DevRel to OSS ratio | 4.9x | Not applicable | Not applicable |

**61 - US profiles with open-source titles in Refolk's index**

These are titles like Open Source Engineer, Maintainer, Program Manager, or Head of, and only 4 in a 25-profile sample held the exact title Open Source Engineer.

The US developer relations figure may be capped at 300, so the ratio is at least 4.9x and could be higher. The same pattern holds in another market:

| Market | Total profiles | Top title (n of 25 sample) | Example employers |
|---|---|---|---|
| United States | 300 (possibly capped) | Developer Advocate (15) | Stripe, Google, Roblox |
| Germany | 34 | Developer Advocate (16) | JetBrains, SAP, Siemens |
| US to Germany ratio | 8.8x (lower bound) | Not applicable | Not applicable |

Berlin accounts for 9 of the 25 profiles in the German sample. The claim that triage falls mostly to developer relations and staff engineers is my inference from these counts. It has not been measured directly. It matters because large projects see more slop (31% against 25% overall), and the people triaging that volume are rotating through it alongside other work. Three practical consequences follow:

- **Write the read down.** The log line and the Ask comment are the parts that survive a handoff.
- **Calibrate across reviewers.** Have two people score the same handful of PRs independently and compare where they land.
- **Staff the rotation from developer relations titles.** If you are hiring or borrowing reviewers, searching for open-source titles will turn up mostly founders. Refolk lets you ask for developer advocates who maintain or review community repos by name and place, which is the pool that actually does this work.

## Keeping the read current

The read depends on three things that change without warning: GitHub's PR controls, tool defaults, and disclosure conventions. Re-check each one whenever you review the triage policy, and note the date you checked in the policy file.

- **GitHub's limit and bypass settings.** Planned features, including counting drafts and automatic bypass from trust signals, would change steps 3 and 7. Check the changelog and the repository's interaction-limits settings directly rather than relying on this guide's description.
- **Tool defaults.** anti-slop, contributor-report, pr-slop-stopper, and pr-slop-gate all publish their thresholds in their docs. If you borrow a cut line, re-read the source before you change your own.
- **Disclosure conventions.** 76% of projects with policies mandate disclosure, and 76% require contributors to personally test and review AI-assisted code. The kernel's template is being withdrawn. Keep disclosure as context so the score does not depend on a trailer.
- **The bypass list itself.** Review it on a schedule. Remove anyone whose recent merges were all reviewed by a single maintainer.

#### Before you call the read done

- [ ] All four static dimensions have a recorded score from 0 to 2.
- [ ] Any Close cites at least two zero-scored dimensions, and account history is not the only one.
- [ ] The Close comment states a reason the author can read and explains how to reopen.
- [ ] The Ask for Proof question names a specific line or case that the description does not mention.
- [ ] Accounts commenting or approving in favor of the merge have been checked for age and footprint.
- [ ] The author's open drafts in this repo have been counted and logged.
- [ ] A Review outcome names a human reviewer who will comment before merge.
- [ ] Every bypass addition shows merges reviewed by more than one maintainer, including at least one substantive diff.

## Frequently asked questions

### How do I tell if a pull request is AI slop?

Don't try to prove who or what wrote it. Score whether the author understands the change and stands behind it. In practice two checks do most of the work. First, does the diff do exactly what the description claims, with no unrelated files? Second, can the author answer a question about a specific line or a failing case that the description never mentions? A fluent but generic reply is the usual sign of slop, whether or not a model was involved.

### Should I close pull requests from brand-new GitHub accounts?

No, not on age alone. A sparse new account can trip enough default checks to be closed with no AI involved at all, and anti-slop's own docs say stricter age gates may affect legitimate new users. Require a second negative signal before closing, such as a diff that doesn't match its description. In one narrow study of security-related AI PRs, first-time contributors actually had higher odds of being merged.

### Who should be on the pull request limit bypass list?

Authors with repeated merges in your repo that were reviewed by more than one maintainer, including some substantive diffs. An accepted typo fix is not enough, because one innocuous merge is exactly what attackers farm. Bypass only exempts someone from the open-PR cap and does not give them write access, but it does take away a brake on volume. Review the list on a schedule and remove anyone who has gone quiet.

### Do draft pull requests count toward GitHub's open PR limit?

No. GitHub counts only open, non-draft pull requests, and only from users without write access. That means an author can keep many drafts open without reaching the cap. GitHub has described a setting to count drafts as planned, so check the current repository settings before you rely on either behavior. Until then, sort the author's open drafts as part of the cross-repo check.

### What are the signals of a bot or agent GitHub account opening PRs?

Look for many PRs across unrelated repositories in a short window, a merge history made of self-merges with no independent maintainers, and generic answers to specific questions. GitHub's open-PR limit applies per repository and does nothing about someone opening PRs across hundreds of repos at once, so this check has to be done by hand. PRs opened by Copilot or other agents do count toward the per-repo limit.

### Do I need an AI disclosure policy before using this triage read?

No. The read works with or without one, because disclosure is recorded as context and never scored. Only 35% of projects have a policy permitting AI-assisted contributions, and conventions are changing: the Linux kernel's Assisted-by trailer is being withdrawn from its documentation. A read that depends on a disclosure trailer being present will break when the convention shifts.

---

*From the Refolk guide library. I revise these guides rather than replacing them, so the current version is always at https://www.refolk.ai/guides/outside-contributor-pr-triage-read*