Refolk
StandardProcess, data, and compliance

The Vanished-Upstream Repo Standard: Keep, Freeze, Redact, or Purge

Grade any changed upstream repo into one of eight events and four verdicts, then carry it through clones, contributor rows, emails, and exports with a log that holds up.

21 min readLast reviewed November 22, 2026Read as Markdown

Key takeaways

  • Freeze every flagged repository on detection and never Purge on a status code alone, because GitHub lets an owner restore a deleted repository for 90 days.
  • A 404 to an anonymous request cannot tell a deleted repository from one made private, and a 451 is not always DMCA, because some trade-control blocks now return 451 too.
  • A legal trigger overrides the upstream event: an erasure request must be answered within one month, extendable by two, and covers every copy you hold, including data still public in a fork.
  • A ghost author is not anonymization, because the stored git commit objects still carry the original name and email after the account is deleted.
  • A Purge is not done until a re-query by repository ID, email, and name returns nothing across clones, derived contributor rows, cached emails, and exports, and every export recipient has been told.
  • In Refolk's index there are about 148 UK Data Protection Officers for every UK Privacy Engineer (738 to 5), so the person who signs the verdict and the person who runs the deletion should be named separately.

Your refresh has flagged repositories you already cloned and mined that are now gone, private, taken down, rewritten, or tied to deleted accounts. Someone has to decide, row by row, what happens to the stored copy and every record built from it. This standard is for the recruiting ops, revenue ops, or data lead who signs that decision. It gives you eight named upstream events, four verdicts with written criteria, a propagation checklist, and a log entry built to hold up in an audit.

What counts as a vanished upstream?

A vanished upstream is any change at the source that leaves your stored copy different from what the owner now publishes. That covers deletion, a switch to private, a takedown, a history rewrite, and deletion of the account or organization behind the repository. I sort these into eight events. The taxonomy is Refolk's own framework, not something GitHub or a regulator publishes. Treat the codes as house vocabulary that lets two reviewers compare answers.

  • E1 Renamed or transferred. The repository moved. This is not an incident.
  • E2 Repo deleted. The owner deleted it. It may still be restorable.
  • E3 Made private. The owner withdrew public visibility.
  • E4 Org deleted. The owning organization is gone.
  • E5 Account deleted. A contributor's account is gone, and their work now shows as the ghost user.
  • E6 History rewritten. A force-push removed an email, a secret, or other content from the branch you mirrored.
  • E7 DMCA takedown. GitHub disabled the content under a copyright notice.
  • E8 Other legal block. GitHub blocked access for a reason other than DMCA, such as trade controls.

Six of the eight have documented GitHub behavior:

EventRefresh signalDocumented GitHub behaviorReversible?
Repo deleted404Restorable for 90 daysYes, 90 days
Made private404 to anonymous requestPublic forks split into a new networkYes, owner choice
DMCA takedown451, reason dmcaForks hit only if named, or network over 100 and alleged infringingCounter-notice
Renamed or transferred301Redirects to /repositories/{id}Not an incident
Account deletedAuthor shown as ghostComments and issues reattributed to ghostUsername reusable after 90 days
Org deleted404Not restorable (community answer)No

The "Reversible?" column is my reading of the documented behavior, not a GitHub statement. The org-deletion row rests on a community answer, not the docs, so confirm it before you act on it.

The other two events give you no clean signal. E6 returns a normal 200, because the repository still exists. E8 returns a 451, the same code as DMCA. GitHub also takes down content for exposing private information. How those takedowns treat forks is not established in the sources behind this standard. If a notice cites private information, file it as E8 and send it to counsel.

How do you read the refresh signal?

Re-request every stored repository without following redirects, and record the raw status code and body. On its own, a status code only identifies the event in two cases: a 301, and a 451 whose reason is "dmca". Everything else needs a second check before you classify.

Refresh signal triage

  1. Raw re-request
    No redirect following; store status code and body per row
  2. 301
    Follow to /repositories/{id} with auth kept; re-key and close as E1
  3. 451
    Parse the block reason; dmca goes to E7, anything else to E8
  4. 404
    Run an authenticated repo-ID lookup; if still 404, hold as E2, E3, or E4
  5. 200
    Check stored HEAD SHA reachability for E6 and ghost authors for E5
Only a 301 and a dmca-reason 451 identify an event on their own; every 404 and every 200 needs a second check.

Here is what each signal proves, and what it looks like when it misleads you:

SignalWhat it provesWhen it lies
301The repo moved, and the numeric ID survivesClient drops auth on redirect and logs a 404
451A legal block is in placeReason is not dmca: trade-control blocks also return 451
404You cannot see it anonymouslyRepo is private, not deleted
200Repo exists and is visibleHistory was rewritten under the same name

A concrete 301: GET /repos/joyent/node returns a 301 pointing to /repositories/211666. Key every stored row on that numeric ID, not the owner/name pair, and renames stop looking like deletions.

The 451 changed meaning over time. GitHub used to return a 403 for DMCA blocks, then moved to 451 with "reason": "dmca" in the body. A later REST API breaking-change release moved some trade-control blocks to 451 as well. Parse the reason every time, and re-read the API's breaking-changes page whenever you upgrade the API version you pin.

What do Keep, Freeze, Redact, and Purge mean?

Each verdict is a defined state with a test for when it is done. Keep returns the data to active use. Freeze stores it but blocks all use. Redact removes specific personal data or objects and keeps the rest. Purge deletes everything within a stated scope.

  • Keep. The data goes back into queries and exports. Proves: an authenticated re-request returns 200 at the same numeric ID, or a 301 resolves to it, and no legal trigger is on file. Lies when: the repo is visible again but its history was rewritten (E6), so compare the HEAD SHA before you call it Keep.
  • Freeze. The copy is stored intact, unqueryable, and excluded from every export, with an expiry date. This matches the ICO's definition of restriction: "you are permitted to store the personal data, but not use it." Lies when: a Frozen row still feeds a nightly export or a cached search index.
  • Redact. Named fields (names, emails, profile links) or specific objects (a rewritten commit, a secret) are removed from the clone and every derived table. Non-personal content stays. Lies when: you redact the API author field but leave the email in the stored commit objects.
  • Purge. Every record within the scope is deleted. Purge always has a scope: repository scope (all content and derived records for one repository) or person scope (every record and identifier for one person across all repositories). Done when: a re-query by repository ID, email, and name returns nothing in scope.

One limit matters for Redact. No regulator or standards body has published a numeric level at which aggregated contributor counts count as anonymized. Don't treat "it is only counts now" as anonymization without your DPO's sign-off. Read the ICO's anonymisation guidance yourself before you cite it, because it was not reviewed for this standard.

Verdict criteria when no legal trigger exists

These are the events that can reverse, or that are not yet confirmed:

CodeEventOn detectionVerdict at review
E1Renamed or transferredRe-key, no freezeKeep
E2Repo deletedFreezeFreeze to day 90; Keep if restored, else Redact
E3Made privateFreezeFreeze to day 90; Keep if public, else Redact
E8Other legal blockFreezeFreeze pending counsel

These are the events that remove data:

CodeEventOn detectionVerdict at review
E4Org deletedFreezeRedact once confirmed, else treat as E2
E5Account deletedFreeze the person's rowsRedact the person
E6History rewrittenFreeze affected objectsRedact the removed data
E7DMCA takedownFreezePurge named content once judged legitimate

Trigger overrides

Record a trigger code next to the event code. The pair is the criterion you log.

  • T0 None. Apply the event row above.
  • T1 Erasure request. Purge at person scope, across every repository, including E1 rows and repositories still returning 200.
  • T2 Restriction request. Freeze at person scope until the request is resolved.
  • T3 Objection. Freeze at person scope pending the DPO's decision. The response clock for objections was not established in the sources behind this standard, so confirm it with counsel.
  • T4 DMCA or published takedown notice naming your copy or its network. Freeze, then Purge at repository scope once counsel judges the notice legitimate.

Why does E3 end in Redact? GitHub's Acceptable Use Policies, section 7, say you may not use information from the service "for spamming purposes, including for the purposes of sending unsolicited emails to users or selling personal information, such as to recruiters, headhunters, and job boards". They also say you will "only use that personal information for the purpose for which that User has authorized it." These clauses bind anyone holding a clone. No clause specifically covers keeping data after the user restricts it upstream. That gap is real and not established either way. But making a repository private also removes it from the GitHub Archive Program, and that is the clearest signal of withdrawn authorization you will get. Software Heritage, which claims a public-interest basis, still masks first and alters names on request. A commercial recruiting dataset has a weaker basis and can claim no stronger retention right.

Reversibility against legal trigger

Legal trigger on fileNo legal trigger
Trigger, reversible
Trigger wins; Redact or Purge inside the clock even though the owner could restore
Trigger, irreversible
Purge in scope and notify every recipient
No trigger, reversible
Freeze until day 90, then re-check and decide
No trigger, irreversible
Redact personal data; keep non-personal content only
Upstream event reversibleUpstream event irreversible
A legal trigger moves a row to Redact or Purge regardless of whether the upstream could still reverse; without one, reversibility decides between Freeze and Redact.

Which clocks govern the decision?

Two families of clock run at once. GitHub's 90-day timers set how long the upstream event can still reverse. Data protection clocks set how long you have to answer a person. When both apply, the person's clock wins.

ActorTriggerPublished clock
UK GDPR controllerErasure request1 month, plus up to 2 months extension
UK GDPR controllerRestriction request1 calendar month
Software Heritage mirrorTakedown"Timely", no day count
GitHubRepo deletion90-day restore window, then purge

The GitHub timers line up, and that is the strongest argument for Freeze before Purge:

  • A deleted repository can be restored within 90 days, unless it belonged to a fork network that is not empty. It can take up to an hour after deletion before it becomes restorable.
  • On GitHub Enterprise Server, "unless a legal hold is in effect on a user or organization, after 90 days the repository is purged and deleted forever."
  • After an account is deleted, "after 90 days, the account name also becomes available to anyone else to use on a new account," and the associated email "becomes available for use with a different account."

A Purge on detection is permanent while the upstream event is still reversible. Restriction, "store, not use", is the ICO-sanctioned middle state, and it is exactly what Freeze implements.

On the data protection side, the ICO says you must respond to an erasure request "without undue delay and at the latest within one month". You can extend by two months if you give the reason inside the first month. If your ticketing system needs a fixed day count, the ICO's law-enforcement guidance suggests "a 28-day period to ensure compliance is always within a calendar month." That guidance was written for a different regime, but borrowing it as an internal deadline costs nothing.

Software Heritage's mirror policy is the closest published model for a stored copy: data removed from the main archive "should be made inaccessible, but not deleted, as soon as possible. If legitimate, data should be removed from the mirror as well." It sets no day count.

Who signs the verdict, and who executes it?

Two people own each row. The DPO and the data lead sign the verdict. A named technical executor carries it out across every system. In most organizations these are different people, and a standard that names only the signer will produce Purges that stop at the clone.

The people who own the decision rarely own the deletion. Refolk's index of professional profiles shows the imbalance:

QueryTotalExact title share, top-25 sampleTop employer
DPO, United Kingdom73813/25 = 52%NHS (4)
DPO, Germany2119/25 = 36%tts (2)
Privacy Engineer, United Kingdom55/5Meta (4)
148:1
UK Data Protection Officers per UK Privacy Engineer in Refolk's index
738 DPO title matches against 5 Privacy Engineer matches, four of whom are at Meta and one at Google.

Read the Germany figure as a floor. Many German profiles use German job titles, so the 211, and the 3.5x UK-to-Germany ratio it implies, probably undercount. The point stands either way: dedicated privacy engineering is scarce, so for most teams the executor will be a data engineer who has never run an erasure. Name that person in the log by role and by name. Give them the propagation checklist, not just a verdict.

If no one on staff has built a deletion pipeline before, you need to find one, and keyword search on job titles finds almost nobody because the title barely exists. Refolk takes the requirement in plain English and returns people whose work history shows it, across LinkedIn, GitHub, and the open web.

The same goes for the signing side. If your team runs without a DPO, Refolk can return data protection officers at UK recruitment agencies or RPO firms, which gives you peers who can review your verdict criteria against how their own firms handle requests.

Step by step: working a flagged batch

Run every flagged row through the same nine steps in order. Detection and redirect resolution take minutes per batch. Classification takes under an hour per row. The legal check can take days, which is why the Freeze comes before it.

Vanished-upstream procedure

  1. Detect
    Re-request every stored repository without following redirects and record the raw status code and body. Done when every row carries 200, 301, 404, or 451.
  2. Resolve redirects
    Follow each 301 to /repositories/{id} with the auth header kept. Done when renames and transfers are re-keyed and removed from the incident list.
  3. Freeze first
    Make every remaining flagged copy inaccessible the same day, before any review, without deleting it. Done when the row is unqueryable and excluded from exports.
  4. Classify
    Two reviewers independently assign one of the eight event codes. Done when both chose the same code; disagreements go to the data lead.
  5. Check legal triggers
    The DPO checks for erasure, restriction, or objection requests, DMCA notices, and published takedown notices. Done when the trigger or its absence is logged with its deadline.
  6. Assign a verdict
    The DPO and data lead apply the event and trigger codes to the written criteria. Done when the verdict and the criterion are both recorded.
  7. Propagate
    The named executor applies the verdict to the clone, derived contributor rows, cached emails, and exports. Done when a re-query by repository ID, email, and name returns nothing out of scope.
  8. Notify recipients
    The ops lead tells everyone the erased data was shared with about the erasure, inside the response month. Done when each notification is dated in the log.
  9. Log and re-check
    Close the entry and schedule a re-check after GitHub's 90-day window for every deleted or private repo still Frozen. Done when the re-check has a date and an owner.

The day-90 re-check needs one caveat. Holding a Freeze until day 90 only makes sense when no erasure request is pending. If a T1 arrives on day 10, the person's one-month clock governs and the 90-day timer no longer matters for their data.

How do you propagate a verdict to every copy?

Propagation means applying the verdict to every layer that holds data derived from the repository, outermost first, then proving it with a re-query. A verdict applied only to the clone is not applied.

Where a vanished repo's data lives

  1. Downstream exports
    CSVs, CRM pushes, and shared lists already sent to recipients
  2. Derived contributor rows
    Person records, scores, and links built from commit authorship
  3. Cached emails
    Commit-author emails lifted into contact or enrichment tables
  4. Stored clone
    Git objects, including commit metadata and rewritten history
Work from the exports inward, because the outer layers are the ones that leave your control.
  • Downstream exports. Find every export that included the repository or the person since first ingest. The ICO says organisations "should also tell anyone else they have shared your data with about the erasure." Notification is part of the job, not a courtesy.
  • Derived contributor rows. Join on your internal person ID, not the GitHub login. The login can be reissued to someone else 90 days after an account is deleted.
  • Cached emails. Search by the literal email string across every table. The same email can be reassigned to a different GitHub account, so never re-match it upstream after a Redact.
  • Stored clone. For Redact under E6, remove the specific objects. For Purge at repository scope, delete the clone and any packed copies you made.

Fork survivors are part of scope. After a repository goes private or is deleted, its public forks persist, so the same personal data may still be upstream somewhere. That does not cure an erasure request. If you cloned any fork from the network, that copy is in scope too.

Use one log entry per row. Key it on internal IDs, because a single shared ghost user cannot tell two deleted contributors apart. GitLab issue 35890 documents how that breaks audit trails.

Vanished-upstream log entry
LOG ID:
REPOSITORY ID (numeric):
LAST KNOWN OWNER/NAME:
DETECTED ON (date):
RAW STATUS / BODY REASON:
REDIRECT RESOLVED TO (if 301):
FROZEN ON (date):
EVENT CODE, REVIEWER A (E1-E8):
EVENT CODE, REVIEWER B (E1-E8):
AGREED EVENT CODE:
TRIGGER CODE (T0-T4) AND RECEIVED ON:
RESPONSE DUE (internal 28 days / statutory 1 month):
VERDICT (Keep / Freeze / Redact / Purge):
SCOPE (repository / person, internal person ID):
CRITERION APPLIED (event row + trigger override):
VERDICT SIGNED BY (name, role):
EXECUTED BY (name, role):
LAYERS DONE: exports [ ] contributor rows [ ] cached emails [ ] clone [ ] forks held [ ]
RE-QUERY EVIDENCE (query run, date, rows returned):
RECIPIENTS NOTIFIED (who, date):
RE-CHECK DUE (day 90, if Frozen E2/E3):

One entry per repository row; add a person-scope entry for each T1, T2, or T3 request.

How does this go wrong?

Most failures come from reading a status code as proof of something it does not prove, or from stopping a deletion one layer too early. Each failure mode below has a check you can run before you sign a verdict.

1. A 301 read as a deletion

HTTP clients that don't follow redirects, or that drop the auth header on redirect, turn renames into errors or 404s. Discourse and tty7 have both shipped fixes for exactly this. Check: re-request without following redirects and inspect the Location header before anything is classified.

2. A 404 read as a deletion when the repo went private

To an anonymous request, a private repository and a deleted one look the same. Purging on day one destroys a copy the owner could restore within 90 days. Check: run an authenticated repo-ID lookup, and Freeze rather than Purge until day 90 unless a legal trigger exists.

3. A 451 assumed to be DMCA

Trade-control blocks now return 451 too. Treating an E8 as an E7 sends you to the wrong process and the wrong verdict. Check: parse block.reason on every 451.

4. A ghost author treated as anonymized

The UI shows the ghost user, but the commit still carries the name and email in its git metadata. GitHub falls back to "the nickname on the commit". Check: grep the stored commit objects for the email, not just the API author field.

5. A force-push rewrite missed

The repository still returns 200, but the email or secret is gone from the default branch while it remains in your clone. Check: confirm whether your stored HEAD SHA is still reachable upstream. GitHub has not documented a detection method in the sources behind this standard, so test this heuristic on a known rewrite before you rely on it.

6. A Purge that stops at the clone

The clone is deleted, but derived contributor rows and exports survive, and the person is still in your CRM. Check: re-query by email across every table, then notify export recipients as the ICO requires.

7. Fork survivors ignored

After a repository goes private or is deleted, public forks persist, so the personal data is still upstream. That is not a defense against an erasure request, and it is not a reason to Keep. Check: treat any request as covering every copy you hold, including forks you cloned without the parent.

8. A network-wide DMCA notice missed

A single notice can reach your whole clone set. GitHub processes networks over 100 repositories together when the submitter alleges network-wide infringement. One notice covered a network of 102K repositories, and another covered 4,195. Small networks can be fully covered too: one full-network takedown reached just 8 repositories because the submitter listed every known fork. Check: for every E7, read the notice in the public github/dmca archive and match its scope against every fork you hold from that network.

9. Old logins and emails re-matched to new people

This one is my inference from GitHub's account rules, not a documented incident. Deleted usernames become available after 90 days, and the associated email can be attached to a different account. A refresh that re-matches by login or email can attach a stranger's new profile to a deleted contributor's history. Check: never re-resolve a Redacted or Purged identifier upstream, and key all person rows on internal IDs.

A Purge is finished when your own re-query returns nothing, not when the upstream goes quiet.

How to keep the standard current

Treat this standard as a policy with dependencies you re-verify, not a page you wrote once. The facts it rests on are GitHub's status codes and retention windows, the ICO's clocks, and the scope rules for DMCA notices. Each can change, and each has a primary page you can re-read.

  • Every API version upgrade: re-read GitHub's REST API breaking-changes page for any new meaning of 451 or 301.
  • Every policy review: re-read the ICO erasure and restriction pages, which are flagged as under review, and GitHub's Acceptable Use Policies, section 7.
  • Every E7: read the notice in github/dmca rather than inferring its scope from the status code.
  • Every day-90 re-check: re-run the authenticated lookup and move each row to Keep or Redact. Don't let a Freeze run past its expiry by default.
  • When a name-change request arrives: treat it as a T2 at minimum. Robyn Speer's public account of asking Software Heritage to change the name attached to the ftfy package shows the request is real, and that even archives get it wrong.

Before you mark any batch closed, verify every item below.

Before the batch is closed

  • Every row was re-requested without following redirects and has a raw status recorded.
  • Every 301 was resolved to /repositories/{id} with auth kept and re-keyed on the numeric ID.
  • Every 451 has its block reason parsed and is coded E7 or E8 accordingly.
  • Every flagged row other than E1 was Frozen on the day of detection.
  • Two reviewers independently chose the same event code for every row.
  • The legal trigger, or its documented absence, is logged with a response date inside 28 days.
  • Each verdict names its criterion as an event code plus a trigger code.
  • Each verdict has a named signer and a separately named technical executor.
  • Stored commit objects were searched for every redacted or purged email, not only the API author field.
  • A re-query by repository ID, email, and name returned nothing out of scope across clones, contributor rows, cached emails, and exports.
  • Every forked copy from the same network that you hold was brought into scope.
  • Every export recipient of erased data was notified, with the date logged.
  • Every Frozen E2 or E3 row has a day-90 re-check scheduled with an owner.

Questions practitioners ask

What should I do with a cloned repo after it is deleted on GitHub?

Freeze it, don't delete it. Make the copy unqueryable and pull it from exports the same day, but keep it stored. GitHub lets an owner restore a deleted repository for 90 days, unless it belonged to a fork network that is not empty, so a Purge on day one can't be undone even though the upstream deletion still can. Re-check after day 90. If the repository is back, Keep it. If it is still gone, Redact the personal data. An erasure request overrides all of this and means Purge.

If a GitHub repo is made private, do I have to remove it from my dataset?

Under this standard, yes: Freeze straight away, and Redact the personal data if the repository is still private after the 90-day window. An anonymous request can't tell a private repository from a deleted one, which is why both run on the same timer. GitHub's Acceptable Use Policies limit personal information collected from the service to the purpose the user authorized. Going private is the clearest withdrawal signal you will get, even though no clause names it specifically.

Does a DMCA takedown on GitHub apply to my mirror copy?

Treat it as applying whenever the notice names your copy or covers the whole fork network your copy came from. GitHub only disables forks named in the notice, unless the network has more than 100 repositories and the submitter alleged that most forks infringe. One such notice reached a network of 102K repositories. Freeze on the 451, check the public github/dmca archive for the notice's scope, then Purge the named content once counsel judges the notice legitimate.

How do I handle a contributor erasure request when their data is in commit history?

Purge at person scope: every record and identifier for that person, in every repository you hold, including repositories that still return 200. Respond within one month. You can extend by two months if you give the reason inside the first month. Search the stored commit objects for the email, not just the API author field, and tell export recipients. The data still being public in a fork is not a defense against the request.

A repo force-pushed its history to remove an email, but my clone still has it. What now?

Classify it as E6, history rewritten, and Redact the removed email or secret from your stored objects and every derived row. The repository still returns 200, so status codes won't catch this. Check instead whether your stored HEAD SHA is still reachable upstream. GitHub has not documented a detection method in the sources behind this standard, so treat that check as a local heuristic and test it before you rely on it.

What retention policy should I have for scraped GitHub data?

Adopt a written per-event policy rather than a single retention period. Freeze on detection, use the eight event codes and four verdicts, and add trigger overrides for erasure, restriction, objection, and takedown. Bind it to GitHub's Acceptable Use Policies, which forbid selling personal information to recruiters and limit use to the purpose the user authorized. Log every verdict with the criterion it rests on, a named signer, and a named executor.

Try it on the search you came here for

Stop building boolean strings. Just describe the person.

Type one sentence. I plan the search, read GitHub, public LinkedIn and Crunchbase records, and the open web as it is right now, and hand back a ranked list with the reason next to every name.

  1. 01Describe them

    One plain sentence. Role, city, stack, stage, whatever matters to you.

  2. 02I read the web live

    GitHub, public LinkedIn and Crunchbase records, the open web. Not a database that went stale last quarter.

  3. 03You read the shortlist

    Ranked, with the reasoning under every name. Open a profile, ask a follow-up, narrow it down.

  • No boolean, no filters, no seat to buy. One box.
  • Read at search time, so a profile updated yesterday counts today.
  • Every step visible as it runs, every name with its reason.

500 free credits on sign-up. No card, no demo call. See real searches.

Read next