The Breaking-Change Blast Radius Playbook: Planned Diff to Sent Notices

This playbook is for the maintainer about to remove or change a public API, and for the engineering manager, developer-relations lead, or technical sourcer helping them get it out the door. It takes one planned breaking change and produces four artifacts: a deduplicated list of dependents, a confirmed-break subset backed by call-site evidence or a baseline-versus-candidate test run, a ranked notify list with a reachable human for each project, and a deprecation window set from measured upgrade lag. It is also plain about what no public source can show you: private repositories and application-level consumers.

What is a breaking change's blast radius, and how much of it can you see?

The blast radius of a breaking change is the set of downstream projects that will fail to install, build, or pass tests once your release ships. You can only measure the public part of it, so the goal is a defensible lower bound with evidence attached to every row, not a complete census.

A few terms used throughout:

• Dependent (or reverse dependency): a project that declares your package as a dependency.
• Call site: a line of code in a dependent that actually uses the symbol you are changing.
• Baseline-versus-candidate run: testing each dependent twice, once against your released version and once against your development version, and reporting only the difference.
• Library residue: the share of projects still using an older version of a library after a newer one exists.
• Deprecation window: the time between announcing a removal and shipping the release that removes it.

Each source you will use sees a smaller slice than the one before it. The work is narrowing from the widest slice you can list to the slice you can prove.

The outermost layer is the honest limit of this playbook. On GitHub Enterprise Server, GitHub does not calculate dependents at all, so anything internal to a company running it is invisible. Application code that never gets published as a package rarely appears in registry metadata either. Your notice plan has to include a public announcement for that invisible layer, not just direct messages to the rows you found.

Which sources list who depends on my library?

No single source lists all your dependents. Pull from the GitHub dependency graph, your registry's reverse-dependency data, and ecosystem-specific revdep tooling, then merge, because each source misses different projects.

The GitHub dependency graph

For public repositories, GitHub's dependency graph lists dependents. The "Used by" sidebar only appears when all four of these hold:

1. The dependency graph is enabled.
2. The package is on a supported ecosystem.
3. Within the ecosystem, the package links to a public repository where its source is stored.
4. More than 100 repositories depend on it.

Transitive dependencies are only partly visible. GitHub identifies indirect dependencies only if they are defined in a lock file or have been submitted through the dependency submission API. Coverage of ecosystems has widened over time, so re-check the current supported list in GitHub's dependency graph documentation rather than relying on memory.

On exporting the list: I found no documented REST or GraphQL endpoint for listing dependents, but I could not confirm that none exists. Treat it as unverified, check the current API reference yourself, and until then record dependents from the web view into your tracking sheet.

Registry reverse dependencies

Your package registry's reverse-dependency data is the source that does not depend on GitHub's thresholds. For R, CRAN runs its own reverse-dependency checks after a package passes initial checks, and revdepcheck builds its list from CRAN and Bioconductor. For scale, the r-pkgs release guide shows one check covering 231 reverse dependencies (228 from CRAN plus 3 from Bioconductor), while Yihui Xie reports knitr's revdeps reached 10,000.

What each source gives you on version usage

You will also need to know which version your dependents and users sit on. The sources differ sharply:

If you maintain a crate, check crates.io's own documentation for per-version download data before you plan around it; I have not verified what it offers.

The procedure: from planned diff to sent notices

Work through these nine steps in order, and do not skip to notices before you have evidence. Each step has a clear done state, so you can stop mid-way and resume without losing the thread.

How long it takes

I do not have measured durations for these steps, and I would distrust anyone quoting a fixed number, because the effort scales with your dependent count. What is fixed: revdepcheck caps check time at 10 minutes per package, so step 5 has a known per-row ceiling in R; and npm's per-version data covers only 7 days, so step 6 has to start the week you announce. Budget the rest by counting rows after step 3.

One sheet for the whole job

Keep every step in one table so each row carries its own evidence. This header is the shape I use:

project | repo_url | package_name | sources | symbols_used | call_site_url | baseline_status | candidate_status | flag | log_reviewed | usage_signal | contact_route | contact_type | notice_sent_date | removal_date | rerun_flag
example-lib | https://github.com/example/example-lib | example-lib | github;registry | OldClient | (file URL or "no evidence") | pass | fail | - | yes | (downloads or share) | (issue URL or address) | issue | (date) | (date) | +

The sources column matters more than it looks. A row listed by the registry but not GitHub tells you something about GitHub's coverage of your ecosystem, and a row listed by GitHub but not the registry is often an application, not a library, which changes how you notify it.

How do I confirm a dependent actually breaks?

Confirmation comes in two tiers: call-site evidence shows a dependent uses the symbol, and a baseline-versus-candidate test run shows it newly fails. Notify on test evidence where you can get it, and on call-site evidence only where you cannot.

Call-site evidence from code search

GitHub's web UI supports regular expressions in bare search terms and inside many qualifiers, written between slashes. The API is a different engine, and the difference decides how you run this step:

• The API is powered by a legacy code search engine whose results might not match github.com.
• Regex characters are simply ignored by the API.
• The API returns at most 1,000 results per query.
• One practitioner found the legacy API covers only recently searched repositories.

So step 4 cannot be fully automated through the official API. A tool that pulls API results and filters them locally can never see results beyond the API cap. Run confirmation searches in the web UI, and use the is:fork and is:archived qualifiers to separate forks and archived repositories from live projects.

An import-aware pattern ties the symbol to your package's import path instead of matching any identifier with the same name. For a Python symbol OldClient exported from mylib.client, the searches look like this:

/from\s+mylib\.client\s+import\s+[^\n]*\bOldClient\b/
/\bmylib\.client\.OldClient\b/

What a match proves: the project's code references your symbol through your import path at the commit GitHub indexed. What it looks like when it lies: the match sits in a vendored copy, a test fixture, a commented-out block, or a fork nobody maintains; or the project pins an old version and will never pick up your release. No public figure compares false positives from bare name matching against import-aware matching, so if this matters to you, run both on your own symbol and count the difference.

Test evidence from baseline-versus-candidate runs

A single test run against your candidate cannot tell your break apart from a project that was already broken. revdepcheck solves this by running R CMD check twice for each reverse dependency, once with the CRAN version and once with your local development version, and reporting the difference. Its flags:

The "still" flags are the pre-existing-failure rule in action: they failed before your change and they fail after it. The two-run design matters more as a library grows, because pre-existing failures and infrastructure noise grow with the number of revdeps. A single-run check at knitr's scale would send false alarms to innocent maintainers on every release.

For ecosystems outside R, I have not verified the equivalent procedures, such as Fedora's Packit downstream tests or Rust's crater. The design transfers regardless: pin each dependent, run its tests against your released version, run them again against your candidate, and record only rows that pass the first run and fail the second.

How do I rank dependents and find a reachable maintainer?

Rank by two things together: how strong your break evidence is and how much real usage the dependent carries. Then find a person, not an address, for every row you plan to message directly.

Usage signals come from step 6: the dependent's own download counts and the share of its users on affected versions. A dependent with heavy downloads that pins an old major version of your library is lower priority than its raw numbers suggest, because it will not pick up your release until it chooses to.

Finding a human behind each project

Commit metadata is the first place people look, and it fails often. GitHub gives accounts created after July 18, 2017 a noreply address in the form ID+USERNAME@users.noreply.github.com, and mail sent to a noreply address bounces. Username-based noreply addresses have a second trap: if the user changes their username, those commits stop being associated with their account. No published figure says what share of commit identities resolve only to noreply addresses, and no published comparison ranks issues, discussions, and maintainers or security files by response rate. In practice, use this order and record which route you used:

1. A maintainers file or security policy in the repository that names a real address.
2. A public profile for the top committer that lists a working route.
3. A clearly titled issue on the project, which is public, linkable, and never bounces.

The harder problem is that the people who own these projects rarely describe themselves as maintainers. In Refolk's index of professional profiles, I sampled 25 US profiles matching "open source maintainer": the most common titles were Software Engineer and Principal Engineer, and only one title named a maintainer role ("Lead Maintainer AboutCode and Scancode").

The pools also vary widely by place and language, which tells you where a title-only search will run dry fastest:

The US pool is 2.53 times Germany's, and the Python pool is 18.0 times Rust's. Read the Python number with care: it includes employees of a company called MaintainX, a keyword collision, and 9 of the top 10 titles in that set are founders or C-level. For Python dependents, your notice may land with a decision-maker rather than the engineer who wrote the call, so lead with the date and the cost of not migrating.

This is where matching code to people stops being a spreadsheet job. Refolk searches across the public GitHub graph, LinkedIn, and the open web from a plain-English request, so you can ask for the people behind the repositories that call your symbol instead of reverse-engineering identities from commit emails.

The notice itself

A notice works when the recipient can act on it without replying to ask what you mean. Name the symbol, show the call site you found, give the version and date, and link the migration path.

Subject: <your-package> <version> removes <fully.qualified.Symbol> on <removal date>

Hi <name or project team>,

I maintain <your-package>. Version <version>, planned for <removal date>, removes <fully.qualified.Symbol>.

Why I am writing to you: <project> uses it here: <call_site_url>
<If tested:> Your test suite passes against <current version> and fails against the release candidate: <log URL>

What to do:
- Replace <old usage> with <new usage>. Migration notes: <link>
- To test now, install the candidate with: <install command>

Timeline:
- Deprecated as of: <date>
- Removed in <version> on: <removal date>

If this date does not work for you, reply here before <date> and tell me what you need. If I have the wrong contact for <project>, I would be grateful for a pointer to the right one.

<your name>

How long should a deprecation period be?

A deprecation period should last until your measured share of users on the affected version has fallen, and never be shorter than published norms for a comparable surface. Pick a starting date from the norms, then gate the actual removal on data you archive from announcement day.

The evidence on upgrade behaviour says users move slowly. Kula et al. studied 4,659 Java projects and found that 81.5% of the studied systems still kept their outdated dependencies, with high library residue (x̄=85.7%, σ=22.2%). One version of log4j, 1.2.15, had a library residue of 98%.

I found no study that measures time to adopt a new major version in days, so there is no evidence-based universal number. The closest thing to a public benchmark is a large project's published policy:

Map your change to the row it most resembles. A stable, documented API you told people to depend on is the GA row; an experimental surface is closer to beta.

Measuring lag instead of guessing it

• npm: per-version counts exist only for the previous 7 days, via GET https://api.npmjs.org/versions/{package}/last-week. There is no history, so the lag curve exists only if you save a snapshot every week starting at announcement. The npm range endpoints are no substitute for long histories either: they are silently capped at roughly 18 months, returning exactly 549 days of data, and they are not per version.
• PyPI: each download writes a row to the bigquery-public-data.pypi.file_downloads table, including a file.version field, so history is queryable after the fact. Google gives 1 TiB of free BigQuery quota per month. pypinfo supports PEP 440 version matching, which lets you query stats for a whole major version at once.

Put both conditions in your notice: a date no earlier than your chosen floor, and a statement that you will check the affected-version share before removing. That gives downstream maintainers a firm planning date and gives you an honest reason to extend if the numbers have not moved.

How this goes wrong: failure modes and false positives

Most blast-radius mistakes are either undercounts that make a release look safe or false positives that send alarming notices to people who were never affected. Both cost trust, and the second costs it with exactly the maintainers you will need next time.

Two more belong on the list:

• Notices sent to noreply addresses. They bounce, and username-based ones also detach from the account after a username change. If the domain is users.noreply.github.com, open an issue instead.
• Keyword contamination in index counts. The Python "maintainer" pool of 557 in Refolk's index includes MaintainX employees. Inspect titles before quoting any people count, including your own.

Why these happen

The search undercount is structural, not a bug you can configure away. The API ignores regex and stops at 1,000 results, so any pipeline built on it returns a confident, complete-looking list that is missing rows. The only defence is comparing the API's total against the same search in the web UI, and treating a gap as a reason to confirm by hand.

Infrastructure noise grows with scale. Yihui Xie notes that CRAN's reverse dependency check will almost surely find false positive problems, for example from connectivity issues, and at around 10,000 revdeps he gets failure reports on every submission. A new failure in a log full of download errors is a network problem until proven otherwise.

The CI misread is the most avoidable. One project's gate treated revdepcheck's "no problems" message as a failure and opened a false-alarm issue. If you automate this step, key the gate on the flags in the table above, never on whether an output file has content.

A false alarm to an innocent maintainer costs more trust than a missed notice costs time.

Before you cut the release: re-run, verify, and keep it current

The last job is a second pass: re-run the checks close to release, confirm every notice went somewhere real, and leave behind a sheet the next breaking change can start from. The r-pkgs guide's advice is direct: when your release date rolls around, re-run your checks to see how many problems have been resolved.

Sources differ slightly on order. The r-pkgs workflow runs revdeps before submission and leans on CRAN's own reverse-dependency re-check afterwards, so in R the final re-run can effectively happen after notices go out. That is fine, as long as someone reads the post-submission result and follows up on rows still failing.

For rows still breaking at re-run time, escalate in matrix order: high-usage confirmed breaks get a second, personal message and, where you can, a migration pull request. If a large share of high-usage rows have not moved, that is your evidence for extending the window, and you can say so publicly with numbers.

Keeping the work current

Do not throw the sheet away after release. The contact routes, the noreply flags, and the list of which sources found which projects are the expensive parts, and they decay slowly. Next time you plan a breaking change, start from last release's sheet, re-pull dependents to catch new rows, and re-verify contacts on the high-usage quadrant first. If maintainers have moved on since the last notice, Refolk can find where the people behind a project work now, so a stale contact becomes a reachable one before you send anything.