# From Raw Commit Log to a Defensible Count of Distinct Humans

*You can turn a repo's raw commit data into a deduplicated roster of distinct humans that a second analyst reruns to the same count.*

- Canonical URL: https://www.refolk.ai/guides/commit-log-to-distinct-humans
- Pillar: Process, data, and compliance
- Format: Playbook
- Published: 2026-08-08
- Last reviewed: 2026-08-08
- Reading time: 15 min
- Keywords: merge git contributor aliases, deduplicate github contributors, filter bots from contributor list, count real contributors repository, same person multiple github accounts

## Key takeaways

- A defensible contributor count sits below the raw git shortlog total, which overcounts humans via aliases, and above GitHub's graph, which caps at the top 100 and only strips [bot] Apps.
- The numeric prefix of a users.noreply.github.com address is the only free, reliable cross-account key: it survives username renames while the username prefix does not.
- Real repositories run about 4% bot by headcount. A NumFOCUS census of 853 contributors found 34 bots, 13 GitHub Apps, and 4 automation services against 802 humans.
- RABBIT reaches AUC 0.97 and weighted F1 0.94 on a 644-bot / 691-human dataset while running over 50x faster and needing 3.5x less data than prior tools, which keeps it under GitHub's 5,000-query hourly API limit.
- Raw Git treats 'John' and 'john' as two authors, but mailmap matches case-insensitively, so a single mailmap pass collapses duplicate rows no manual eyeball would catch.
- The audit artifact that makes the count reproducible is the committed .mailmap plus a recorded bot-exclusion ledger and command log, not the final number itself.

You have a repository's raw contributor data and someone has asked how many people actually built it. This guide is for the recruiting-ops, revenue-ops, or diligence owner who has to answer that question and defend the number. It delivers one auditable pipeline that collapses duplicate identities, strips bots, and links one person's multiple accounts, so two analysts running the same files reach the same count.

The problem is that the two easy numbers are both wrong, in opposite directions. GitHub's contributors graph flatters: it caps at the top 100, counts only default-branch commits, and drops merge and zero-change commits. A raw `git shortlog` fragments: the same person shows up three times because they committed from two laptops and once with a capitalised name. Your defensible count sits below the raw log total and above the graph total, and getting there is a procedure, not a guess.

## What a defensible contributor count actually is

A defensible count is the number of distinct human contributors that survives a documented alias-merge, a documented bot-exclusion, and a documented cross-account link, with every decision recorded in files a second analyst can rerun. It is not the number the graph shows and not the number `git shortlog` shows.

The reason this needs a standard at all is that no single authority publishes the ordering as one pipeline. The primary sources support each stage on its own: the `.mailmap` spec covers aliasing, tools like BoDeGiC and RABBIT cover bots, and GitHub's numeric noreply IDs cover cross-account identity. Nobody stitches them together and states what "reproducible" means. That gap is what this document fills.

#### Where a real count sits between the two easy numbers

| Stage | Figure | Note |
| --- | --- | --- |
| Raw git shortlog rows | widest | aliases split one human into several |
| After alias merge | narrower | one row per human identity |
| After bot exclusion | narrower still | user-account and App bots removed |
| After cross-account link | narrowest | distinct humans |

*The raw log overcounts humans through aliases while GitHub's graph undercounts them, so the defensible figure lives between them.*

The three forces that move the number are worth naming before you touch a command, because each has a signal you can check and a way it lies.

- **Alias fragmentation** inflates the count. Its signal is two rows with the same person's name but different emails, or the same email under two spellings. It lies when two genuinely different people share a generic name or a shared machine email.
- **Bots** inflate the count. Their signal is a `[bot]` suffix, repetitive commit messages, or a name like `release-bot`. They lie when a bot has a human-looking name and a regular user account, because GitHub will not strip it for you.
- **Cross-account identity** inflates the count. Its signal is two accounts sharing one numeric noreply ID. It lies when you match on the username prefix instead, which changes on rename.

## Why raw Git and GitHub disagree with each other

Raw Git and GitHub's graph disagree because they apply different rules to the same commits, and neither rule produces a clean human count. Understanding the exact mismatch tells you which number to trust for which purpose.

Raw Git treats author name and email as case-sensitive strings. A `user.name` of `John` and one of `john` are two different authors, and Git will show them as two rows. It does no identity resolution at all: every distinct name/email pair is a distinct author. That is why the raw `git shortlog` total is an upper bound on humans, not a headcount.

GitHub's graph goes the other way. It shows the top 100 contributors, excludes merge commits and zero-change commits, counts only commits on the default branch, and auto-excludes GitHub App identities whose usernames end in `[bot]`, such as `dependabot[bot]`. Three of those rules undercount humans, and the fourth removes only one class of bot. A release bot running as a regular user account keeps every one of its commits, and there is no built-in way to exclude a specific user from that view.

**4% - Share of a real repository's contributors that are bots or automation**

A NumFOCUS census of 853 contributors across 59 orgs found 34 bots, 13 GitHub Apps, and 4 automation services against 802 humans.

That roughly 4% is small, but it is exactly the kind of margin that fails an audit when someone recomputes your number and finds a `[bot]` in your headcount. The census also shows the shape of the problem: most bot volume is Apps and automations you can match by pattern, with a smaller tail of user-account bots that need a classifier or a hand check.

## The mailmap, and what it does and does not do

A `.mailmap` file at the repository top level maps author and committer names and emails to canonical real names and addresses. It is the standard, non-destructive way to declare that several identities are one committer, and it is the core audit artifact of this whole procedure.

The critical property is what it does not touch. Mailmap only modifies the visual output of commands like `git shortlog` or `git log --use-mailmap`. It does not rewrite commit history and does not prevent future commits with varying names. That is a feature, not a limitation: you get a committed, reviewable record of identity decisions without a history rewrite that would break every existing hash and clone.

There is one asymmetry that quietly does a lot of the deduplication work for you. Raw Git is case-sensitive, so `John` and `john` are two authors. But mailmap matching is case-insensitive for both names and emails. A single mailmap pass therefore collapses case-only duplicates that no manual eyeball would reliably catch across a long list.

> **Watch out:** Mailmap does not apply everywhere
>
> While git shortlog uses the mailmap file automatically, other subcommands do not. A clean git shortlog can coexist with a git log or git blame that still shows every alias. Always verify with git log --use-mailmap, or you will believe a merge happened that has not.

The `.mailmap` syntax maps aliases to a canonical identity, one per line. Keep it in the repo root so it is picked up automatically and versioned alongside the code it describes.

**.mailmap skeleton**

```
# Canonical Name <canonical@email> Alias Name <alias@email>
Jane Doe <jane@company.com> Jane Doe <jane@laptop.local>
Jane Doe <jane@company.com> jane doe <jane.doe@gmail.com>
Jane Doe <jane@company.com> <12345+janedoe@users.noreply.github.com>
Sam Okoro <sam@company.com> Samuel Okoro <sam.okoro@oldjob.com>
```

*Left of the alias in <> is the canonical identity. Right is an alias to fold into it. Replace names and emails with the clusters you flagged.*

## Choosing a bot detector

Choose a bot detector by matching its data appetite to your roster size and your rate-limit budget, not by chasing the top F1 score. All the credible tools are accurate enough that the deciding factor is how much data each one pulls per account.

The tools split by primary signal. Commit-message tools like BoDeGiC read only what is already in your clone, so they cost no API calls but see less. Account-activity tools like RABBIT query the GitHub Events API, so they see richer behaviour but spend against your 5,000-query hourly limit. For a large roster, that budget is the real constraint, which is why RABBIT's efficiency matters as much as its accuracy: it runs over 50x faster and needs 3.5x less data than the prior state of the art.

| Tool | Primary signal | Best reported P / R / F1 | Data note |
|---|---|---|---|
| BoDeGiC | Commit-message patterns | precision 0.80 | commit-message only, no API |
| RABBIT (BIMBAS) | 6 account-activity features | ~0.919 / 0.919 / 0.919 | >50x faster, 3.5x less data |
| BotHunter | 19 profile+activity features | F1 0.924 | heavier data pull |
| BIMAN | name suffix + repetition + files | 0.667 / 0.866 / 0.754 | commit-data only |

RABBIT's headline validation is AUC 0.97 and weighted F1 0.94 on a ground-truth set of 644 bots and 691 humans, with the model trained on 60% of the data. Those are strong numbers, but read the recall column too. BIMAN's commit-only recall of 0.866 against precision 0.667 means it flags many humans as bots; that trade is fine as a first pass you then hand-check, and wrong as an unreviewed final filter.

> **Rule:** No classifier gets the last mile
>
> No bot tool here claims 100% precision, and the authors say so. Every exclusion the classifier makes must be checked against the account's raw commits before it leaves your roster. The tool proposes; you dispose, and you log why.

## Linking one person across multiple accounts

To link one person's multiple accounts, match on the numeric prefix of their `users.noreply.github.com` email. That numeric ID is the only free, reliable cross-account join key, because it is stable where names and usernames are not.

GitHub issues two noreply formats, and the difference decides whether you can trust the address. This is the single most common place people mislink, so treat the format as a hard gate.

| Account created | Noreply format | Stable across rename? |
|---|---|---|
| After 2017-07-18 | ID+USERNAME@users.noreply.github.com | Yes, via the numeric ID |
| Before 2017-07-18 (private set early) | USERNAME@users.noreply.github.com | No |

Commits made with the private noreply email are linked directly to the GitHub account, and for the newer format the numeric prefix survives every username change. Match on that number. Do not match on the username portion: it changes on rename, so username matching mislinks one account to another after either party renames. Name-based cross-account linking beyond the numeric ID is not publicly established as a deterministic method, so where you have no numeric ID, treat two accounts as separate unless you have external evidence you can cite.

I ran this search: `Find people whose GitHub commits use a users.noreply.github.com email and who now work at Vercel` - [see the full result list](https://www.refolk.ai/s/qh0h3m0nk3).

*Returns engineers whose commit identity ties back to a stable noreply address and whose current employer is confirmed, so you can resolve who a fragmented identity is today.*

Once your roster is deduplicated to distinct humans, the next question is usually who those people are and where they work now. That resolution across the open web is what [Refolk](/) is built for: you ask in plain English and get the right people across the public GitHub graph, LinkedIn, and the open web, which is the step that turns a clean count into a list you can actually reach.

## The procedure, start to finish

Run these eight stages in order and record each one. The whole pass takes roughly two to four hours for a mid-size repository, most of it in the manual duplicate review and the bot run.

#### Raw commit log to distinct humans

1. **Extract the raw identity list** - Run git shortlog -sne --all --no-merges to summarise every author by name and email with a commit count across all branches, excluding merge commits. Done when you have a flat list of every name/email pair and its count. About 5 minutes.
2. **Spot duplicate identities** - Sort the list so near-identical names and emails sit adjacent, piping through awk and sort to cluster them for review. Done when every candidate alias cluster is flagged. About 15 to 30 minutes.
3. **Write the .mailmap** - Map each alias to one canonical name and email in a .mailmap file at the repo top level. Done when re-running git shortlog -sne collapses each cluster to one row per human. 30 to 60 minutes for a mid-size repo.
4. **Re-run and verify the merge** - Re-run git shortlog -sne to confirm the mailmap took effect, then check git log --use-mailmap agrees, since non-shortlog subcommands do not apply mailmap automatically. Done when you have a deduplicated author roster. About 5 minutes.
5. **Detect bots** - Run BoDeGiC over commit messages or RABBIT over account activity from the Events API across the deduplicated roster. Done when each identity carries a human/bot label and a score. 15 minutes to hours depending on account count.
6. **Exclude bots and record why** - Remove [bot] GitHub App identities and any flagged user-account bots, logging the reason for each exclusion. Done when you have a human-only roster plus an exclusion ledger. About 15 minutes.
7. **Link accounts across identities** - Resolve numeric users.noreply.github.com IDs to collapse one person's multiple accounts, matching on the numeric prefix not the username. Done when you have a distinct-human count. About 20 minutes.
8. **Freeze the audit trail** - Commit the .mailmap, the bot-exclusion ledger, and the command log to the repository. Done when a second analyst rerunning the same files reaches the same number.

On the order of stages 3 and 5, the sources disagree, and the right choice depends on your repo. Bot-first-then-dedupe is defensible when App bots dominate, because you strip the obvious `[bot]` suffixes cheaply first. Dedupe-first is cleaner when the bots are user accounts, because collapsing aliases reduces the surface a bot classifier has to judge. Pick one, and write down which, so the count is reproducible.

> The count is not the deliverable. The mailmap, the ledger, and the log are the deliverable; the count just falls out of them.

## How this goes wrong

Most bad contributor counts come from a handful of repeatable failures, each with a false positive that looks like success. This section is the one to keep open while you work, because every item here has burned someone who trusted a clean-looking output.

- **The mailmap looks applied but is not.** You see a clean `git shortlog` and declare victory while `git log` and `git blame` still show the aliases, because those subcommands do not auto-apply mailmap. Check with `git log --use-mailmap` before you trust any merge.
- **You over-merge distinct humans.** Two different people sharing a generic name or a shared machine email get collapsed into one row. Before merging any cluster, confirm the numeric noreply IDs match; if they differ, keep the people separate.
- **A user-account bot survives.** A release bot with a human-looking name passes every automated filter because GitHub only strips `[bot]` App identities, not regular users. Hand-check the top committers for accounts that commit like machines.
- **The classifier misfires on low activity.** BoDeGiC and RABBIT mislabel sparse accounts, and the tools warn they cannot reach 100% precision. Check every exclusion against the account's raw commits, and be most suspicious of accounts with very few commits.
- **Username-prefix matching breaks on rename.** Matching on the username portion of a noreply address mislinks accounts after either side renames. Only the numeric ID is stable, so join on the number.
- **You count the wrong branch.** Using GitHub's graph or a default-branch-only query undercounts, because only default-branch commits appear there. Check against `git shortlog --all`.
- **Merge and empty commits inflate the total.** Zero-change and merge commits skew the raw numbers. Always pass `--no-merges` when you extract the list.

#### When a candidate merge is safe

Horizontal axis runs from Numeric noreply IDs differ to Numeric noreply IDs match. Vertical axis runs from Names differ to Names match.

| Quadrant | What it means |
| --- | --- |
| Different name, different ID | Keep separate; almost certainly two people |
| Different name, same ID | Merge; one person who renamed or changed email |
| Same name, different ID | Do not merge on name alone; likely two people sharing a common name |
| Same name, same ID | Merge; the clean case, a single person with aliases |

*Merge two identities only when both the name and the numeric noreply ID agree; anything else needs a check first.*

## Sizing the market your count implies

A clean contributor roster is often the input to a bigger question: how large is the talent pool behind this kind of work, and where is it. Your deduplicated headcount for one repo is a data point; the market it sits in is what a hiring or diligence decision actually turns on.

Refolk's index gives a concrete sense of scale for open-source engineers by market. Among profiles tagged "Software Engineer" with an open-source-development skill, the index holds 79 in the United States against 24 in Germany.

| Market | Matching profiles | At Google | Google share (derived) |
|---|---|---|---|
| United States | 79 | 11 | 13.9% |
| Germany | 24 | 4 | 16.7% |

The US-to-Germany ratio is about 3.3x, and a single large employer already accounts for roughly one in seven of these profiles in each market. The lesson for your count is proportion, not absolute size: once you know a repo has, say, 40 distinct human contributors, the market data tells you whether that is a large slice of a small pool or a rounding error in a large one. That framing is what turns a defensible number into a decision.

## Keeping the count current and reproducible

A contributor count decays the moment you compute it, because new commits, new aliases, and new bots keep arriving. Keep it current by re-running the same frozen files on a schedule rather than starting over, and by treating the audit trail as the thing you maintain.

Before you call any run done, verify the artifacts, not just the number.

#### Before you publish the count

- [ ] git shortlog -sne --all --no-merges was the extraction command, so all branches are covered and merge commits are excluded.
- [ ] The .mailmap is committed at the repo top level and git log --use-mailmap confirms the aliases actually collapsed.
- [ ] Every merged cluster was checked so that matching numeric noreply IDs, not just matching names, justified the merge.
- [ ] All [bot] App identities are removed and the top committers were hand-checked for user-account bots.
- [ ] Every classifier exclusion is logged with a reason and checked against the account's raw commits.
- [ ] Cross-account links used the numeric noreply prefix, never the username prefix.
- [ ] The .mailmap, the bot-exclusion ledger, and the command log are committed so a second analyst reruns them to the same number.

When you re-run, the `.mailmap` and bot ledger carry forward, so a refresh only has to review new identities that appeared since the last pass. That is the payoff of the audit trail: the second run is a fraction of the first. And when someone challenges your number against the raw graph, you do not defend the number. You hand them the three files and let them rerun it, which is the only defence that holds.

## Frequently asked questions

### Why does my contributor count differ from GitHub's graph?

GitHub's contributors graph shows only the top 100 contributors, counts only default-branch commits, drops merge and zero-change commits, and auto-excludes only [bot] GitHub Apps. So it undercounts humans in two ways while leaving user-account bots in. A raw git shortlog --all overcounts in the other direction through unmerged aliases. Your defensible count sits between the two and requires the alias-merge and bot-exclusion pipeline to produce.

### Does a .mailmap file change my commit history?

No. A .mailmap maps author and committer names and emails to canonical identities, but it only modifies the visual output of commands like git shortlog and git log --use-mailmap. It does not rewrite history or prevent future commits with varying names. That is exactly why it is the right audit artifact: it records that different names are the same person without a destructive rewrite.

### How do I tell if two GitHub accounts belong to the same person?

Match on the numeric prefix of a users.noreply.github.com address. Accounts created after 18 July 2017 get an ID+USERNAME@users.noreply.github.com address whose numeric ID survives username renames. That numeric ID is the reliable join key. The username prefix is not, because it changes on rename. Name-based cross-account linking beyond the numeric ID is not established as a deterministic method.

### Which bots does GitHub already remove for me?

Only GitHub Apps whose usernames end in [bot], such as dependabot[bot]. A release bot or CI account that commits as a regular user has all its commits counted in the contributors view, and there is no built-in way to exclude a specific user from that view. You must detect and remove user-account bots yourself, which is why hand-checking the top committers is part of the procedure.

### Should I remove bots before or after merging aliases?

It depends on which kind of bot dominates. Bot-first is defensible when GitHub App [bot] identities dominate, since they are trivially matched by suffix. Dedupe-first is cleaner when the bots are regular user accounts, because collapsing aliases first reduces the surface a bot classifier has to judge. Whichever order you pick, record it in the command log so the count is reproducible.

### How accurate are automated bot detectors?

Good ones are strong but not perfect. RABBIT reports AUC 0.97 and weighted F1 0.94; a RABBIT/BIMBAS configuration reports precision, recall, and F1 near 0.919 each. BoDeGiC on commit messages reaches precision 0.80 after retraining. None claims 100% precision, so every exclusion should be checked against the raw commits, especially for sparse accounts where classifiers misfire.

---

*From the Refolk guide library. I revise these guides rather than replacing them, so the current version is always at https://www.refolk.ai/guides/commit-log-to-distinct-humans*