Estimating Talent-Pool Representation From Public Data

A strategy lead asks how diverse the external market is for a role, and you have hours to turn a pile of public profiles into a number someone will act on. This guide is for talent-intelligence analysts and market-sizing operators who need an aggregate representation estimate that survives scrutiny: with subgroup-specific error bounds, an honest non-classification rate, and a verdict on whether a gap is a narrow market or a broken sourcing channel. It never attaches an inferred demographic label to a single person, because that is both the legal line and the only way the number stays defensible.

Most DEI content measures the applicant pool and treats name and photo inference as a solved black box. The academic literature that documents the error rates offers no operating procedure. This document turns those documented error rates into a method with confidence bounds and a market-versus-sourcing verdict, and it tells you exactly when your number is trustworthy, which subgroups it undercounts, and what you are forbidden from doing with it.

What this estimate is, and what it can never be

This is an aggregate estimate of a talent pool's demographic composition, expressed as subgroup shares with confidence bounds, produced from public profiles without labeling any individual. It is not, and cannot be, a demographic tag on a person.

The distinction is the whole method. Name-based and geography-based inference is wrong often enough that no single prediction is safe to act on, but the errors partly average out across a large pool, so an aggregate share with an honest error bound is defensible. That is why the compliant design and the accurate design are the same design: GDPR discourages per-person labels, and inference error is only tolerable in aggregate.

You are estimating three things at most: the gender split, the race and ethnicity split where lawful, and the share you could not classify at all. You are explicitly not estimating non-binary representation, because name-based tools cannot infer it and any figure you produce would be invented.

Why one accuracy number is a trap

A single overall accuracy figure hides the failures that matter, because inference errors concentrate in exactly the subgroups your estimate exists to count. Report accuracy per subgroup or your number is worse than no number.

The definitive evidence is Lockhart, King and Munsch (Nature Human Behaviour, 2023), which compared self-identified demographics against name-based imputations for 19,924 scholars across four gender tools and four race and ethnicity tools. An overall gender error near 4.6% sounds reassuring. It masks Chinese women being misgendered 43% of the time, and Black people from highly educated families having their race misclassified 80% of the time.

The paper's central argument is the one to internalize: if the 5% who are misclassified are disproportionately women of color or scholars from East Asia, the tool is systematically erasing the people most likely to be undercounted. Any study that uses name-based inference to measure gender gaps will undercount the very groups experiencing the largest gaps.

Race inference has a direction, not just a magnitude. Bayesian Improved Surname Geocoding (BISG), the reference method that combines surname and location, ran a 15.1% overall error rate on Florida voter data and underestimated Hispanic populations in 88% of Florida counties. It reads minorities in wealthy, educated areas as White. So your estimate biases toward "less diverse" precisely in high-status talent pools, which are exactly the pools an intelligence team is usually asked about.

The subgroup error rates you attach

These are the published rates you carry into your bounds. Attach the relevant row to each subgroup so a reviewer can see why one share has a wide interval and another does not.

BISG almost never predicts "Other" (a 99% false negative rate), while correctly identifying White individuals at a high rate (6.7% false negative in Florida). RAND reports aggregate C-statistics of 0.94 for Asian and Pacific Islander, 0.93 for Black, 0.94 for Hispanic, and 0.93 for White. Those C-statistics describe ranking ability in aggregate; they are not a license to label a person.

The non-classification rate usually kills precision first

The non-classification rate, not the misclassification rate, is what most often destroys your margin. Report it separately, per subgroup, before you touch a confidence interval.

Tools return three outcomes per name: right, wrong, or unknown. genderize.io produces the most non-classifications among common tools. In one large bibliometric study, 32% of authors came back gender-unknown. If you silently drop those, your denominator shrinks and minority shares inflate. If you keep them honestly, your usable sample collapses: a nominal pool of 1,000 with 32% unknown yields roughly 680 classified records, which widens your true margin well beyond a planned plus-or-minus 3%.

So the sequencing matters. Retain unknowns through the dataset and exclude them only at the specific step that requires a binary classification. The PeerJ name-to-gender benchmark took the opposite route, using only the 5,779 names with a defined label and excluding unknowns from every calculation, which is fine for benchmarking a tool but dishonest for measuring a market, because it hides how many people the tool could not see.

Sizing the pool for the bound you promised

The minimum sample is a function of the margin you want and the subgroup share you expect, and it counts classified records, not raw profiles. Use the standard proportion formula and size up so the smallest subgroup you must report still clears its threshold.

The formula is n = p(1-p)(z/E)^2. At 95% confidence with the conservative p=0.5 and a plus-or-minus 5% margin, n = 0.25 x (1.96/0.05)^2 = 385. When the subgroup share is known to be smaller, n falls; when you tighten the margin, it rises fast.

The last two rows carry the lesson. A 10% subgroup at plus-or-minus 5% needs only about 139 classified records, but tightening to plus-or-minus 3% at p=0.5 demands 1,068. This is why a "narrow market" verdict on a rare subgroup is so often just an underpowered sample. And because these are classified counts, a 32% NA rate means you need a raw pool roughly half again larger to hit them.

A narrow-market verdict on a rare subgroup is usually not a market. It is an underpowered sample.

Where pool size actually comes from

Before you can size a subgroup, you need to know how many people exist for the role and place at all, and that raw pool varies enormously by geography. Anchor your denominator in a real count, not a guess.

In Refolk's index of professional profiles, the same job title returns wildly different pools across countries. The United States returns 349,839 profiles for "Software Engineer"; Germany returns 22,422 for the identical title. That is a 15.6x difference in raw supply for one role.

The operational point: a subgroup that is comfortably estimable in the US pool may be statistically invisible in the German one for the same role. If you owe a plus-or-minus 5% bound on a 10% subgroup, you need about 139 classified members of that subgroup, and in a 22,422-person pool with a real NA rate that can be tight. Know your raw denominator before you promise a margin.

Pulling the pool this way, in plain English, removes the first hour of friction: you get a scoped, de-duplicated count for the exact role and place instead of scraping and re-keying. Refolk gives you the denominator; the inference and bounds are still your job.

Running the estimate, start to finish

This is the full procedure, in order, with owners and rough durations. Each step has a clear "done" state so you know when to move on. Retain unclassified records throughout and exclude them only where a binary metric forces it.

Choosing the external anchor and rendering the verdict

The market-versus-sourcing verdict is decidable only against an occupation-specific, geography-specific anchor. Compare your estimated shares to the external labor market for the same occupation and place, then read the direction of the gap.

The U.S. benchmark is the Census EEO Tabulation, which for more than five decades has served as the primary external benchmark for comparing the race, ethnicity, and sex composition of an organization's workforce against the analogous external labor market within a specified geography and job category. The current release is the 2014-2018 EEO Tabulation, built on 5-year ACS data; the prior edition covered 488 detailed occupation categories. Degree-conferral pipelines are the analogous supply-side anchor where a role maps cleanly to a credential.

The verdict rule is simple once the anchor is right:

• If your pool's composition matches the external market cell, within your error bounds, the gap you were asked about is a genuine market constraint. The supply is thin; sourcing harder in the same channels will not fix it.
• If your pool is narrower than the external market cell, the gap is a sourcing-channel problem. The people exist; your channels are not reaching them.

The trap is the anchor. Benchmarking to national totals instead of the occupation-by-geography cell manufactures a false gap, because national demographics look nothing like the demographics of a specific occupation in a specific metro. Match the occupation code and the geography before you compare anything.

How this goes wrong

The failure modes below are where defensible estimates turn into liabilities. Each one has a false positive that looks clean and a specific check that catches it. Treat this as the most important section: a wrong representation number gets quoted in a board deck and drives real hiring decisions.

• Reporting one overall accuracy. The false positive reads "95% accurate, we're fine." The Nature 43% and 80% figures prove the overall number hides the failures. Check: recompute error per subgroup and refuse to publish a single headline accuracy.
• Dropping unclassified names silently. A shrunken denominator inflates minority shares and looks tidy. Check: report the NA rate separately; if it exceeds 10%, recompute effective sample size and widen the margin.
• Forcing a hard label on ambiguous names. You get a clean-looking distribution built on unisex and transliterated names, the documented weak point. Check: keep probability outputs and never coerce a most-likely label into a fact.
• Running BISG in affluent, educated areas. Minorities get misread as White, undercounting diversity exactly where you were asked to measure it. Check: flag high-income geographies and state the expected directional undercount.
• Imputing non-binary identity. Tools cannot do this, so any figure is fabricated. Check: state it as an unmeasurable blind spot, never a number.
• Benchmarking to the wrong anchor. National totals instead of the occupation-by-geography EEO cell produce a false gap. Check: match the occupation and geography before comparing.
• Persisting a person-level label. This is direct legal exposure under Article 9. Check: audit for stored per-record inferences and confirm only aggregates survive.
• Treating a small-pool CI as precise. A plus-or-minus 3% claim needs roughly 1,068 classified records at p=0.5. Check: if the classified pool is under that, widen the stated bound or decline the point estimate.

The pre-publish checklist and how to keep it current

Before you send the estimate, walk this list. It is the difference between a number a reviewer trusts and one a lawyer flags.

To keep this method current, re-check three moving parts rather than memorizing values. First, the external anchor: the EEO Tabulation is reissued on a multi-year cycle tied to ACS releases, so confirm you are on the latest edition and that its occupation categories still map to your role. Second, the published error rates: as inference tools change, the subgroup rates you attach should be refreshed from the current literature, and you should re-run a small internal validation where you can. Third, the legal position: special category rules vary by jurisdiction and evolve, so before extending the method to a new country, confirm the local basis, since gender is not special category data in France but race is protected across the EU, and other regimes such as Australia's Workplace Gender Equality Act impose their own collection duties.

The reusable artifact is the report skeleton itself. Keep it as a template so every estimate you ship has the same defensible shape.

Role and geography: <role> in <geography>
Raw pool N: <count>  |  Classified N: <count>  |  Non-classification rate: <pct>
Subgroup shares (aggregate only):
  <subgroup> : <share> +/- <margin>  (attached error rate: <rate>; bias direction: <note>)
  <subgroup> : <share> +/- <margin>  (attached error rate: <rate>; bias direction: <note>)
Blind spot: Non-binary identity is not estimable from names and is not reported.
External anchor: EEO Tabulation cell for <occupation> in <geography>: <anchor shares>
Verdict: <market constraint | sourcing-channel gap | no detectable gap at this sample size>
Legal basis: Article 6 <basis> + Article 9 <condition>; DPIA: <yes/no>
Retention: person-level inferences deleted on <date>; only this table retained.