The Off-List Reference Playbook: Finalist History to Cleared, Booked Calls

This playbook is for the in-house recruiter, sourcer, or founder who has a finalist close to offer and wants to hear from people who actually worked with them, not only the three names the candidate supplied. It takes one finalist's work history and turns it, within two working days, into a ranked map of 5 to 8 off-list referees, each with a dated overlap window, relationship type, evidence grade, and clearance status, with the calls booked before the offer goes out. You also get a coverage table you can attach to the offer approval, where any career period with no reachable referee is marked uncovered instead of being held against the candidate.

What an off-list reference map contains

An off-list reference map is a ranked list of 5 to 8 people who worked closely with the finalist but were not named by them. Each one is tied to a specific career period and has been cleared to call. An off-list reference (also called a backchannel or backdoor reference) is a conversation with someone who worked with the candidate but was not put forward by them.

Every referee on the map carries five fields:

• Overlap window: the start and end month when both people were at the same place, taken from dated public records.
• Relationship type: manager, peer, report, or cross-functional (someone in another function who depended on the finalist's work).
• Evidence grade: how strongly the public record shows the two actually worked together, graded A to D (defined below).
• Period and competency: which part of the career they can speak to, and which competency for the role they saw first-hand.
• Clearance status: candidate notice logged, current-employer exclusion passed, and jurisdiction recorded.

How many referees is enough

Five to eight. The published guidance spans that range:

• Gem's founder hiring guide recommends 2 to 3 formal references, "a mix of managers and peers who worked closely with the candidate", plus "at least 2-3 back-channel references".
• AVRA Talent targets "7 calls depending on level: 3 past bosses, 2 peers or customers, 2 subordinates".
• Bain Capital Ventures' field guide says to "aim for fewer but more relevant references".

I use 5 to 8 because it is enough to cover every material career period with someone who sat close to the work, and small enough that one hiring manager can run every call before the offer date.

Why the references the candidate gave you are not enough

Formal reference channels are often empty by policy. SHRM quotes a Jackson Lewis attorney: companies often send reference requests to HR, which confirms only dates of employment and positions held under neutral reference policies, and "even supervisors are often trained to send reference inquiries to HR". My inference from this, which is not a sourced finding, is that people who have since left the employer where they overlapped with the candidate are usually the only route to substance. The whole procedure below is built on that.

Is it lawful to call references the candidate did not list?

Usually yes, if your own team makes the calls and the candidate was told up front that vetting would happen. Which obligations apply depends on who compiles the information and which jurisdiction's rules cover the candidate. This section is a working summary for practitioners, not legal advice. Where the sources conflict, I say so, and those points are for counsel.

United States: the FCRA follows whoever assembles the report

If an employer calls a reference itself, searches public court records, or does its own research without a third-party consumer reporting agency, the FCRA does not apply to that activity. If any third party (a vendor, a private investigator, or a background check platform) helps assemble the information, the FCRA likely applies. The most common investigative consumer report is a reference check, so outsourcing reference calls brings in the investigative consumer report rules:

• A standalone disclosure and written authorisation.
• The investigative consumer report notice, made no later than three days after the employer requests the report.
• Before any adverse action, a copy of the report and a Summary of Rights under the FCRA.

One law firm goes further and argues that any third-party tool that scores or filters applicants may qualify as a consumer report. That view is not settled law. If your process puts software between the referee and the decision, ask counsel where it sits.

California: a conflict to flag

Under California's ICRAA, if the employer runs the check itself, the candidate's written permission is not required, and the employer does not have to hand over copies of non-public-record information such as reference checks. Public records do have to go to the candidate within seven days under Cal. Civ. Code 1786.53, unless the candidate waived that by ticking a box. "Public records" here means records of an arrest, indictment, conviction, civil judicial action, tax lien, or outstanding judgment, so a referee call does not normally produce one.

The sources disagree, though. One reads ICRAA as covering information gathered "through any means", which could include personal interviews and reference checks. If you hire in California, put that question to counsel before you rely on the in-house exemption.

United Kingdom and EU: notice, not consent

The ICO describes pre-employment vetting as employers making their own enquiries about a candidate from third parties, and says "you must tell candidates if you intend to use it". Its guidance is to "make it clear early in the process that vetting will take place and how". On lawful basis, the ICO's draft employment-records guidance says to avoid relying on consent under article 6. You should also make clear to workers and referees whether you treat references as confidential. That matters because confidential references are exempt from subject access.

For the EU, the Article 29 Working Party's Opinion 2/2017 says consent "is highly unlikely to be a legal basis for data processing at work" unless people can refuse without adverse consequence. The candidate must be told about the processing (for example in the job description) before they engage with the recruitment process. Neither regulator has published guidance on off-list referee calls specifically. That gap is real. Do not read more certainty into these rules than they contain.

Who to call for a backchannel reference

Call people who overlapped with the finalist at a past employer and have since left it. Give manager referees priority because they are the scarcest, and fill out coverage with peers, reports, and cross-functional partners. Each relationship type can speak to different things, and each fails in its own way.

Manager referees are the bottleneck

There are few managers relative to the people they manage. That means each career period may have only one or two possible manager referees, and the exclusion screen can remove those too. In Refolk's index of professional profiles, there are 43,215 people titled Engineering Manager in the United States and 10,955 in the United Kingdom, against 44,153 UK profiles matching Software Engineer.

Two practical points come out of these counts. They come from a 25-profile sample, so treat them as rough.

1. Title matches overcount. Only 15 of the 25 sampled UK "Software Engineer" profiles carried that exact title. The rest included CEOs and founders. Read every profile before you tag a relationship type, and do not trust the search match alone.
2. Big logos dominate. Meta was the most frequent employer in the UK engineering manager sample, at 3 of 25. At employers that size, "worked at the same company" tells you very little, which is the first failure mode below.

Recency is not settled. None of the investor or search-firm guidance sets a limit on how far back a referee can be useful. Set a house rule for each role family, write it on the coverage table, and apply it to every finalist the same way.

How to grade overlap evidence

Grade every name on what the public record proves about whether they worked with the finalist, not on whether they shared an employer. There is no published taxonomy of overlap evidence, and no measured rate of people who shared an employer and dates but never worked together. The four grades below are my working convention, not an industry standard.

What each grade needs before you accept it:

• A, joint work. Check that the interaction was two-way and repeated. For example, the referee reviewed the finalist's pull requests across several months, or the two co-presented as a pair rather than as part of a large panel. One commit to a repo with hundreds of contributors is grade D.
• B, team or reporting line. Both profiles name the same team, or one names the other as manager, and the overlap window is stated to the month. If either profile gives only years, record the window as approximate and do not round it up to B.
• C, adjacent function. Use C only to fill a cross-functional slot, and only if you can name the shared project or product surface. If you cannot, drop it to D.
• D, shared employer. Never call a D. Either find evidence that raises it or drop it.

The best referees combine two kinds of evidence. A grade B manager who also shows up as a reviewer on the finalist's grade A repository work is about as strong as public evidence gets. When one person has both, rank them above someone who has only one.

The procedure: from finalist history to booked calls

The method has eight steps and takes roughly 8 to 10 hours of desk work before the calls, spread over two working days. Do the research early and make the calls late. Gem says to "start back-channel references early", while the ICO principle is to vet "as late a stage as is practicable". Building the map early and calling once you are "relatively sure you're going to make a hire" (Gem's own test for formal references) meets both.

Day one: clearance, periods, long list

Step 1, clearance pre-check (30 to 60 minutes, recruiter, with legal if needed). Find the moment the candidate was told about vetting: the job description, an early stage email, or the application notice. Record the date and the wording. If you find nothing, send the notice now and wait before making any calls. Write down the jurisdiction and the name of the person who will compile the notes. If that would be anyone outside your organisation, stop and go back to the FCRA section.

Step 2, career periods (about 1 hour, sourcer). One period per role, not per employer. A promotion from senior engineer to engineering manager at the same company creates two periods with different likely referees. Mark gaps explicitly so nobody later reads an empty period as a missing referee.

Step 3, long list (2 to 4 hours, sourcer). AVRA notes that "you can find backchannel references through thirty minutes of internet research". That is true for one strong name. A list of 12 or more covering every period takes longer. For each period, search for former teammates, managers, and reports who have since left, plus anyone who shows up in the finalist's public work: repositories, talks, papers. Refolk lets you write that search the way you would say it, and it returns matching people from LinkedIn, GitHub, and the open web together, which saves you running three separate searches for each period.

A search tool finding names is a different thing from a third party compiling what referees say about the candidate. Keep the calls, the notes, and the scoring in-house.

Day two: grade, screen, rank, book

Step 4, grade the overlap (about 2 hours, sourcer). Apply the A to D grades. For technical finalists, the public GitHub graph is often the fastest route to grade A. In Refolk, ask for people who reviewed pull requests on the finalist's repositories during a given window. That surfaces collaborators who never appear on the finalist's LinkedIn profile.

Step 5, exclusion and conflict screen (about 1 hour, recruiter). Gem's guidance is to "be extremely careful about contacting colleagues at the current employer; consider prior employers instead", and if you must cover the current role, "reach out to someone familiar with their work who has already left". For each recent leaver, record their departure month and whether they still work with current staff. There is no documented procedure for second-order risk, such as a referee who shares investors or board members with the current employer. I check for it anyway and exclude when in doubt, because the cost of exposing a candidate's search is far higher than the cost of one fewer call.

Step 6, rank and build the coverage table (about 1 hour, recruiter). Rank on two axes: how strong the overlap evidence is, and how close the person sits to the current employer. Then fill slots by relationship type, starting with managers because they are scarce.

FINALIST: <name>    ROLE: <role>    JURISDICTION: <US state / UK / EU member state>
NOTICE: logged <date>, via <job description / stage email>, wording ref <link or ID>
COMPILED BY: <in-house name>    DECIDED BY: <different in-house name>
RECENCY RULE FOR THIS ROLE: <house rule>
CONFIDENTIALITY STATED TO REFEREES: <confidential / not confidential>

PERIOD 1: <employer> | <team> | <level> | <start month> to <end month>
  Referee | Relationship | Overlap window | Grade | Competency | Clearance | Call
  <name>  | manager      | <mm/yy> to <mm/yy> | B | <competency> | cleared | booked <date>
  <name>  | peer         | <mm/yy> to <mm/yy> | A | <competency> | cleared | booked <date>

PERIOD 2: <current employer> | <team> | <level> | <start month> to present
  STATUS: EXCLUDED - current employer. Not a negative.

PERIOD 3: <employer> | <team> | <level> | <start month> to <end month>
  STATUS: UNCOVERED - reason: <no reachable referee after exclusion>. Not a negative.

GAPS: <start month> to <end month> - career gap, no referee expected.

Step 7, outreach and calls (hiring manager). Gem puts a call at 15 to 30 minutes. The structured procedure in Taylor et al. used "a 10 - 15 minute telephone call with each referee". Short and structured beats long and wandering. Book every call before the offer date, and tell each referee up front whether their answers will be treated as confidential.

Subject: 15-minute reference call about <finalist first name>

Hi <name>,

I'm hiring a <role> at <company>, and <finalist first name> is a finalist. I understand you worked together on <team> at <employer> around <overlap window>. <Finalist first name> knows I speak to people beyond the references they provided.

Could you spare 15 minutes this week? I ask everyone the same short set of questions. Your answers will be treated as <confidential / not confidential>.

<slot 1> or <slot 2>?

Thanks,
<your name>

CALL SCRIPT (same order, every call)
0. How did you work together, between which months, and how often?
1. What was <finalist> responsible for when you overlapped?
2. <Competency 1>: tell me about a specific time you saw this. Rate 1 to 5 against others at the same level.
3. <Competency 2>: tell me about a specific time you saw this. Rate 1 to 5 against others at the same level.
4. What would <finalist> need to do differently to succeed as a <role>?
5. Would you work with them again, and in what capacity?

RECORD: rating per competency | one verbatim quote per competency | direct observation: yes / partial / no

Step 8, score and let the candidate explain. Score every call against the same rubric. The ICO says "the candidate should have the chance to explain, and the employer should consider the explanation". Do that before any adverse finding changes the decision.

How off-list reference checks go wrong

Most failures fall into seven patterns. Each one turns a fair, defensible check into a false signal or a legal exposure. Read this section before you start, then come back to it before you score.

1. A shared employer is read as real overlap. Two people at a company of more than 1,000 have overlapping dates but sit in different orgs. The referee answers anyway, usually vaguely and positively, and the vague answer gets counted as evidence. Check: every called referee is grade A or B, or grade C with a named shared project. Ask question 0 on every call and drop the call if the referee cannot describe direct contact.
2. A tool assembles the referee report. A vendor or platform runs the calls or compiles the findings. That turns a lawful direct call into an FCRA investigative consumer report made without the required disclosure. Check: the coverage table names the compiler, and it is an in-house person.
3. A recent leaver passes the exclusion screen. Gem's "already left" workaround fails if that person still talks to their old team every day. Your call becomes news inside the current employer by the afternoon. Check: record the departure month and the current working relationship with former colleagues. If either is unclear, exclude.
4. A missing referee is read as a negative. A period with no reachable referee gets quietly marked down, often because the current employer had to be excluded. That punishes the candidate for your caution. Check: the coverage table labels the period uncovered with a reason, and the scoring sheet has no field for it.
5. A neutral HR answer is treated as a signal. A former employer's HR team confirms dates and title only. That reflects its neutral reference policy, not the candidate's performance. Check: any HR-only response is logged as "policy response" and carries no score.
6. Unstructured impressions outweigh structured scores. A memorable anecdote from one call overrides consistent ratings from five others. No published validity evidence supports giving unstructured impressions that weight. Check: every call used the same competency questions in the same order, and the decision memo cites the scores.
7. Researcher bias. The person who dug up the referees also decides the hire. The ICO warns against exactly this pattern for social media research: staff who research candidates "should not also make the recruitment decisions". Check: the coverage table names a different person under compiled-by and decided-by.

Failure modes 2 and 7 are the easiest to miss, because the process can look fine from the outside. Both are about who touched the information, not what it said. That is why the coverage table records names, not just outcomes.

How to score the calls and keep the step defensible

Score every call against the same competency questions, read mixed answers as a real signal, and let the candidate respond to anything adverse before it changes the decision. Structure is the part of reference checking with evidence behind it. Who you happen to reach is not.

Validity (r) is the correlation between a selection method's result and later job performance. Higher means the method predicts better.

What this does and does not tell you:

• Structure is the evidenced lever. A short, structured call matched or beat the meta-analytic reference figure. Put your effort into consistent questions and rating scales, not into finding a ninth referee.
• References are a supporting method, not a lead one. Structured interviews and work samples sit at roughly double the validity. Use the reference map to test what the interviews found, not to replace them.
• The evidence base is thin. A later reanalysis by Sackett and colleagues left reference checks out for lack of sufficient research. Nobody has published how much unstructured backchannel impressions add. Do not claim more precision than this.

Reading the answers

Gem's observation is that "references come back positive by default, so a mixed reference is usually a bad signal". Treat hedged praise, long pauses before a rating, or "they were fine" from a grade A or B referee as worth a follow-up question. Do not treat the same answer from a grade C referee the same way, because they may simply not know.

When something adverse comes up, put it to the candidate in neutral terms and record their explanation next to the finding. If a third party compiled any part of the report in the US, the FCRA pre-adverse-action steps apply: a copy of the report and the Summary of Rights, before you act. For UK candidates, the referee was told whether their answers are confidential, so honour that when you frame the question.

Before the offer goes out, and keeping the map current

Run the checklist below before the coverage table goes into the offer approval. Every item can be checked against the table itself. If an item fails, fix it or hold the offer. Do not attach the table with a known gap in clearance.

To keep the method current across hires:

• Re-check the law by mechanism, not by memory. The UK position rests on ICO employment guidance, which the ICO revises and finalises over time, and the EU position rests on WP29 Opinion 2/2017 as applied under GDPR. Before each new jurisdiction, read the current ICO pre-employment vetting page and check with counsel whether any regulator has since issued guidance on off-list referee calls.
• Hold the recency rule steady. Whatever recency limit you set for a role family, apply it to every finalist for that role, and change it only between searches, not during one.
• Reuse the method, not the notes. Referee notes were collected for one hiring decision under one notice. Do not carry them into another candidate's file or another role without fresh notice. Keep them under your normal recruitment retention policy.
• Keep the long-list queries. Saving the period-by-period searches from step 3 cuts the next finalist's sourcing time, especially at employers you hire from often. That is where repeated plain-English searches in Refolk pay off: the query for "former managers at that employer who have since left" is the same each time, and only the window changes.