# Vetting a Platform Vendor Before You Build a Core Feature on Its API

*You can vet a specific API vendor end to end from public evidence and reach a graded build, build-with-abstraction, or walk-away verdict.*

- Canonical URL: https://www.refolk.ai/guides/vetting-platform-vendor-core-feature
- Pillar: Engineering and open source
- Format: Playbook
- Published: 2026-09-09
- Last reviewed: 2026-09-09
- Reading time: 16 min

Building a core product feature on a third party's hosted API is a bet that the vendor will still honor that API years from now. This guide is for the technical founder or engineering lead who has to make that bet on a specific vendor and defend it. It gives you an ordered method to reconstruct the vendor's deprecation and versioning track record from public artifacts, weigh it against reliability and community signals, and end at one of three verdicts: build, build-with-abstraction, or walk away.

The real risk here is not a bad demo or a missing endpoint. It is a forced migration or a quiet sunset that arrives long after you have shipped. So this is not a feature comparison. It is a discipline audit of a company you cannot see inside, run entirely from what it has published.

## What "safe to build on" actually means

Safe to build on means the vendor has a demonstrated, public record of giving enough notice before breaking changes, a reliability history you can inspect, and a shape of organization that is not telegraphing withdrawal. It does not mean the API is currently pleasant to use.

The distinction matters because the failure you are guarding against is temporal. A vendor can have excellent docs, a clean status page today, and still cut its deprecation notice period next quarter. Your job is to read the track record, not the current mood.

By the end of a proper assessment you should be able to answer four questions in writing: what you use, how often you use it, how long a migration would take, and whether you have an abstraction layer in place. If you cannot answer all four, you have not finished vetting; you have finished admiring.

> **Rule:** Decide against a written threshold, not a vibe
>
> Before you score anything, write down the notice window you require and the reliability floor you accept. Sources disagree on the standard, so your number is the standard for this decision.

## The three verdicts and when each applies

There are three defensible outcomes, and the middle one is the point of the exercise. Build means the vendor's record clears your bar and the feature is not existential if it breaks. Build-with-abstraction means the vendor is acceptable but you require an adapter that isolates every vendor call. Walk away means the record fails your bar and no abstraction makes the dependency worth it.

Most real vendors land in the middle. That is not indecision. Teams with a documented deprecation-response framework and an adapter migrate in 2-5 days on average, while teams without one take 2-4 weeks. Abstraction is the single highest-leverage control you have, because it converts weeks into days without changing the vendor at all.

**4-8x - Migration speed-up from a documented framework plus an adapter**

Teams with isolation migrate in 2-5 days; teams without take 2-4 weeks.

#### Vendor record against feature criticality

Horizontal axis runs from Feature is peripheral to Feature is core and existential. Vertical axis runs from Vendor record is strong to Vendor record is weak.

| Quadrant | What it means |
| --- | --- |
| Strong record, peripheral feature | Build directly, revisit annually |
| Strong record, core feature | Build, but keep a thin adapter as insurance |
| Weak record, peripheral feature | Build-with-abstraction or accept the churn |
| Weak record, core feature | Walk away, or abstract heavily and plan the exit |

*The upper-left corner is where an abstraction layer stops being optional.*

## Reconstructing the deprecation track record

The deprecation record is the spine of the whole vetting, and you rebuild it from five public artifacts: the changelog, any deprecation or sunset pages, public OpenAPI specs, the status-page incident archive, and the developer forum. Together these let you see how the vendor has actually behaved across breaking changes, not how it says it behaves.

The best vendors make this easy on purpose. Google publishes per-product deprecation tables tied to its Terms of Service that list each deprecated feature and its scheduled shutdown date. That is a durable public record: a feature stays available for at least the period defined in the Terms after deprecation, and a table shows the deprecation and its related shutdown schedule. When a vendor publishes a table like that, you can audit it. When a vendor buries deprecations in scattered blog posts, that opacity is itself a finding.

Request several years of history. A short incident or deprecation history proves little at scale, so two to three years is a defensible reconstruction. Note that no published standard fixes an exact lookback in years, so treat the two-to-three-year span as a working convention rather than a rule handed down from anywhere.

> **Watch out:** A policy on paper is not a policy honored
>
> A published 12-month notice window is worthless if past changes broke clients early. Cross-check the written policy against the dated changelog, and watch for security exceptions that some policies allow to ship without warning.

### Scoring notice discipline

Once you have the dated list, score each past breaking change against your required window. The window you choose is contested, so choose deliberately and state it. The table below is the spread of named policies you are choosing among.

| Vendor/standard | Stated notice window |
|---|---|
| Internal APIs (best practice) | 30-90 days |
| Antler industry-standard range | 3-8 months |
| Public APIs (best practice) | 6-12 months |
| Google (public commitment) | ~12 months |
| Microsoft Graph | ~24 months |

For a public-facing API your product depends on, 6-12 months minimum is the common floor. IBM Verify, for context, sets a 12-month deprecation period with at least 12 months of notice before end-of-life. If the vendor you are vetting has historically given less than your chosen floor, that is a fail per change, and enough fails should move you toward walk-away regardless of how good the API looks.

The most dangerous pattern is not a low number. It is a shrinking one. Google moved App Engine, Maps/Earth, and the YouTube API from a 3-year deprecation policy to a 1-year policy in 2014 while insisting the APIs stayed live. That policy change telegraphed reduced commitment years before any removal. A vendor quietly cutting its notice period is a leading sunset indicator, so diff the current policy against older captures of the same page.

## Diffing successive API specs for versioning discipline

Versioning discipline is what separates a vendor that plans its breaking changes from one that ships them into your running code. You measure it by pulling successive public OpenAPI specs and diffing them, then checking whether breaking changes were gated behind new major versions.

Pull consecutive spec versions from the docs or from Git history, then diff them with oasdiff, a command-line tool that compares OpenAPI specs and detects breaking changes. Run it locally, in CI via a GitHub Action, or in a hosted PR-review workflow. It detects 681 distinct changes across 10 spec areas, supports OpenAPI 3.0, 3.1, and 3.2, and returns exit code 0 when no breaking changes are found and 1 when they are, which is what lets you gate on it in CI. Azure/openapi-diff is a workable alternative if you are already in that ecosystem. For the versioning-discipline reference itself, Google AIP-185 is the standard to read.

What you are looking for is gating. Well-run vendors incorporate deprecations and breaking changes into a new API version, so users of existing versions are undisturbed and can upgrade at their own pace. If your diff shows breaking changes landing inside the same version clients already call, that is a discipline failure no reliability number offsets.

**oasdiff breaking-change check between two spec versions**

```
# Compare last-known-good spec against the newest published spec
oasdiff breaking specs/vendor-v1.yaml specs/vendor-v2.yaml

# In CI: fail the build on any breaking change
oasdiff breaking specs/vendor-v1.yaml specs/vendor-v2.yaml \
  --fail-on ERR
# exit 0 = clean, exit 1 = breaking changes found
```

*Point the two paths at consecutive public spec versions; a non-zero exit means breaking changes to review.*

Two cautions before you trust the count. First, tools can report a breaking change that in practice cannot break your application, so verify each flag against how your client actually uses the endpoint. Second, and more serious, a diff only sees what the contract declares. A change in behavior your OpenAPI definition does not describe is invisible to any specification-based tool. That makes spec faithfulness itself a vetting criterion: a vendor with a thin or stale spec can ship breaking behavior changes no diff will ever catch.

## Reading reliability without being fooled by the headline number

Reliability is not the uptime percentage. It is the incident count, the longest single incident, and the time to recovery behind that percentage. Pull those from the status-page archive over 12 or more months, because the percentage alone is an average that hides exactly the tail risk that breaks core features.

The arithmetic makes the point. Here is what each SLA tier actually buys you.

| SLA tier | Downtime/month | Downtime/year |
|---|---|---|
| 99.9% | ~43 min | ~8 h 46 m |
| 99.99% | ~4 m 19 s | ~53 min |
| 99.999% | ~26 s | ~5 m 15 s |

A 99.9% month allows about 43 minutes of downtime. That could be one clean 43-minute incident or dozens of short interruptions that make your feature flap all day. The percentage cannot tell you which, so pair the monthly percentage with incident count, longest incident, and time to recovery. Those three reveal whether the service had one contained failure or remained unstable throughout the month.

Be equally skeptical of SLA credits. A generous credit hides a weak record, and it activates only after the target has already failed. A documented failover test is more persuasive than any credit, because it shows the vendor practices recovery rather than compensating you for its absence.

> The uptime percentage is an average, and averages are where core features go to die quietly.

## Assessing community and organizational health

Community health tells you whether the platform is actively maintained or drifting toward abandonment, and you read it from commit frequency, PR activity, issue-resolution rate, release cadence, and bus factor. Organizational longevity is a weaker public signal, but headcount and DevRel investment give you a cheap read.

Bus factor is the concentration risk that matters most. Measure it from commit-author concentration: if 90% of commits in the last six months come from one email address, the risk is critical. The trap is that raw activity can hide this. Toxic gatekeeping keeps bus factor at one while looking healthy on every metric that counts commits and closed issues, right up until the one person stops and there is no successor. This is not theoretical. In the 2018 event-stream incident, the original author handed the package to a volunteer who added a wallet-stealer to a downstream dependency; concentration risk is how a single handoff becomes a supply-chain event.

Developer-relations headcount is a useful, hard-to-fake proxy for deliberate community investment. In Refolk's index, the US has about 21x more SREs than DevRel staff (6,535 versus 307), so a vendor spending visibly on DevRel is spending on the community on purpose, not by default. Top current employers of US DevRel professionals in Refolk's index include Stripe, Google, Roblox, Telnyx, and AssemblyAI, which is a rough map of who treats their platform as a product.

| Segment | Count | Derived vs US DevRel |
|---|---|---|
| DevRel, United States | 307 | 1.0x (baseline) |
| DevRel, United Kingdom | 50 | 0.16x |
| SRE, United States | 6,535 | 21.3x |

Judge that count against the local baseline. US DevRel supply is roughly 6.1x the UK's, so a UK-based vendor with a small DevRel team may still be proportionally healthy. An absolute headcount comparison across geographies will mislead you.

**21x - US SREs per unit of US DevRel staff in Refolk's index**

Because DevRel is scarce, visible investment in it is a deliberate signal.

When you want to check whether a vendor is genuinely staffing its platform team, or size the talent behind a competing option, describing the people you want in plain language beats scraping org charts. [Refolk](/) turns a request into a ranked list across the public GitHub graph, LinkedIn, and the open web.

I ran this search: `Developer advocates and developer relations engineers currently at Stripe, Twilio, and Telnyx in the United States.` - [see the full result list](https://www.refolk.ai/s/4knhnafgfz).

*Returns named DevRel professionals at those vendors, so you can gauge how deliberately each staffs its developer community.*

## The vetting procedure, start to finish

Here is the full method in order. Each step names an owner, a rough duration, and what done looks like, so a small team can run it in three to four days of effort spread across the right people.

#### Vet a platform vendor end to end

1. **Scope the dependency** - List every vendor endpoint your core feature calls and how often. Done is a written inventory of endpoints, call volume, and the exact failure your product suffers if each endpoint disappears. (analyst/eng lead, 0.5 day)
2. **Reconstruct deprecation history** - Pull the changelog, deprecation tables, and Sunset-header usage across two to three years. Done is a dated list of past breaking changes and the notice each got. (analyst, 0.5-1 day)
3. **Score notice discipline** - Set your required window, commonly 6-12 months for public APIs, and check whether historical notices met it. Done is a pass/fail per past change with your threshold stated. (eng lead, 0.5 day)
4. **Diff successive API specs** - Run oasdiff across consecutive public OpenAPI versions to count real breaking changes per release. Done is a breaking-change count per version and whether breaks were gated behind new major versions. (engineer, 0.5 day)
5. **Pull incident history** - Archive the status page for sev-1 count, longest incident, and MTTR over 12+ months. Done is a table of incidents with durations. (analyst, 0.5 day)
6. **Assess community and org health** - Measure commit cadence, issue resolution, release frequency, bus factor, plus DevRel and headcount signals. Done is a healthy/at-risk verdict per signal. (analyst, 0.5 day)
7. **Specify the abstraction layer** - Define the adapter interface that isolates every vendor call so a future break touches one module. Done is a written interface spec and estimated migration cost. (eng lead, 0.5 day)
8. **Grade and decide** - Combine into build / build-with-abstraction / walk-away. Done is a signed one-page verdict with the abstraction requirements attached. (eng lead + stakeholders, 0.5 day)

#### How signals collapse into a verdict

1. **Scope** - What breaks, and how badly, if each endpoint disappears
2. **Record** - Deprecation notice history and spec-diff versioning discipline
3. **Reliability** - Incident count, longest incident, MTTR behind the percentage
4. **Community** - Bus factor, cadence, and DevRel investment
5. **Verdict** - Build, build-with-abstraction, or walk away

*Deprecation record and versioning discipline are gates; the rest weight the abstraction decision.*

## How this goes wrong: failure modes and false positives

Most bad vendor decisions come from trusting a signal that lies, so this is the section to read twice. Each of the following is a way a clean-looking metric hides the risk you were trying to measure.

- **Healthy-looking metrics, dead project.** High commit and closed-issue counts can hide a single hostile maintainer. Check bus factor by commit-author concentration, not raw activity, because toxic gatekeeping looks healthy on every count-based metric until the one person leaves.
- **Spec-diff false positives.** A tool flags a breaking change that cannot break your real client. Verify each flag against actual client usage before scoring it, or you will walk away from a safe vendor over a phantom break.
- **SLA credit as false comfort.** A generous credit hides a weak record and only pays out after the target has already failed. A documented failover test is worth more than any credit.
- **Uptime percentage masks instability.** 99.9% can be one clean incident or dozens of interruptions. Demand incident count and longest incident, not just the percentage.
- **Notice window on paper, not honored.** A published policy is worthless if past changes broke early. Cross-check the policy against the dated changelog, and note that some policies allow security exceptions to ship without warning.
- **Contract-scope gap.** A spec-based tool only sees what the spec declares. Undocumented behavior changes will not appear in any diff, so a thin or stale spec is itself a red flag.
- **Shrinking window read as stability.** A vendor quietly cutting its notice period, as Google did from three years to one, signals reduced commitment even while the API stays live. Read the trend, not just the current value.

> **Tip:** When two signals conflict, trust the harder-to-fake one
>
> Commit counts are easy to inflate; bus factor from author concentration is not. A published policy is easy to write; a dated changelog of honored notices is not. Weight the evidence that would cost the vendor something to fake.

## Specifying the abstraction and closing the decision

The abstraction layer is what turns a weak or uncertain vendor record into a survivable one, so specify it before you decide, not after you get burned. Define an adapter interface that isolates every vendor call behind your own contract, so a future break touches one module instead of your whole codebase.

This is the highest-leverage control in the entire method. A documented deprecation-response framework plus that adapter is the difference between a 2-5 day migration and a 2-4 week one. Write the interface spec, estimate the migration cost against it, and attach both to your verdict. If the estimated migration cost is small because the adapter is thin and well-defined, build-with-abstraction becomes a genuinely safe choice even for a vendor whose record you only half trust.

Run this checklist before you call the vetting done.

#### Before you sign the verdict

- [ ] Every endpoint the core feature calls is inventoried with its call volume and its failure mode
- [ ] A dated list of the vendor's breaking changes over two to three years exists, scored against a written notice threshold
- [ ] oasdiff (or an equivalent) has been run across consecutive spec versions and breaking changes are counted and confirmed against real client usage
- [ ] Incident history over 12+ months records sev-1 count, longest incident, and MTTR, not just an uptime percentage
- [ ] Bus factor is assessed by commit-author concentration, and DevRel/headcount is judged against the local baseline
- [ ] An adapter interface spec exists with an estimated migration cost attached
- [ ] The one-page verdict names build, build-with-abstraction, or walk-away and lists the abstraction requirements

## Keeping the verdict current

A vendor vetting is a snapshot, and the signals it rests on drift. Re-run the deprecation and spec-diff steps at least once a year, and immediately whenever the vendor announces a new major version, changes its published deprecation policy, or has a visible incident cluster. The cheapest early warning is the policy page itself: capture the current deprecation window now, and compare it on each review, because a shrinking window is the signal that tends to arrive first.

Watch the community signal on the same cadence. A DevRel team that thins out, a release cadence that slows, or a bus factor that concentrates further are all leading indicators that the platform you built on is drifting toward the sunset you were trying to avoid. If any of them turn, pull the abstraction spec back out and re-price the migration before the vendor prices it for you.

## Frequently asked questions

### How many years of deprecation history should I reconstruct?

Aim for two to three years. No published standard fixes an exact lookback in years, so a two-to-three-year reconstruction is defensible rather than authoritative. A short window proves little at scale, because one quiet quarter can look identical to genuine discipline. Pull the changelog, any deprecation or sunset tables, and Sunset-header usage across that span so you can see whether the vendor holds its notice policy across multiple real changes, not just one.

### What notice window should I require before I score a vendor pass or fail?

State your own threshold, because published standards disagree. Best practice for public APIs is 6-12 months minimum, one industry survey puts it at 3-8 months, Google commits to about 12 months, and Microsoft Graph declares about 24 months for retiring versions in many cases. Pick the window your migration cost actually demands, write it down, then check historical notices against that single number instead of arguing the standard mid-assessment.

### Can a spec diff tell me everything about a vendor's versioning discipline?

No. A tool like oasdiff only sees what the OpenAPI contract declares, so a change in behavior your definition does not describe is invisible to any specification-based tool. A vendor with a thin or stale spec can ship breaking behavior changes no diff will catch. Treat spec faithfulness itself as a criterion, and cross-check the diff against how your client actually calls the API before you score anything as safe.

### Is uptime percentage enough to judge reliability?

No, because the percentage is an average that hides tail risk. A 99.9% month allows about 43 minutes of downtime and can be one clean incident or dozens of short interruptions. Pair the monthly percentage with incident count, longest incident, and time to recovery. Those three reveal whether the service had one contained failure or stayed unstable all month, which is the difference that actually breaks a core feature.

### How much does an abstraction layer actually save on migration?

Enough to change the decision. Teams with a documented deprecation-response framework and an adapter that isolates vendor calls complete migrations in 2-5 days on average, while teams without one take 2-4 weeks. That is a 4-8x reduction driven entirely by isolation and preparation. This is why build-with-abstraction is a real third option: it converts a forced multi-week scramble into a scoped few-day swap.

### Why does developer-relations headcount matter for vendor health?

It is a cheap, hard-to-fake proxy for deliberate community investment. In Refolk's index the US has about 21x more SREs than DevRel staff, so a vendor spending on visible DevRel is doing it on purpose, not by default. Judge the count against the local baseline, though: US DevRel supply is roughly 6.1x the UK's, so a UK vendor with a small DevRel team may still be proportionally healthy.

---

*From the Refolk guide library. I revise these guides rather than replacing them, so the current version is always at https://www.refolk.ai/guides/vetting-platform-vendor-core-feature*