The AI-Engineer Footprint Reference: What Each LLM Artifact Proves

You are screening an engineer's public AI and LLM work, and the footprint is crowded. This reference is for engineering managers, technical founders, developer-relations leads, and technical sourcers who need to triage a candidate's AI artifacts in minutes, not infer skill from a blog post that lists signals without saying when each one lies. It gives you one row per artifact type: what it proves about production capability, the failure mode that makes it look stronger than it is, and how long the signal stays fresh.

The scale is the first problem. GitHub counted 4.3 million AI-related repositories, nearly doubled in under two years, with 1.1 million public repositories importing large language model SDKs and 693,867 of those created in the past twelve months. The footprint is not thin. The skill inside it is.

Why the AI footprint needs its own reference

The AI footprint needs its own reference because the artifacts flooding profiles share names with production systems but rarely share their guts. A RAG app, an agent, an eval harness, a fine-tune, an MCP server, and a vector pipeline each prove a different thing, and each lies in a different way. Reading them as one undifferentiated pile of "AI work" is how a tutorial clone gets a phone screen.

The ecosystem grew fast enough that the ratio of keyword to capability inverted. Lightcast data shows generative AI skill postings rose from 16,000 in 2023 to 66,000 in 2024, large language modeling from 5,000 to 20,000, and prompt engineering from 1,400 to nearly 6,300. Demand pulled every engineer toward listing the stack. Supply of engineers who can actually ship a reliable AI feature with guardrails, monitoring, and measurable quality did not keep pace.

That ratio is the spine of this document. The keyword is cheap because engineers list the tool they imported, not the capability they proved. The eval table is scarce because it is expensive to produce and trivial to verify. Every row in the reference below is really a way of asking the same question: did this person import a framework, or did they ship and measure a system?

The artifact reference: what each type proves

Each artifact type sits on a spectrum from "ran a model locally" to "operated a system in production." The table below is the fast lookup. Jump to the row for the artifact in front of you, read what it proves, and note the failure mode that inflates it.

The ordering matters. An eval harness is the highest-value artifact precisely because it is the one most candidates skip. The failure a harness catches is specific: the system answers beautifully in the launch review, then quietly degrades. An embedding update shifts retrieval, a knowledge-base refresh changes chunk boundaries, a prompt tweak makes the generator embellish, and every one of those ships without an error. An engineer who built a regression gate has met that failure and designed against it. An engineer with a RAG demo and no evals has not yet learned it exists.

Reading the stack as layers

A single AI system stacks concerns, and strong artifacts show evidence at more than one layer. The weak ones stop at the model.

Most portfolios live entirely at the model layer. Around 70% of enterprise AI work involves observability, yet most portfolios omit it entirely. When you see the operations layer addressed at all, you are looking at a candidate who has operated something, not just invoked it.

The production markers that separate build from demo

The markers that separate production work from a demo cluster tightly, and practitioners converge on the same list. Look for all of them, and note which are present rather than scoring on any single one.

• Production RAG: hybrid retrieval plus reranking plus citations, not a single-vector lookup.
• An eval suite: faithfulness, context precision or recall, and a stated hallucination rate.
• A live deployment URL: a running endpoint, not a screenshot.
• README as product spec: problem statement, architecture diagram, eval numbers, and cost.
• Observability and cost tracking: instrumentation that records what each request cost and how it behaved.
• Error handling: retry, backoff, and cost caps.
• Quantitative results: numbers shown, not claimed.
• Open-source PR and issue history: evidence of collaboration, not a solo tutorial clone.

The single most reliable tell of a demo dressed as production is missing evaluation. A model with no stated accuracy, no baseline comparison, and no discussion of failure cases reads as unfinished even if the code runs. Evaluation is the part of the job that separates an engineer from a script. The second most reliable tell is cost and latency silence: production AI systems live and die on cost per request and response time, and if a repo never mentions either, the candidate is signaling they have only run a model locally.

The keyword is cheap and the eval table is scarce, so build your screen around what is expensive to fake.

This is where a sourcing tool earns its place. If you are starting from scratch rather than reviewing an inbound candidate, you want to find engineers who already show these markers, not read a thousand profiles to find the few who do. Refolk lets you ask for the capability in plain English and get back people whose public footprint actually carries it, so you spend your screening time on the markers above instead of on triage.

How to screen an AI footprint in under an hour

To screen an AI footprint, work from classification to verdict in a fixed order so you never over-read a single artifact. The procedure below takes roughly an hour per candidate and produces a one-line reason per artifact.

Sources disagree on weighting, and the disagreement is worth knowing. Some reviewers lead with the README as the product spec; others lead with commit and issue behavior as the authorship proof. Treat the README as the claim and the commit history as the evidence. When they conflict, the commit history wins.

How each artifact misleads: the false positives

Each artifact type has a characteristic false positive, and knowing it is more valuable than knowing what the artifact proves. The table below is the failure-mode lookup. When an artifact looks strong, check its matching row before you advance the candidate.

Two of these deserve extra weight because they catch the most polished fakes.

Stars and forks are not authorship. Not all GitHub activity in ML is worth your time. Someone can star 400 repositories and watch videos about PyTorch without writing a single line of production code. The fix is mechanical: open the contributors view, read the commit cadence, and ask whether this person wrote the core logic or forked a working project and renamed it.

Keyword inflation is the default state, not the exception. In Refolk's index, LangChain is self-reported roughly 41 times more often than RAG. The keyword is cheap and the eval table is not. When a profile lists RAG, LangChain, and agents but no repo carries a golden set or a live URL, you are reading a keyword, not a capability.

Signal freshness: when each artifact goes stale

AI artifacts go stale fast, and freshness is dictated by framework re-architectures rather than a fixed clock. AI orchestration frameworks change faster than the content teaching them can keep up. LlamaIndex and LangChain each re-architected their entire package layout in 2024 and removed their original headline abstractions within a year, so a course recorded twelve months ago often fails on the first import. The same logic applies to a candidate's repo: imports that predate a cutoff prove dated familiarity, not current skill.

No published study quantifies an exact signal half-life per artifact, so treat this as a reasoned inference from the deprecation calendar rather than a measured number. The mechanism is sound even where the precise value is not established: if you see ServiceContext or LLMChain in the imports, you are looking at skill that was current before the cutoff and has not been refreshed since. Re-run this check every time you screen, because the calendar keeps moving.

Eval metrics age more slowly than orchestration code. A golden set and a faithfulness measurement remain meaningful even when the surrounding framework has moved, because the discipline of measuring does not expire the way an import path does. When you find a dated repo whose evaluation approach is still sound, weight the discipline over the stale import.

The sourcing picture: how thin the verified pool really is

The verified pool is thin, which is the whole reason this screen pays off. In Refolk's index, 21 US AI/ML/LLM engineers list Retrieval Augmented Generation as an explicit skill, against 872 who list LangChain. Explicit RAG tagging runs 3.5 times higher in the US than Germany, where only 6 engineers carry the tag. Top employers for the US RAG pool include Atlassian, Grammarly, Apple, Databricks, and Truveta; in Germany the names include Recare, Fraunhofer HHI, and Gallatin AI.

The implication for screening is direct. Because the keyword is cheap and verified capability is scarce, a single eval-gated RAG repo with a live URL outranks a long list of profiles that merely mention the stack. You are not looking for the candidate with the most AI keywords. You are looking for the one whose smallest verified artifact survives every row of the failure-mode table. The market agrees with this emphasis: generative AI engineer postings rose 7 times from 2022 to 2024, and AI skills in other IT roles rose 35 times, which means demand concentrates on shipping capability, exactly the thing the production markers measure.

Before you call it done

Before you move a candidate to interview or reject, run this checklist against the strongest artifact in their footprint. It is the fastest way to confirm you screened for capability and not for keywords.

Keep this reference current by re-checking the deprecation calendar whenever you screen, because the cutoffs that date an artifact are the fastest-moving part of this document. The artifact types and their failure modes are stable; the import paths that mark a repo as fresh are not. When a new framework re-architecture lands, add its cutoff to the freshness table and re-read your pending candidates against it. The engineers worth interviewing will have repos that pass the newest cutoff, not just the ones that were current when they shipped.

The one durable rule underneath all of this: screen for what is expensive to fake. A live deployment plus an eval table is costly to produce and cheap for you to verify. A keyword costs nothing and proves nothing. Build every screen on the gap between those two, and the crowded footprint sorts itself out.