The pipeline behind the Credential Value Index
This pipeline turns raw credential records into a public index of non-degree credentials. States use the index to fund programs.
I led it end to end and wrote most of it, with a small team. Provider matching was built with a colleague.
- about 20data sources reconciled
About this number
Provider records from the sources feeding the Credential Value Index. As of Oct 2026.
- 1 millioncanonical providers
About this number
Distinct provider names after entity resolution, in the July 2026 release of the Credential Value Index. As of Jul 2026.
- about 4.3 millionprovider descriptions written
About this number
Provider descriptions written for the Credential Value Index. As of Oct 2026.
Why it mattered
The Credential Value Index is the most comprehensive public index of non-degree credentials. States use it to judge programs, and Idaho has used it to assess programs for LAUNCH grant funding. For that to work, every credential has to be named the same way. Each one also has to be tied to the provider that offers it and sorted by what it teaches.
What the pipeline does
Every quarter, the pipeline takes in fresh credential records.
- The same provider arrives from many authoritative lists under different names. A colleague and I combined them into one record per real provider.
- Each credential is matched to a clean reference list and given a field-of-study category.
- Each credential is linked to the skills it teaches, drawn from a large skills taxonomy.
- The result is cleaned into a catalog. A person approves each release before it is published.
- A client can check its own credential list against the catalog. The check shows which of its credentials meet its standard and which are already in the catalog.
Result
Each quarterly release of the index comes out of this pipeline. The July 2026 release lists 1 million canonical providers, and about 4.3 million provider descriptions have been written.
Try the providers stage
This demo shows how provider records become one organization, using invented records. A colleague and I built the real version. Choose what two records must share before they are compared, then see which records end up as one organization.
Synthetic data, simplified mechanism
282 invented records of 94 invented organizations (82 training providers and 12 look-alike businesses), from 5 made-up sources. A fixed rule stands in for the judge.
Same state: 90 organizations found (94 true), precision 0.57, recall 0.86.
The demo data did not load; this is the default setting.
- All possible pairs39,621
- Compared (sharing the choice)6,648
- Judged the same organization387
- Organizations found
- 90 against 94 true
- Precision
- 0.57
- Recall
- 0.86
Precision is the share of merged pairs that belong to the same organization. Recall is the share of same-organization pairs that were merged.
- Osgood Community CollegeOsgood, PA · source E(a different true organization)
- Osgood Comm. Coll.Osgood, PA · source A(a different true organization)
- Osgood Coding AcademyOsgood, PA · source E(the main organization)
- The Osgood Coding AcademyOsgood, PA · source C(the main organization)
- OCA Osgoodno town, PA · source B(the main organization)
- Osgood Coding Acad.Osgood, PA · source D(the main organization)
- The Osgood Coding Acad.Osgood, PA · source A(the main organization)
- Osgood Truck Driving SchoolOsgood, PA · source C(a different true organization)
and 3 more
What I got wrong
- What I assumed going in
- That I could improve the catalog while porting it. Reproducing the original exactly first is what made every later fix measurable.
- What I'd measure first next time
- How much each stage changes the final catalog, before tuning any single stage.
- What it can't do yet
- Judge a credential's value on its own; that needs outcome data this pipeline does not hold.