Automating License Attribution for Unversioned Vendor Dumps

A folder named vendor/, third_party/, or lib/external sits in almost every mature repository. Someone dropped five thousand lines of code into that directory four years ago, removed the .git directory to prevent submodule errors, adjusted two imports to resolve compile errors, and moved on. The top-level LICENSE or COPYING file was omitted because it was not needed to make the build pass.

As we covered in vendoring code without a paper trail is unsecured debt, that vendored code works until compliance, diligence, or an audit asks where those lines came from. Without a provenance trail, you cannot produce an accurate license attribution statement. You are left with a folder of hope: hoping the upstream project was MIT rather than AGPL, and hoping no one stripped a mandatory attribution notice.

The failure modes of manual attribution

When teams attempt to reconstruct the provenance of unversioned vendored code, they usually rely on three flawed methods:

1. Grepping comment headers: If the original author included a copyright comment at the top of every file, this yields a name and a year. But many upstream projects keep licensing exclusively in a root LICENSE file. The individual source files have no headers at all. Grepping yields nothing.
2. Package manifest scanners: Standard Software Bill of Materials (SBOM) tools scan package.json, go.mod, or Cargo.lock. In-tree vendor dumps bypass package managers entirely. The scanner reports zero dependencies, giving a clean bill of health to a repository that may contain multiple distinct third-party libraries.
3. Fuzzy web searches: Developers search for unique function names or string literals in a search engine. This might locate the repository, but it rarely identifies the specific version or commit. Attribution requirements often change between major releases, and an incorrect guess creates legal exposure.

Manual attribution does not scale, and file-level license scrapers fail when the license text was never copied into the tree. Rebuilding provenance requires matching the structural signature of the code against known upstream releases.

Automating attribution through structural SBOM diffing

Accurate provenance automation does not rely on file names or comment headers. File paths change, namespaces get refactored, and comments get stripped by formatters.

A reliable pipeline works on structural matching:

[Unversioned Vendor Dump] 
       │
       ▼
[AST Tokenization / Normalisation]
       │
       ▼
[Hash Comparison against Upstream Corpus]
       │
       ▼
[Exact Version / Commit Identified] ──► [SPDX ID + Notice Emitted into SBOM]

1. Normalize the source: Strip local formatting, comments, and project-specific namespace prefixes. Tokenize the abstract syntax tree (AST) or generate structural hashes of core routines.
2. Diff against canonical release archives: Match the normalized token hashes against known upstream release trees. We discuss the mechanics of matching unversioned files to source commits in extracting origin commits from unversioned in-tree vendor dumps.
3. Emit the diff into the SBOM: Once the exact upstream release is identified, the pipeline extracts the license declaration, authors, and required attribution notices directly from the canonical source.

This turns an unversioned vendor dump into a deterministic inventory item: an exact upstream origin, an SPDX license identifier, and the complete text required for distribution notices.

How SHPBL handles vendored provenance

SHPBL eliminates licensing ambiguity by construction. Our software library is model-independent—every component is computed rather than generated, owned outright, and verified against published checksums.

When evaluating external codebases, SHPBL applies that same rigorous verification. Every file and component passing through our pipeline is evaluated by a Build Intent gate. The gate resolves licensing invariants and structural provenance in code before an artifact is admitted or synthesized. It does not guess based on directory names; it checks the code against verified invariants.

You can inspect your repository's existing vendor debt and missing attributions with a free repository evaluation, or integrate license verification into your local pipelines using the typed TypeScript client (@shpbl/sdk).

What to do on Monday

Do not wait for an acquisition audit or an open-source compliance inquiry to identify your third-party code.

1. Isolate your vendor directories: Run git log --diff-filter=A --stat across your repository to find bulk additions of untracked, third-party source files.
2. Stop relying on manifest-only scanners: Verify whether your current SBOM tooling inspects file trees or only parses lockfiles. If it only reads lockfiles, your in-tree vendored code is invisible to your compliance reporting.
3. Extract and commit notices: For every identified vendor dump, locate the exact canonical version, extract the required upstream NOTICE and LICENSE files, and place them alongside the code until the vendored tree can be replaced with an audited, trackable dependency.