Tracking code provenance¶
When an experiment needs explaining after the fact, the question is always exactly what code produced this? The
version number cannot answer it. Because private package registries reject PEP 440 local version segments,
Stained Glass Core strips the commit hash from its version, so two feature branches forked from the same commit produce
identical version strings. And by the time a wheel is installed — especially a bytecode wheel inside a Docker image —
there is no .git directory left to inspect.
stainedglass_core.provenance answers it instead.
Quick start¶
from stainedglass_core import provenance
provenance.build_info()
# {'version': '3.65.0.post1.dev12',
# 'commit': 'd718a6a8d5841f16b4a0c...', # the full 40-character SHA
# 'branch': 'feat/new-loss',
# 'dirty': True,
# 'remote_url': 'git@github.com:protopia-ai/stained-glass-core.git',
# 'build_kind': 'bytecode-wheel',
# 'content_digest': 'sha256:0afbb16caed3a760...',
# 'source': 'build-stamp',
# ...}
This works identically for an editable feature-branch checkout, a plain wheel, a bytecode-protected wheel, and a bytecode wheel inside a Docker image.
Recording a training run¶
Log the short tag so a run is findable, and store the full record so it is reconstructable:
import wandb
from stainedglass_core import provenance
info = provenance.build_info()
wandb.init(
project="my-experiment",
tags=[f"stainedglass_core@{provenance.short_commit(info['commit'])}"],
config={"stainedglass_core": info},
)
build_info() is a plain JSON-serializable dictionary, so it drops straight into a run config, an MLflow parameter set,
or a job manifest. Format the tag to suit your tracker -- Weights and Biases, for instance, caps tags at 64 characters.
The runnable examples under docs/examples/ use this same pattern.
What each layer needs installed¶
The build stamp, direct_url.json and metadata layers need nothing beyond the package itself. The working-tree layer
reads live git state through GitPython, which is an optional extra:
Without it, an editable checkout skips that layer and falls through to whichever one can answer. The repository's own
make install-requirements installs the extra, so this only affects consumers who install stainedglass_core directly
without it.
Knowing how much to trust the answer¶
Every record carries a source field naming the layer that produced it:
source |
Applies to | Trust |
|---|---|---|
working-tree |
Editable or source install | Live git state, read at call time. |
build-stamp |
Any wheel, including bytecode and Docker | Recorded at build time; accurate unless the tree was dirty. |
direct-url |
pip install git+ssh://...@branch |
Commit recorded by pip. No branch dirtiness information. |
metadata-only |
Anything else | Version only. Treat the code as unidentified. |
If you gate on provenance quality, metadata-only is the value to alert on.
Reconstructing the exact code¶
build_info() carries everything needed to recover the code, but only a record that confirms all three of commit,
clean tree and pushed commit can be trusted to do so:
info = provenance.build_info()
# pip fetched a `direct-url` commit from `remote_url`, which shows it was pushed there, unless that URL is a local path.
fetched_by_pip = info["source"] == "direct-url" and not (
info["remote_url"] or ""
).startswith("file:")
pushed = info["on_remote"] is True or fetched_by_pip
if info["commit"] is None or info["dirty"] is not False or not pushed:
raise RuntimeError(
f"cannot confirm this build can be recreated (source={info['source']})"
)
print(f"git clone {info['remote_url']} && git checkout {info['commit']}")
Test each field for the confirming value rather than the failing one, because None means unknown, never fine:
commitisNone— ametadata-onlyrecord, or git could not be read. There is nothing to check out.dirtyisTrue— the build included uncommitted changes, so it never existed as a commit.Nonemeans git could not tell.on_remoteisFalse— the commit was never pushed, so nobody else can fetch it.Nonemeans nobody checked, which is always the case outside a build stamp (see below). Adirect-urlrecord needs no check, since pip fetched the commit fromremote_url, except from afile:URL, which is a local path. For an editable run, push before you launch and confirm it yourself, for example withgit branch --remotes --contains HEAD.
remote_url is recorded with any embedded credentials stripped, so a record built from an authenticated HTTPS checkout
names the repository without carrying its access token into every wheel, image and tracked run. Cloning a private
repository from that URL therefore needs your own credentials.
on_remote is recorded only by the build stamp: resolving it needs a network round-trip, which build_info() does not
perform. It is None from any other source.
Push before you build¶
Building a wheel from an unpushed commit produces an artifact whose recorded provenance nobody can act on. make wheel
and make bytecode-wheel therefore fail when the commit is absent from origin:
ERROR: Commit 0064d8b8... is not present on remote 'origin'. A build stamped with an unpushed
commit records a state nobody else can fetch, so the artifact cannot be reproduced.
Push the commit, or pass --allow-unpushed for a deliberately local build.
Push the branch and rebuild. For a genuinely throwaway local build, forward the escape hatch through make:
BUILD_INFO_FLAGS is accepted by make wheel, make bytecode-wheel and make build-info. Anything stamped that way
should not reach a shared training cluster or a customer.
A commit counts as present when it is reachable from a ref that origin advertises, not merely when it is one — a
pushed commit stops being a branch tip as soon as the next commit lands, and it is still fetchable. If the remote
cannot be reached, or any branch tip origin advertises is not in the local object store, on_remote is recorded as
None and the build only warns — neither is evidence that the commit is missing. A missing branch tip usually means
someone pushed after your last fetch; run git fetch origin and rebuild to settle it.
A pull-request CI run builds the synthetic merge of the branch into its base, which no ref keeps. GitHub regenerates
that commit under a new SHA whenever it rechecks mergeability, so it can disappear from origin mid-build. In that
run only, if the commit has exactly two pushed parents and re-merging them with git merge-tree reproduces its tree,
its code can still be recreated: on_remote is recorded as None and the build warns. The stamp records both parents
in parents, so merging them recreates the code after GitHub has discarded the merge's SHA. A merge holding a
hand-resolved conflict or any other local edit fails that comparison and still fails the build. Outside a pull-request run, an
unpushed merge is treated like any other unpushed commit, however cleanly it re-merges: push it before you build.
Confirming two deployments match¶
content_digest() folds the per-file SHA-256 entries from the installed wheel's RECORD into a single digest. It
identifies the code even when git state is missing or misleading, which makes it the way to confirm that a training job
and an inference proxy were installed from the same wheel:
from stainedglass_core import provenance
training_digest = provenance.content_digest()
# compare against the value reported by the proxy
It returns None for an editable install, whose RECORD lists only path-configuration shims. Use the working-tree
git state in that case.
The digest reports what was installed, not what is on disk now: it re-reads no files, so an edit made after
installation leaves it unchanged. Catching that needs a signature over the artifact — anyone able to rewrite an
installed file can rewrite the RECORD beside it, so no digest the package computes about itself can answer it.
Relationship to utils.git¶
stainedglass_core.utils.git.local_repo_commit_wandb_tags is the older helper, and the two now have
distinct jobs:
build_info() |
local_repo_commit_wandb_tags() |
|
|---|---|---|
| Editable checkout | yes | yes |
| Wheel, bytecode wheel, Docker | yes | no -- returns [] |
| Reports on | this package | any number of locally checked-out repositories |
Reach for build_info() unless you specifically need to describe several repositories at once -- your training repo
alongside this one, say.
Both apply the same ownership check. Repository discovery searches parent directories, so without it any package inside
a virtual environment created within a checkout is attributed to that checkout: asking the old helper about torch
returned stained-glass-core. A discovered repository is now used only when git ls-files --error-unmatch confirms it
tracks the file, so an unowned module yields [] rather than a confident wrong answer.