Arctic oracle provenance and reconstruction

The Arctic pin preserves the upstream oracle’s per-utterance scoring rows. The oracle is historically attributed to stock SphinxTrain, subject to the provenance limits below. Its rows are stored in oracle-sidecar.json. Those rows make the recorded comparison checkable: they retain the reference-word and error counts for each decoded utterance, the aggregate scores, the PocketSphinx and Python identities used for decoding, hashes of the decode dictionary and language model, and hashes of the preserved model files.

That is tamper evidence for the recorded comparison, not independent provenance for the stock training run. The sidecar does not name a SphinxTrain source revision, build toolchain or build command. It does not contain the stock training configuration, corpus archives or training-file list, training log, decode command, or scoring command. Its original model paths point outside the repository. Recomputing the rows from preserved model bytes would check decoding, but would not reconstruct the models.

Keeping the live comparison resource-matched

A paired comparison across two lexicons measures the lexicon as much as the model, so the live oracle cells are decoded again from the preserved model bytes whenever the pinned decode resources change. scripts/regenerate_arctic_oracle.py does that: it authenticates the preserved model directories against the manifest digests already in the sidecar, decodes the preserved multipron-on model through pstrain.benchmarks.arctic.score_model under the current pinned dictionary, language model, and decoder settings, and refuses to emit a sidecar whose stated resource match disagrees with the record. make pin-check reruns that authentication without decoding, and scripts/regenerate_arctic_paired_analysis.py fails outright if the two arms name different dictionary or language-model bytes.

Staging is isolated. Each arm is copied into a directory created empty for that arm, so no artifact from an earlier run can survive into the model the decoder consumes, and the sidecar records an identity for the staged directory that was actually decoded alongside the preserved source digests. Regeneration refuses to proceed unless the staged directory carries exactly the authenticated file set and differs from the source only in the one file it declares it replaced.

Why the sidecar names a different commit than the one carrying it

The sidecar’s engine.git_describe names the commit whose working tree produced the rows, and its tracked_modifications_sha256 is the digest of an empty git status, so that tree was clean when the run started. That is never the commit the finished sidecar is committed as. An artifact cannot record the hash of the commit that carries it, because adding the artifact changes that hash; the producing commit is always an earlier tree, either the carrying commit’s ancestor or its sibling one amend behind.

The current sidecar was generated by the tree at 8b1ebf4 and is carried by a descendant of it, whose only differences from that tree are the generated evidence under evidence/arctic-pin/ and the documentation reporting it.

Read the recorded commit as the producing tree, and check the relationship rather than assuming it. Everything the measurement depends on should be byte-identical between the two, so this should be empty:

git diff "$(jq -r .engine.git_describe evidence/arctic-pin/oracle-sidecar.json)" HEAD \
  -- pstrain scripts tests Makefile benchmarks evidence/arctic-pin/record.json

What legitimately differs is the generated evidence under evidence/arctic-pin/ and the prose that reports it. If anything else differs, the recorded identity no longer authenticates the code that produced the rows, and the oracle should be regenerated rather than explained.

Two properties of the re-decode are measured rather than asserted, and both leave result and row digests in the sidecar that --check recomputes.

The preserved oracle models carry a nine-field SphinxTrain feat.params, and the current complete-model contract requires twenty-three fields. The regeneration completes the staged copy with PocketSphinx’s own built-in defaults, which are the values the absent fields already had. It then decodes the preserved model both ways — once with the original nine-field record and once with the completed record — and refuses to emit a sidecar unless every per-utterance row agrees. front_end_reconstruction.neutrality_control holds both arms’ counts and digests together with the dictionary and engine the control ran under, and the completed arm’s digests are recomputable from the emitted rows. The preserved bytes themselves are never modified, and the recorded preserved-model digests are computed from the originals.

The re-decode is not a reproduction of the 2026-08-11 oracle rows. Decoding the same preserved model with the same historical dictionary on the current engine yields 144 errors on on/slt55 against the historical 138, and 22,638 on on/big against the historical 22,487; that control is rerun during every regeneration and retained in historical_dictionary_control, which keeps the historical rows themselves alongside the digests, so the difference stays checkable from this repository alone. The oracle arm is therefore engine sensitive, and the historical rows are an attested measurement on their producing engine rather than a fixed reference point. That is as far as the evidence goes: it does not establish that the two arms move by the same amount or in the same direction, and it does not allocate any earlier reported difference to the engine alone, since the pstrain arm was also retrained under a different dictionary over the same interval. The same control also shows that the dictionary deduplication did not move the oracle arm at all: decoded under the earlier dictionary the preserved model produces the same rows as under the current one, which is recorded as dictionary_change_moved_rows. The retired off cells keep those historical rows and their producing identity in the sidecar’s historical_provenance; they are reported on their own resources and are never presented as resource-matched to the live pin.

The retired arms are comparable as paired historical decode measurements on their own recorded path, NOT COMPARABLE to the live pin, and NOT COMPARABLE for implementation attribution because their model-producing identities are missing or incomplete.

The live arms are comparable as paired decode measurements because they share the current decode and scoring path, but they are NOT COMPARABLE for implementation attribution because the oracle model’s producing host, architecture, build, training configuration, inputs, and full lineage are unknown. Under that resource-matched comparison SLT-55 sits below the oracle on the point estimate and big sits above it, and both paired 95% intervals include zero, so neither cell shows a statistically significant difference. The measured deltas and intervals are in the pin’s generated baseline table, which is rebuilt from the checked-in evidence and is not restated here.

Procedure for an independently derived stock arm

The following is the work required to create a comparable arm. It is not a record of a procedure executed for the checked-in oracle.

1. Pin a clean stock SphinxTrain checkout by full commit and record the compiler, compiler version, build flags, platform, and hashes of the resulting programs. Development records name 694c10099a50d11fd31ca824044bbe957ba650bd as a stock revision that was rebuilt successfully on a narrower CI-training rung. The oracle sidecar does not bind its models to that revision, so it is a reconstruction candidate, not established oracle provenance. If another revision is used, record why and keep its full identity with the result.

2. Reconstruct the CMU Arctic SLT training project from the corpus identities and 1,043-utterance training-file-list hash in the pin record. The checked-in record supplies archive, transcript, and file-list hashes, but not a stock project tree or a mapping from those inputs to a complete sphinx_train.cfg. Recover or recreate that configuration, then check it in or preserve it verbatim with a digest. Do not treat a configuration inferred only from final model dimensions as the producing configuration.

3. Produce separate stock multipron-off and multipron-on models, recording every effective stock setting, stage input, realized pass count, omitted utterance, and output-model hash. The historical development notes refer to external run.sh and etc/sphinx_train.cfg files and identify CFG_MULTIPRON_TRAINING=yes as the on-arm change, but those files are not in this repository. Until they are recovered or replaced by a complete, reviewed configuration, the historical oracle recipe remains incomplete.

4. For a pstrain comparison intended to mirror stock alignment behavior, set training.retry_beam_factor=1, training.failed_alignment=omit, and training.optional_final_silence=false, as required by the parity and deviations register. Also select the stock-compatible inventory and sharding postures described there, and pin the front-end difference rather than assuming the two arms consume identical feature bytes. These are comparison controls for the pstrain arm, not claims about stock configuration keys.

5. Decode the reconstructed models under the pin’s recorded PocketSphinx, language-model, dictionary, filler-dictionary, beam, language-weight, and word-insertion-penalty identities. Preserve the exact decode and scoring commands. Emit per-utterance rows in the sidecar schema and compare both rows and model hashes with the checked-in oracle. A mismatch is a new measured arm unless the missing producing-run provenance is recovered; it must not silently replace or relabel the preserved oracle.

This procedure becomes a reproduction of the checked-in oracle only when the missing source/configuration/toolchain link is supplied and the resulting models and rows agree. At present it is a route to an independently derived, comparable stock baseline. The preserved oracle remains an attested historical measurement rather than a fully reproducible build.