Numbat Forensics

Abstract

An independent audit of the public burner22/numbat checkpoint, plus a truncated 1-of-114-layer model built from its real tensor bytes. The question is narrow and testable: are those parameters genuinely trained, or can a deterministic program regenerate them? They can be regenerated. 633 norm and convolution tensors reproduce bit-for-bit from SHA256(tensor_name), and 373,620,632 expert-matrix signs match the same name-seeded stream, both with zero fitted parameters. Only the token embedding table shows learned structure.

What this is, and what it is not

This repository is a research artifact, not a checkpoint. It contains a runnable 1-of-114-layer model, the code that rebuilds it from the source shards, and every measurement behind the verdict. The source weights are not redistributed.

It is not an official Anthropic model, and it does not demonstrate full-checkpoint language-model capability. The uploaded card presents itself as a mid-pretraining checkpoint at step 1,830,000 of a ~1.96M step run; that claim comes from the uploader, not from this audit.

• Claimed identity: ~14 T parameters, 772 shards / 7.72 TB, 1,224 MoE experts, NVFP4 quantisation.
• Audited scope: 184.4 M parameters, or 0.0027% of the checkpoint (11.37 M bit-exact plus 173.0 M near-exact).
• Environment: 12 vCPU, 22 GB RAM, no GPU, 233 GB disk.

Method

The audit never downloaded a shard and never executed repository code. It issued HTTP range requests against the safetensors headers, pinned the source revision in every script, and rebuilt the forward pass from the config independently.

Parameter authenticity and provenance are different questions. This work answers the first: whether the bytes can be produced by a deterministic program from tensor names alone. It does not answer who uploaded the checkpoint, or where the embedding table came from.

Findings

633 norm and convolution tensors, covering 11,374,080 values, regenerate bit-for-bit from SHA256(tensor_name) with zero fitted parameters. 373,620,632 signs of non-zero E2M1 codes across 12 expert matrices match the same name-seeded stream, again with zero fitted parameters.

The token embedding table is the exception. It carries real learned semantic structure and is the only component in the audited slice that looks like a trained network.

• Norm and convolution tensors: bit-exact regeneration from the tensor name.
• Expert matrices: sign pattern matches the seeded stream across 12 matrices.
• Embeddings: genuinely trained, provenance unresolved.

Limitations and open questions

The artifact keeps layer 0 only. Layer 0 is dense and Gated-DeltaNet type, so no NVFP4 dequantisation is involved and the bytes are used exactly as stored. There is no MLA layer, no MoE layer, no MTP layer and no deep residual stream, so the model cannot produce coherent text; the published inference output is degenerate and a random-weight control produces the same output.

A truncation carries no provenance information. Nothing should be read into its output, and nothing here confirms or refutes who the uploader is.

• Unresolved: the per-tensor scalar S has no closed form found yet (S / weight_scale_2 ranges 106.2 to 143.7 across tensors).
• Unexplained: the 6.85 T and ~14 T parameter figures both come from the source repository.
• Untested families: MLA projections, mlp.gate routers, input_scale and weight_scale_2 scalar streams.
• Next experiment: recover the closed form of S, then rebuild an MLA layer's projections end-to-end.

Artifacts

The model card on Hugging Face carries the full methodology, per-test numbers and the research report. The truncated loader, inference implementation and analysis code are MIT licensed.

Cite this

@misc{wakharkar_numbat_forensics_2026,
  author = {Aditya Wakharkar},
  title  = {Numbat Forensics: Reproducible Truncated Model and Weight Analysis},
  year   = {2026},
  note   = {Independent audit of burner22/numbat at revision 6b7f8fc5},
  url    = {https://huggingface.co/adityawakharkar/numbat-forensics},
  license = {MIT}
}

The canonical version of this work lives on Hugging Face. If a number here and the card disagree, the card wins.