RESEARCH

Open work, with the numbers attached.

Model cards, dataset cards, an independent weight audit, and the exact fine-tuning configuration behind AstraGPT. Every value on this page comes from a published card, not from a slide.

Research changelog All artifacts on Hugging Face

Papers & reports

5 artifacts
Numbat Forensics — dark artifact slab with glowing fracture lines
SECURITY AUDIT · MODEL FORENSICSNov 2026 · MIT

Numbat Forensics

Independent audit of the public burner22/numbat checkpoint plus a truncated 1-of-114-layer model built from its real tensor bytes. Verdict: 633 norm and convolution tensors (11.37 M values) regenerate bit-for-bit from SHA256(tensor_name), and 373.6 M expert-matrix signs match the same name-seeded stream, both with zero fitted parameters. Only the token embedding looks trained. Not an official Anthropic model, and not a capable LLM — it is a reproducibility artifact.

4.73B
artifact params
633
tensors rebuilt
0.0027%
of checkpoint verified
Read paper Hugging Face
AstraGPT-7B — glowing neural-network illustration of a brain
MODEL CARD · TECHNICAL REPORTApr 2026 · Apache-2.0

AstraGPT-7B

7B decoder-only transformer with a custom BPE tokenizer (64k trained, 152,064 runtime vocab), grouped-query attention at 28 query / 4 KV heads, RoPE θ=1M and a SwiGLU feed-forward. Reasoned with a LoRA fine-tune on 193,841 instruction-reasoning-answer triples, giving native <think> traces.

7B
parameters
32K
context
28
layers
Read paper Hugging Face
AstraGPTCoder-7B — glowing neural-network data flow illustration
MODEL CARDApr 2026 · Apache-2.0

AstraGPTCoder-7B

Coding-specialised sibling of AstraGPT-7B. LoRA fine-tune tuned for code generation with the same think-tag reasoning, runs in roughly 6GB of VRAM at 4-bit quantization.

7B
parameters
~6GB
VRAM at 4-bit
think
reasoning
Read paper Hugging Face
AstraGPT Reasoning Dataset — glowing data network graph
DATASET · 193,841 ROWSFeb 2026 · Apache-2.0

AstraGPT Reasoning Dataset v1

Instruction-reasoning-answer triples where the reasoning trace is the primary artifact: every output opens with a delimited Attempts Blocks section covering exploration, dead ends, self-correction and commitment before the answer.

193,841
triples
4
columns
JSON / Parquet
format
Read paper Hugging Face
High-Reasoning Dataset v1 — data sphere and analytics panels
DATASET · 2,139 ROWS2026 · MIT

High-Reasoning Dataset v1

Multi-AI knowledge-distilled Q&A pairs with deep chain-of-thought, math proofs, production code and edge cases. Built by an autonomous two-model loop: one asking, one teaching.

2,139
Q&A pairs
MIT
license
CoT
distilled
Read paper Hugging Face

Experiment results.

The configuration that produced the published weights, transcribed from the model card. Where a number has not been measured, it is not printed.

Architecture · AstraGPT-7B

ParameterValue
Parameters7B (fp16, 15.2 GB)
Layers28 transformer blocks, pre-norm RMSNorm
Hidden size3,584
AttentionGrouped-Query, 28 query / 4 KV heads
Positional encodingRoPE, θ = 1,000,000
Feed-forwardSwiGLU, intermediate 18,944
Context window32,768 tokens
Vocabulary152,064 (custom BPE lineage)

Fine-tuning recipe

ParameterValue
MethodLoRA (PEFT) via Unsloth
Rank / alpha16 / 32
Target modulesq, k, v, o, gate, up, down projections
Learning rate2e-4, cosine, 5% warmup
Batcheffective 16 (gradient accumulation 8)
PrecisionBF16 mixed
OptimizerAdamW 8-bit
Hardware2× RTX 4090 (48GB), Ubuntu 22.04, CUDA 12.x
Duration~3 to 4 hours
Post-trainingLoRA adapter merged into base weights

Released datasets

DatasetRowsLicense
astragpt-reasoning-dataset193,841Apache-2.0
high-reasoning-dataset-v12,139MIT

What is not here yet

No benchmark leaderboard. The lab has not published MMLU, GSM8K or HumanEval numbers, because it has not run a pinned harness over these weights yet. When that harness lands, the results go on this page with the prompt set, the commit and the exact commands used. Until then, treat any benchmark number you see elsewhere as unverified.