Training AstraGPT-7B on 2× RTX 4090: the actual recipe
The full configuration behind the 7B model: architecture, LoRA settings, batch math, hardware and what we would change next time.
On this page
Why write the recipe down
Most 7B fine-tune posts show a loss curve and stop there. The interesting part is the configuration: what was frozen, what was trained, how long it ran, and which choices were forced by the hardware. This is the exact recipe behind AstraGPT-7B, transcribed from the published model card.
Two RTX 4090s give you 48GB of VRAM and no NVLink. Everything below is shaped by that constraint.
The model we fine-tuned
AstraGPT-7B is a decoder-only transformer with a custom BPE tokenizer. These are the numbers that matter for memory planning:
| Component | Value |
|---|---|
| Parameters | 7B (fp16, 15.2 GB) |
| Layers | 28 blocks, pre-norm RMSNorm |
| Hidden size | 3,584 |
| Attention | GQA, 28 query / 4 KV heads |
| Positional encoding | RoPE, θ = 1,000,000 |
| Feed-forward | SwiGLU, intermediate 18,944 |
| Context | 32,768 tokens |
| Vocabulary | 152,064 |
The 28/4 grouping is what makes long context fit: four KV heads instead of 28 cuts KV-cache memory by roughly 7×, which is the difference between a 32K context that fits and one that does not.
LoRA, not full fine-tuning
Full fine-tuning a 7B model needs weight, gradient and optimizer state in memory — far past 48GB at fp16. LoRA keeps the base frozen and trains low-rank adapters, so the optimizer state stays small.
| Parameter | Value |
|---|---|
| Method | LoRA (PEFT) via Unsloth |
| Rank / alpha | 16 / 32 |
| Target modules | q, k, v, o, gate, up, down projections |
| Learning rate | 2e-4, cosine schedule, 5% warmup |
| Batch | effective 16 (gradient accumulation 8) |
| Precision | BF16 mixed |
| Optimizer | AdamW 8-bit |
| Duration | ~3 to 4 hours |
| Post-training | adapter merged into base weights |
Rank 16 is small. The bet is that reasoning behaviour is a surface-level adaptation — the base model already contains the capability, and the adapter only teaches format and commitment. For a 193k-row dataset, that bet held: the model learned the <think> format without collapsing on general prompts.
The data
Training used 193,841 instruction-reasoning-answer triples from the AstraGPT Reasoning Dataset. Each output opens with a delimited Attempts Blocks section: decomposition, candidate approaches, self-correction and a confidence note, then the final answer.
The format matters more than the volume. Because the trace is fenced and consistent, the model learns where reasoning starts and where commitment begins, instead of blending both into one paragraph.
What we would change
- Log eval loss, not just training loss. Training loss kept falling while held-out reasoning accuracy plateaued.
- Cap thinking length during data generation. Some traces were longer than they needed to be, and the model imitates that.
- Run a small human preference pass before merging the adapter; it catches formatting regressions that loss does not.
- Treat 3–4 hours as a smoke test, not a finished model. More epochs on a curated subset beat more epochs on everything.
The full card, including limitations and the citation, is on Hugging Face.
The artifact
Read the full source behind this post.
AstraGPT-7B model card 
Aditya Wakharkar
Founder, Tantra AI Labs. Models, inference infrastructure and offensive security.