Most peoplefine-tune models.We build them.
Tantra AI Labs builds open-weight language models from scratch — every layer, every weight, every token — plus the reasoning datasets and production infrastructure to run them.
AstraGPT-7B
7B decoder-only LLM. Custom BPE tokenizer, GQA (28Q/4KV), RoPE, SwiGLU. Native thinking chain-of-thought. 32K context.
View on Hugging Face 
AstraGPTCoder-7B
Coding-specialized sibling. LoRA fine-tuned for code generation with think-tag reasoning. Runs in ~6GB VRAM at 4-bit.
Every layer, hand-built.
No AutoModel. No copy-paste. AstraGPT's transformer, BPE tokenizer and supervised fine-tuning loop were written from first principles in PyTorch.
- Grouped-Query Attention
- 28 query / 4 KV heads, 7× less KV memory
- RoPE
- θ = 1,000,000 for long-context positions
- SwiGLU FFN
- intermediate width 18,944, no GELU
- RMSNorm
- pre-norm, no bias terms
Architecture flow: token IDs enter a token embedding, pass through 28 AstraGPT blocks with pre-norm RMSNorm, grouped-query attention, rotary embeddings and a SwiGLU feed-forward network with residual connections, then a final RMSNorm, LM head and logits.
AstraGPT Reasoning Dataset
Instruction–reasoning–answer triples with explicit cognitive traces. Apache-2.0.
Download dataset
High-Reasoning Dataset v1
Multi-AI knowledge-distilled. Deep CoT, math proofs, production code, edge cases. MIT.
Download datasetFrom training rigs to production gateways.
AstraGPT Training
Bare-metal VPS, 2× RTX 4090 (48GB). Custom PyTorch loop — BF16, grad accumulation, cosine LR, W&B logging.
Astra AI Gateway
Production LLM gateway. 40+ models, multi-channel routing, fallbacks, HTTP/2 zero-buffer streaming, cost analytics.
vLLM Serving
Self-hosted inference with prefix caching. Optimized connection pooling cuts per-request latency ~40%.
Try the gateway — one OpenAI-compatible endpoint, 40+ models
curl https://api.modelrouter.in/v1/chat/completionsBuilt by one engineer.
Tantra AI Labs is founded by Aditya Wakharkar — an AI infrastructure and cybersecurity engineer from Nanded, India. From training 7B models on a private GPU rig to shipping production gateways and ethical-hacking tools, the lab operates on one belief: open weights, built honestly.