Mirsee · Silicon Research · July 2026
Can a custom ASIC physically embed the weights of one trained language model — fixed at manufacture in ROM or compute-in-memory — and serve inference of only that model? Yes: the digital mask-ROM + on-chip-SRAM route is credible today, and one company has already fabricated it. This is the full verified evidence base.
The viable route is a digital hybrid: model weights baked into mask ROM, paired with on-chip SRAM for the dynamic state (KV cache) and for small updatable adapter weights. It works only if the model is aggressively quantized — ternary to 4-bit — via quantization-aware training, which currently holds accuracy at the 1–4B scale.
For a 1–3B ternary model, fixed-weight silicon looks practical today: a ~57 mm² die at ~5 W with enormous batch-1 speedups. An 8B model is at the edge — it needs 2-bit weights (accuracy unproven at that scale) or a near-reticle-limit die. The analog compute-in-memory alternative is roughly three orders of magnitude short of the needed capacity.
A deep-research workflow decomposed the question into five search angles, fetched 25 sources, extracted 124 falsifiable claims, and adversarially verified the top 25 with three independent votes each. Every tile carries its evidence tier:
The single most important finding is a real, fabricated, commercially demoed chip (EE Times, Feb 2026): Taalas' HC1 hardwires the entire Llama 3.1 8B model, weights included, into mask ROM on TSMC N6, storing a 4-bit parameter and performing its multiplication on a single transistor. No HBM, no external DRAM. A public demo runs at chatjimmy.ai.
Two details answer the hardest design questions. KV cache: the ROM holds only fixed weights; a small on-chip programmable SRAM holds the KV cache and optional LoRA fine-tune weights. Obsolescence: a structured-ASIC-like approach means only two mask layers change to customize the chip for a new model — model-to-RTL in about a week, new silicon in about two months. The company has raised over $200M with ~25 employees and claims ~30 chips could serve DeepSeek R1 671B at ~12,000 tok/s per user.
All performance figures are Taalas' own claims; no independent benchmarks exist yet.
Autoregressive decode at low batch is memory-bandwidth-bound: fetching weights dominates cost, leaving GPU tensor cores around 28% utilized in decode versus ~92% in prefill. A contrast design (VitaLLM, TSMC 16nm, 0.223 mm²) that streams a 3B ternary model from external DRAM caps at 70.7 tok/s purely on LPDDR bandwidth. On-die weights remove that wall entirely — TOM's aggregate on-chip bandwidth is 200 TB/s, about 41× an H100's HBM3e.
Qualification: the advantage is for batch-1 / low-batch serving. At batch ≥ 32 decode becomes compute-bound; at very long contexts KV traffic dominates instead of weights.
Compiler-generated ROM reaches ~57.8 Mbit/mm² versus ~25.7 for SRAM. Better still, synthesizing ternary weights directly as standard-cell logic — exploiting BitNet's natural zero-density — beats even 3D-stacked DRAM by ~75%. This is the arithmetic that makes billions of on-die parameters plausible.
Synthesis and place-and-route estimates from preprints, not measured silicon.
Both are synthesis + place-and-route studies at TSMC 7nm — no fabricated chip exists for either — but together they bracket the design space.
| Design | TOM (Microsoft Research, 2026) | ROMA (SJTU/PKU/MSRA, 2025) |
|---|---|---|
| Model | BitNet-2B, ternary | 4-bit 3B or 2-bit 8B LLaMA |
| Storage | 498.5 MB ROM + 37.5 MB SRAM | 1.86 GB ROM + 304 MB SRAM |
| Die area | 56.9 mm² | 503.7 mm² |
| Power | 5.33 W power-gated (25.8 W without) | 33.1 W |
| Throughput | 3,306 tok/s peak; ~64× A100 end-to-end, batch 1 | >20,000 tok/s short-context; ~70× RTX 4090 avg |
| Context on-chip | 1,024 tokens (FP8 KV) | up to 4K tokens |
Verifier flags: TOM's headline "465×" is a peak corner figure (typical decode gap vs published bitnet.cpp A100 numbers ≈ 13×); ROMA's 20K tok/s falls to ~10K at 4K context; TOM's density multipliers appear transposed in the paper's prose.
~2–8B-class models; figures are vendor-claimed or simulated, not independently benchmarked, and model sizes differ — read as order-of-magnitude.
Dynamic KV cache. Both design studies hold it in distributed on-chip SRAM, tiled across the context dimension with a flash-decoding-style dataflow. Demonstrated on-chip context is 1–4K tokens — the likely product-defining constraint when users expect 32K–128K windows.
Model obsolescence. ROM base weights are immutable, but ternary QLoRA adapters living in SRAM (reusing the KV-cache arrays) let the chip specialize after manufacture. This fixes task drift, not full base-model obsolescence — Taalas' cheap two-mask-layer respin is the commercial answer to the rest.
BitNet-style ternary (1.58-bit) quantization-aware training maintains competitive accuracy, with the gap versus full precision shrinking as models grow — strongest evidence at 2–4B parameters. LUT-based ternary matmul cores are 2.2× smaller than dequantize-to-FP16 designs. Fabricated analog silicon (NeuRRAM) has validated 4-bit weights-in-memory inference, though only at CNN/LSTM scale.
The refuted-claims tile matters here: generic 2–4-bit post-training quantization being lossless was killed in verification. The feasibility case rests on QAT. Evidence for ternary parity at 7B+ with trillion-token training remains thin.
Analog CIM has actually shipped: IBM's Hermes (14nm + phase-change memory, 17M parameters), NeuRRAM (3M RRAM devices), Mythic's M1076 (weights as analog levels on flash, 40nm). But capacities top out at millions of parameters — about three orders of magnitude short of 1B+.
And NVM can't practically serve attention: dynamic operands must be written into the crossbar, but ReRAM endurance is ~10⁵ cycles versus SRAM's >10¹⁶. Every credible NVM-CIM transformer design routes attention into digital CMOS anyway. Instructively, d-Matrix started analog and pivoted to digital in-memory compute; Mythic ran out of money in 2022 ($140–170M raised) before reaching revenue — reportedly a commercial failure rather than a technical one.
| Approach | What's fixed | Weights live in |
|---|---|---|
| Nvidia GPU | Nothing | HBM (off-die) |
| Groq / Cerebras | Dataflow style | On-chip SRAM, reprogrammable |
| Etched Sohu | Transformer architecture | 144 GB HBM3E (off-die) |
| d-Matrix Corsair | Digital IMC fabric | 2 GB on-die SRAM + DDR |
| Taalas HC1 | Architecture + weights | Mask ROM, on-die |
Even Etched — the most aggressive transformer ASIC, ~$800M raised — keeps weights in off-die HBM. Architecture-fixed silicon already carries obsolescence risk: Sohu cannot run MoE routing, Mamba/SSM, or vision models. Taalas is the only player past the weights-in-silicon line.
| Node class | Mask set | Total NRE |
|---|---|---|
| 3–5nm leading edge | $10–40M | $30–100M+ |
| 7–28nm mid-range | $1–10M+ (7nm >$10M) | $5–30M |
| 28nm mature | ~$1.5M | dominated by engineering, IP and verification, not masks |
| 55/65nm | ~$0.5M | |
| 180nm | <$100K |
Sources repeatedly stress that verification, not masks, is the most underestimated NRE component — mid-range totals assume 20–50 engineers for 18–24 months. The break-even heuristic from the ASIC-Clouds literature (the "two-for-two rule"): if annual TCO on existing hardware exceeds the ASIC NRE by 2× and the ASIC delivers ≥ 2× TCO-per-op improvement, the ASIC pays. Weight density pushes this design toward advanced nodes — Taalas' 815 mm² N6 die reflects that tension — but their two-mask-layer respin converts "new model = full tape-out" into a cheap increment.
Warning: the cheapest route died. Efabless shut down in March 2025, taking chipIgnite and stranding Tiny Tapeout shuttles mid-fab (~$100/design is gone). Tiny Tapeout is migrating to IHP 130nm. The ultra-low-cost open-silicon path is fragile and grant-dependent.