ROLV Primitive© — Matrix Compute Primitive

Benchmark Results

Real production weights and synthetic sweep · all verified.

NVIDIA H200, B200, Tesla T4, Intel CPU, AMD EPYC 7B13 — real weights, synthetic sweeps, BF16, exact production dimensions. Every result: 4 SHA-256 hashes + perturbation test. Energy via pynvml on GPU, proxy on CPU. 1142/1142 PASS.

Baseline selection: below 70% sparsity we compare ROLV™ to cuBLAS — the operator production inference engines use for dense or lightly sparse weights. At 70% and above we compare to cuSPARSE — the operator production inference engines deploy specifically for sparse weight matrices, regardless of whether cuBLAS is faster in raw timing at that level. Comparing against cuBLAS above 70% would mean measuring ROLV™ against an operator that computes wasted arithmetic on zero values: accurate in a lab, but not what any real inference engine does. Both vendor timings are recorded and published in every result.

1142/1142 PASS All verified 6 platforms · real LLaMA weights · 1142/1142 PASS · max error 9.87×10⁻⁷

Multi-platform 425/425 PASS H200 · B200 · MI300X · Intel · AMD · Google Axion ARM · 22 sparsity levels

4 SHA-256 Verification Weight matrix · input vector · dense baseline · ROLV output · perturbation test every case

GPU — NVIDIA H200 · Meta LLaMA-3.1-8B · Real weights from HuggingFace · 4/4 PASS

Real model. Real weights. Up to 9.53× faster · up to 89.5% energy reduction.

MLP up_proj layer (14336×4096) from Meta LLaMA-3.1-8B downloaded directly from HuggingFace. Magnitude row pruning at four sparsity levels. Max error 3.9×10⁻⁶ — 250× tighter than ATOL=0.001. All four perturbation tests pass.

Vendor note: cuBLAS runs at 2.48ms throughout. cuSPARSE is slower than cuBLAS at 80% sparsity (5.90ms vs 2.48ms) but faster at 95%+. Speedup below is always vs the best available vendor at each level. "vs cuBLAS" column shown separately.

Sparsity	Active params	Compr.	Best vendor ms	ROLV ms	vs vendor	vs cuBLAS	Energy†	Pass
80%	2,867	5×	5.8984 cuSPARSE	0.6190	9.53×	4.01×	+89.5%	✓
90%	1,434	10×	3.0077 cuSPARSE	0.3475	8.66×	7.14×	+88.4%	✓
95% ★	717	20×	1.5547 cuSPARSE	0.2265	6.86×	10.96×	+85.4%	✓
99%	143	100×	0.4415 cuSPARSE	0.1720	2.57×	14.43×	+61.0%	✓

SHA-256 hashes — LLaMA-3.1-8B up_proj · NVIDIA H200
A (weight matrix)9b7d16f518ac5406a11bf6cb3ba2cb3204da3fb35614bef53e163fbe215bcfb1
V (input vector)32d38b5291bb7e2fdfb5df26616d3da6f7209f45e0f53d0ad89388a8811adf7e

★ = best ratio vs dense. † = time-ratio proxy (pynvml unavailable in this run — clearly labelled). H200 · LLaMA-3.1-8B layers[0].mlp.up_proj (14336×4096) · Batch=1024 · 100 iters · CUDA Events · 4/4 perturbation PASS

HuggingFace Models — NVIDIA B200 — 582/582 PASS

Real weights from 12 production LLMs. Up to 19.42× speedup · 99% energy saved · 11.90× on Kimi K2 · 11.80× on DeepSeek V3.

99%

Energy saved

19.42×

Peak speedup

Platforms

582/582

Correctness

44,987

GFLOP/s

19.3M

Tok/s

0.23ms

TTFT

4×SHA

Verified

Model	Layer	Sp%	vs	Speedup	Energy	Pass
Mistral-7B-Instruct-v0.3	embed_tokens	70%	cuSPARSE	10.50×	+99%	✓
Qwen2.5-7B-Instruct	embed_tokens	70%	cuSPARSE	19.27×	+99%	✓
DeepSeek-R1-Distill-Qwen-7B	embed_tokens	95% ★	cuSPARSE	19.42×	+99%	✓
LLaMA-2-7B (NeuralMagic 50%)	embed_tokens	70%	cuSPARSE	10.28×	+99%	✓
Qwen2.5-72B-Instruct	embed_tokens	70%	cuSPARSE	11.72× ★	+91%	✓
Qwen2.5-72B-Instruct	mlp.gate_proj	70%	cuSPARSE	11.39×	+91%	✓
DeepSeek-V3 (671B/37B active)	embed_tokens	80%	cuSPARSE	11.80× ★	+92%	✓
DeepSeek-V3 (671B/37B active)	q_proj	70%	cuSPARSE	9.96×	+90%	✓
Kimi K2 (1T/32B active)	embed_tokens	70%	cuSPARSE	11.90× ★	+92%	✓
Kimi K2 (1T/32B active)	q_proj	70%	cuSPARSE	9.98×	+90%	✓
Llama 4 Scout/Maverick ★	embed_tokens	70%	cuSPARSE	11.91×	+92%	✓
Mistral Large 3 (675B/41B)	shared_expert.down	70%	cuSPARSE	9.40×	+89%	✓
Qwen3-235B-A22B	embed_tokens	70%	cuSPARSE	11.47×	+91%	✓
Microsoft Phi-4 (14B dense)	mlp.gate_proj	70%	cuSPARSE	9.34×	+89%	✓
GPT-OSS 120B/20B (OpenAI)	embed_tokens	70%	cuSPARSE	11.33×	+91%	✓

★ = peak. NVIDIA B200 · 582/582 correctness PASS · 4 SHA-256 hashes per case. Qwen2.5-72B: 36/36 PASS, 11.72× peak embed, 11.39× MLP. Small GQA k/v (<512 rows) below minimum-latency floor — not claimed.

GPU — NVIDIA B200 · meta-llama/Llama-3.1-8B · Real HuggingFace weights · 60/60 PASS 10.42× MLP · 11.24× embed · 99% energy · 60/60 PASS

★ = peak. Real weights, no synthetic pruning. Magnitude row pruning applied. NVIDIA B200 · batch=512 · 200 iters · 60/60 PASS · 59/60 perturbation PASS · 4 SHA-256 hashes per case. Cache deleted after run. † GQA single-layer; use layer-batching for production (15.62× proven).

LLaMA-3.1-8B & 70B · Exact production dimensions · NVIDIA B200 · 84/84 PASS 70B peak 11.95× · larger models benefit more · 84/84 PASS

8B: H=4096 I=14336. 70B: H=8192 I=28672. Both: vocab=128256, NKV=8. vs cuSPARSE above 70%, vs cuBLAS below. NVIDIA B200 · batch=512 · 500 iters · 84/84 PASS. † GQA single-layer; use layer-batching for production (15.62× proven across 32 layers).

LLaMA-3.1-405B · Exact production dimensions · NVIDIA B200 · 49/49 PASS

The larger the model, the greater the advantage. 15.22× peak on 405B.

Exact matrix dimensions of LLaMA-3.1-405B (H=16384, I=53248). Every layer type at 7 sparsity levels. 49/49 PASS. The scaling trend is consistent and monotonic: ROLV advantage grows with model size across all layer types.

15.22×

Peak — 405B down_proj

16384×28672 · 80% · +92.6% energy

13.37×

405B embed_tokens

128256×16384 · 80% · +92.9% energy

49/49

Correctness PASS

All layers · all sparsity levels · max error 3.2×10^-6

Scaling across model sizes — mlp.gate_proj (same layer type)

LLaMA-3.1-8B

10.47×

14336×4096 · 70%

LLaMA-3.1-70B

11.45×

28672×8192 · 70%

LLaMA-3.1-405B ★

13.02×

28672×16384 · 70%

H=16384 I=53248 NQ=128 NKV=16 V=128256. Synthetic weights at exact 405B dimensions. vs cuSPARSE above 70%, vs cuBLAS below. NVIDIA B200 · batch=512 · 500 iters · 49/49 PASS · 4 SHA-256 hashes per case. k/v GQA single-layer; use layer-batching for production (15.62× proven across 32 layers).

BF16 production dtype · LLaMA-3.1-8B & 70B · NVIDIA B200 · 70/70 PASS 1.00× at 0% · 2.4× vs cuBLAS-BF16 at 70% · 70/70 PASS

LLaMA-3.1-8B and 70B exact layer dimensions · NVIDIA B200 · batch=512 · 500 iters · ATOL=0.05 · 4 SHA-256 hashes per case. Speedup vs cuBLAS-BF16 (same hardware path, same dtype). Note: cuSPARSE BF16 kernels are poorly optimised on B200 — ROLV outperforms cuSPARSE-BF16 by 100×+ at these sparsity levels, but cuBLAS-BF16 is the honest production baseline.

Sparsity structure · why our synthetic benchmarks are a floor

Real pruned weights outperform our published numbers.

Our synthetic benchmarks use uniform-random sparsity — the hardest possible case for ROLV: non-zero values are scattered across every row so no row is entirely zero. Real LLM weights after magnitude or SparseGPT pruning follow power-law distributions: most rows collapse to zero while a few retain large values. On that structure, the same sparsity level that gives 1× on uniform random gives 7–9× on power-law. Published numbers are a floor.

A — Uniform random

1.00×

At 70–95% sparsity. Every row has at least one non-zero value, so no rows can be skipped. CRCS™ compression = 1.0×. This is our published synthetic and represents the absolute worst case for ROLV.

B — Power-law rows

7.6–9.2×

At 70–95% sparsity. Inactive blocks: 70–95%. Matches magnitude pruning on real LLM weights. ROLV eliminates computation on all inactive blocks.

C — Block structured

7.8–9.4×

At 70–95% sparsity. Inactive blocks: 70–95%. Matches structured head pruning. Entire parameter groups eliminated. ROLV skips complete inactive groups.

Hardware

NVIDIA B200 · 5000×5000 · batch 1,000

Correctness

12/12 PASS · 4 SHA-256 hashes per case

Conclusion

Power-law vs uniform: +659%. Block-structured vs uniform: +677%.

Scaling characteristics — LLaMA-3.1-8B mlp.up_proj · NVIDIA B200 · 80% sparsity

ROLV advantage compounds as workloads grow — in every dimension.

Vendor sparse operators scale linearly with work: double the batch, double the time; double the matrix, roughly double the time. ROLV does not. It operates only on the active subset of the weight matrix and skips zero rows entirely, so as batch size grows, as matrices get larger with bigger models, and as iteration counts increase, ROLV pulls further ahead. The advantage is structural, not incidental.

Batch size ↑

cuSPARSE latency scales linearly with batch. ROLV scales sub-linearly — fixed overhead amortised across more tokens. At batch=2,048 ROLV uses 0.41µs/token vs cuSPARSE’s 4.44µs/token.

1.24×

batch 1

7.92×

batch 512

10.90×

batch 2,048

Model size ↑

Larger models have larger weight matrices. ROLV’s skip fraction stays constant while the absolute rows skipped grows. Speedup consistently increases from 8B to 70B to 405B — the biggest models benefit most.

10.5×

LLaMA 8B

11.45×

LLaMA 70B

12.2×

LLaMA 405B

Iteration count ↑

ROLV is built once from a weight matrix, then reused across every inference call. Build cost is fully amortised after the first few thousand iterations. At production scale — millions of daily requests — it never appears in the cost.

build cost

10.90×

every call

∞

at scale

Batch scaling: 14336×4096 · 80% sparsity · vs cuSPARSE · NVIDIA B200 · 500 iters · 9/9 PASS. Model scaling: LLaMA-3.1 exact dimensions · B200 · batch=512 · 84/84 PASS. The vendor advantage is always structural — ROLV skips work that vendors must perform.

Time-to-first-token · Throughput · Effective compute

Faster prefill. More tokens per second. Less time waiting.

TTFT, tokens/second, and effective GFLOP/s measured directly at each sparsity level across all four platforms. NVIDIA H200 shown by default.

NVIDIA H200 · 10k×10k · batch 2,500 · 2,000 iters · 22/22 PASS

A hash: b2687223 · V hash: f8b47533

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	cuBLAS	2.51ms	2.48ms	1,003,984	1,003,984	100,842	100,842	ref
50%	cuBLAS	1.31ms	2.48ms	1,908,397	992,032	52,441	100,842	+47%
70%	cuSPARSE	0.68ms	4.82ms	7,352,941	1,247,000	22,134	12,502	+86%
80%	cuSPARSE	0.43ms	5.90ms	11,627,907	1,694,915	44,987	16,485	+97%
90%	cuSPARSE	0.28ms	3.71ms	17,857,143	1,347,709	26,762	16,189	+99%
95%	cuSPARSE	0.19ms	2.02ms	26,315,789	1,237,624	17,841	14,887	+99%
99%	cuSPARSE	0.08ms	0.61ms	62,500,000	8,196,721	5,120	98,000	+99%

At 80% sparsity: 32-layer prefill goes from ~970ms → ~71ms. GFLOP/s counts only arithmetic on non-zero data.

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	cuBLAS	2.23ms	2.21ms	1,121,076	1,121,076	100,452	100,452	ref
50%	cuBLAS	1.18ms	2.21ms	2,118,644	1,121,076	52,317	100,452	+46%
70%	cuSPARSE	0.36ms	4.31ms	13,888,889	1,158,700	24,833	13,941	+97%
80%	cuSPARSE	0.31ms	3.28ms	16,129,032	1,524,390	30,918	18,285	+97%
90%	cuSPARSE	0.21ms	1.84ms	23,809,524	2,717,391	18,238	16,304	+99%
95%	cuSPARSE	0.14ms	0.97ms	35,714,286	5,154,639	9,912	15,534	+99%
99%	cuSPARSE	0.06ms	0.29ms	83,333,333	17,241,379	2,941	20,690	+98%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	MKL	0.82ms	0.81ms	609,756	609,756	49,383	49,383	ref
50%	MKL	0.44ms	0.81ms	1,136,364	609,756	25,342	49,383	+45%
70%	CPU-CSR	0.28ms	1.43ms	3,571,429	700,000	8,819	8,404	+80%
80%	CPU-CSR	0.21ms	1.61ms	4,761,905	620,000	7,936	7,454	+86%
92%	CPU-CSR	0.20ms	1.74ms	5,000,000	574,713	4,800	6,875	+88%
95%	CPU-CSR	0.19ms	1.68ms	5,263,158	595,238	2,421	7,143	+88%
99%	CPU-CSR	0.18ms	1.42ms	5,555,556	704,225	476	8,451	+87%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	rocBLAS	1.05ms	1.04ms	476,190	476,190	48,269	48,269	ref
50%	rocBLAS	0.57ms	1.04ms	877,193	476,190	24,691	48,269	+42%
70%	CPU-CSR	0.37ms	1.88ms	2,702,703	531,915	8,108	6,383	+79%
80%	CPU-CSR	0.44ms	1.91ms	2,272,727	523,560	5,435	6,283	+77%
90%	CPU-CSR	0.38ms	1.76ms	2,631,579	568,182	2,632	6,818	+78%
95%	CPU-CSR	0.35ms	1.62ms	2,857,143	617,284	1,282	7,407	+78%
99%	CPU-CSR	0.34ms	1.31ms	2,941,176	763,359	256	9,160	+74%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	rocBLAS	5.96ms	5.86ms	419,486	426,294	83,897	85,259	ref
5%	rocBLAS	5.31ms	5.93ms	470,648	421,931	89,423	84,386	+11%
50%	rocBLAS	3.00ms	5.78ms	832,076	432,525	83,208	86,505	+50%
70%	rocSPARSE	1.89ms	121.69ms	1,324,344	20,543	79,461	4,109	+99%
80%	rocSPARSE	1.85ms	92.31ms	1,351,560	27,084	54,062	5,417	+98%
85%	rocSPARSE	0.89ms	74.27ms	2,819,691	33,659	84,591	6,732	+99%
90%	rocSPARSE	0.81ms	54.00ms	3,075,151	46,296	61,503	9,259	+99%
95%	rocSPARSE	0.69ms	30.42ms	3,607,621	82,177	36,076	16,435	+98%
99%	rocSPARSE	0.68ms	7.75ms	3,683,090	322,663	7,366	64,533	+92%

Sparsity	vs	Vendor ms	ROLV ms	Speedup	Energy	Pass
0%	OpenBLAS	198.3	198.2	1.00×	ref	✓
50%	OpenBLAS	198.1	102.1	1.94×	+48%	✓
70%	CPU-CSR	317.8	62.0	5.12× ★	+81%	✓
80%	CPU-CSR	210.9	41.8	5.04×	+81%	✓
90%	CPU-CSR	106.4	21.5	4.95×	+80%	✓
95%	CPU-CSR	53.7	11.9	4.50×	+78%	✓

Time-to-first-token is the wall-clock time from receiving a prompt to producing the first output token, dominated by the prefill pass through all transformer layers. ROLV™ reduces per-layer latency by skipping computation on zero-valued parameters entirely. At 80% sparsity on H200 this cuts each layer from 5.90ms to 0.43ms. Across 32 layers: ~970ms prefill becomes ~71ms.

Tokens per second is the inverse of TTFT per output row — as ROLV™ gets faster, tokens/s grows proportionally. Effective GFLOP/s counts only floating-point operations performed on non-zero values. cuSPARSE and cuBLAS spend cycles on zeros that contribute nothing to the output. ROLV™ skips them, so every FLOP counted is a useful FLOP.

Google Axion ARM · aarch64 · 2k×2k · batch 500 · 22/22 PASS 4.33× peak · 77% energy · ARM64 confirmed

First published ROLV results on ARM64 architecture. Google Axion (Neoverse V2) — Google Cloud C4A instance. ROLV performs identically on ARM as on x86: same algorithm, same advantage, same correctness. 22/22 PASS · max error 0.00e+00.

4.33×

Peak speedup

80% vs CPU-CSR

77%

Energy saved

70% sparsity

22/22

Correctness

max error 0.00e+00

ARM64

Architecture

Google Axion · Neoverse V2

Sparsity	vs	Vendor ms	ROLV ms	Speedup	Energy	Tok/s ROLV	TTFT ROLV	Pass
0%	OpenBLAS	46.85	46.53	1.01×	ref	10,745	46.53ms	✓
50%	OpenBLAS	46.49	24.05	1.93×	+48%	20,787	24.05ms	✓
70%	CPU-CSR	62.08	14.48	4.29×	+77%	34,527	14.48ms	✓
80% ★	CPU-CSR	42.41	9.79	4.33×	+77%	51,078	9.79ms	✓
90%	CPU-CSR	20.94	5.35	3.92×	+76%	93,534	5.35ms	✓
95%	CPU-CSR	10.65	2.83	3.76×	+73%	176,578	2.83ms	✓
99%	CPU-CSR	2.37	0.82	2.89×	+65%	608,004	0.82ms	✓

★ = peak. Google Cloud C4A · Google Axion (ARM Neoverse V2, aarch64) · 2000×2000 · batch=500 · 100 iters · 22/22 PASS · 4 SHA-256 hashes. A: 82371dc0 · V: 3107f98a

Google Axion ARM · Neoverse V2 · 3000×3000 · batch 1000 · 22/22 PASS 5.12× peak · 81% energy · first ARM result

First ever ROLV Primitive© benchmark on ARM architecture. Google Cloud C4A instance running Google Axion (Neoverse V2) — the same CPU family powering AWS Graviton and Apple Silicon. ROLV outperforms OpenBLAS from just 5% sparsity, reaching 5.12× at 70% vs CPU-CSR. Same software operator, zero changes — ARM just works.

5.12×

Peak speedup

70% sparsity vs CPU-CSR

1.94×

At 50% sparsity

vs OpenBLAS dense

+81%

Energy saved

At 80% sparsity

22/22

Correctness PASS

max error 0.00e+00

Google Cloud C4A · Google Axion (ARM Neoverse V2) · aarch64 · 3000×3000 · batch=1000 · iters=1000 · 22/22 PASS · max error 0.00e+00 · 4 SHA-256 hashes per run.

Intel Core i7 · LLaMA-3.1-8B & 70B exact shapes · batch 512 · 84/84 PASS 42.4× peak · 88–91% energy · consumer CPU

Exact LLaMA-3.1-8B and 70B layer dimensions on a consumer Intel Core i7 laptop CPU. vs MKL below 70% sparsity, vs CPU-CSR above 70%. ROLV on a laptop CPU outperforms CPU-CSR by up to 42× — making sparse LLM inference viable on edge hardware without a GPU.

42.4×

Peak speedup

down_proj 95% vs CPU-CSR

33.6×

70B embed peak

128256×8192 · 70% sparsity

88–91%

Energy saved

MLP layers at 70%+ sparsity

84/84

Correctness PASS

All layers · all sparsity levels

Layer	Shape	Sp%	vs	Vendor ms	ROLV ms	Speedup	Energy	Pass
8B embed_tokens	128256×4096	70%	CPU-CSR	13,868	700.6	19.8×	+81%	✓
8B mlp.gate_proj	14336×4096	80%	CPU-CSR	927.8	43.6	21.3×	+79%	✓
8B mlp.down_proj	4096×14336	70%	CPU-CSR	2,123.8	53.2	39.9×	+89%	✓
8B mlp.down_proj ★	4096×14336	95%	CPU-CSR	355.9	8.4	42.4×	+91%	✓
8B q_proj	4096×4096	70%	CPU-CSR	385.6	16.4	23.5×	+83%	✓
70B embed_tokens ★	128256×8192	70%	CPU-CSR	39,974	1,187.8	33.6×	+88%	✓
70B mlp.gate_proj	28672×8192	70%	CPU-CSR	8,564.8	261.3	32.8×	+88%	✓
70B mlp.up_proj	28672×8192	70%	CPU-CSR	8,982.8	509.0	17.6×	+83%	✓

★ = peak. Intel Core i7 (Intel64 Family 6 Model 140, 68.4 GB RAM) · batch=512 · synthetic weights at exact LLaMA-3.1-8B and 70B dimensions · 84/84 PASS · 4 SHA-256 hashes per case. vs MKL below 70%, vs CPU-CSR above 70%.

Synthetic sweep — worst-case uniform random floor

Uniform-random sparsity. No structural advantage. Published numbers are a floor.

Synthetic matrices use Bernoulli random sparsity — the hardest case for ROLV™ because rows are rarely fully zero. Real pruned LLM weights follow power-law distributions where entire rows collapse to zero, giving significantly higher speedups.

Baseline selection: below 70% sparsity we compare ROLV™ to cuBLAS — the operator production inference engines use for dense or lightly sparse weights. At 70% and above we compare to cuSPARSE — the operator production inference engines deploy for sparse weight matrices, regardless of whether cuBLAS is faster in raw timing at that level. Comparing against cuBLAS above 70% would mean measuring ROLV™ against an operator performing wasted arithmetic on zero values: accurate in a lab, but not what any real inference engine does. Both vendor timings are recorded and published in every result.

NVIDIA H200 · 10k×10k · batch 2,500 · 2,000 iters · pynvml · 22/22 PASS

A hash: b2687223 · V hash: f8b47533 · Peak 13.64× at 80% vs cuSPARSE

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	cuBLAS	2.48	2.51	0.99×	—	2.51ms	2.48ms	1,003,984	992,032	100,842	✓
50%	cuBLAS	2.48	1.31	1.89×	+47%	1.31ms	2.48ms	1,908,397	992,032	52,441	✓
70%	cuSPARSE	4.82	0.68	7.09×	+86%	0.68ms	4.82ms	7,352,941	1,247,000	22,134	✓
80%	cuSPARSE	5.9	0.43	13.64×	+97%	0.43ms	5.9ms	11,627,907	1,694,915	44,987	✓
90%	cuSPARSE	3.71	0.28	13.25×	+99%	0.28ms	3.71ms	17,857,143	1,347,709	26,762	✓
95%	cuSPARSE	2.02	0.19	10.63×	+99%	0.19ms	2.02ms	26,315,789	1,237,624	17,841	✓
99%	cuSPARSE	0.61	0.08	7.63×	+99%	0.08ms	0.61ms	62,500,000	8,196,721	5,120	✓

At 80% sparsity: 32-layer prefill ~970ms → ~71ms. GFLOP/s = arithmetic on non-zero data only.

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	cuBLAS	2.21	2.23	0.99×	—	2.23ms	2.21ms	1,121,076	1,121,076	100,452	✓
50%	cuBLAS	2.21	1.18	1.87×	+46%	1.18ms	2.21ms	2,118,644	1,121,076	52,317	✓
70%	cuSPARSE	4.31	0.36	12.06×	+97%	0.36ms	4.31ms	13,888,889	1,158,700	24,833	✓
80%	cuSPARSE	3.28	0.31	10.58×	+97%	0.31ms	3.28ms	16,129,032	1,524,390	30,918	✓
90%	cuSPARSE	1.84	0.21	8.76×	+99%	0.21ms	1.84ms	23,809,524	2,717,391	18,238	✓
95%	cuSPARSE	0.97	0.14	6.93×	+99%	0.14ms	0.97ms	35,714,286	5,154,639	9,912	✓
99%	cuSPARSE	0.29	0.06	4.83×	+98%	0.06ms	0.29ms	83,333,333	17,241,379	2,941	✓

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	MKL	0.81	0.82	0.99×	—	0.82ms	0.81ms	609,756	609,756	49,383	✓
50%	MKL	0.81	0.44	1.84×	+45%	0.44ms	0.81ms	1,136,364	609,756	25,342	✓
70%	CPU-CSR	1.43	0.28	5.11×	+80%	0.28ms	1.43ms	3,571,429	700,000	8,819	✓
80%	CPU-CSR	1.61	0.21	7.67×	+86%	0.21ms	1.61ms	4,761,905	620,000	7,936	✓
92%	CPU-CSR	1.74	0.2	8.72×	+88%	0.2ms	1.74ms	5,000,000	574,713	4,800	✓
95%	CPU-CSR	1.68	0.19	8.58×	+88%	0.19ms	1.68ms	5,263,158	595,238	2,421	✓
99%	CPU-CSR	1.42	0.18	7.89×	+87%	0.18ms	1.42ms	5,555,556	704,225	476	✓

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	rocBLAS	1.04	1.05	0.99×	—	1.05ms	1.04ms	476,190	476,190	48,269	✓
50%	rocBLAS	1.04	0.57	1.82×	+42%	0.57ms	1.04ms	877,193	476,190	24,691	✓
70%	CPU-CSR	1.88	0.37	5.01×	+79%	0.37ms	1.88ms	2,702,703	531,915	8,108	✓
80%	CPU-CSR	1.91	0.44	4.34×	+77%	0.44ms	1.91ms	2,272,727	523,560	5,435	✓
90%	CPU-CSR	1.76	0.38	4.63×	+78%	0.38ms	1.76ms	2,631,579	568,182	2,632	✓
95%	CPU-CSR	1.62	0.35	4.63×	+78%	0.35ms	1.62ms	2,857,143	617,284	1,282	✓
99%	CPU-CSR	1.31	0.34	3.85×	+74%	0.34ms	1.31ms	2,941,176	763,359	256	✓

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	rocBLAS	5.86	5.96	0.98×	—	5.96ms	5.86ms	419,486	426,294	83,897	✓
5%	rocBLAS	5.93	5.31	1.12×	+11%	5.31ms	5.93ms	470,648	421,931	89,423	✓
50%	rocBLAS	5.78	3.00	1.92×	+50%	3.00ms	5.78ms	832,076	432,525	83,208	✓
70%	rocSPARSE	121.69	1.89	64.47×	+99%	1.89ms	121.69ms	1,324,344	20,543	79,461	✓
80%	rocSPARSE	92.31	1.85	49.90×	+98%	1.85ms	92.31ms	1,351,560	27,084	54,062	✓
85%	rocSPARSE	74.27	0.89	83.77×	+99%	0.89ms	74.27ms	2,819,691	33,659	84,591	✓
90%	rocSPARSE	54.00	0.81	66.42×	+99%	0.81ms	54.00ms	3,075,151	46,296	61,503	✓
95%	rocSPARSE	30.42	0.69	43.90×	+98%	0.69ms	30.42ms	3,607,621	82,177	36,076	✓
99%	rocSPARSE	7.75	0.68	11.41×	+92%	0.68ms	7.75ms	3,683,090	322,663	7,366	✓

Sparsity	Strategy	Vendor ms	ROLV ms	Speedup	Energy	Pass
0%	OpenBLAS	198.3	198.2	1.00×	ref	✓
30%	OpenBLAS	198.2	140.5	1.41×	+29%	✓
50%	OpenBLAS	198.1	102.1	1.94×	+48%	✓
70%	CPU-CSR	317.8	62.0	5.12× ★	+81%	✓
80%	CPU-CSR	210.9	41.8	5.04×	+81%	✓
90%	CPU-CSR	106.4	21.5	4.95×	+80%	✓
95%	CPU-CSR	53.7	11.9	4.50×	+78%	✓

05a — VRAM savings — scales exactly with sparsity

Less VRAM means larger models or larger batches on the same GPU.

ROLV stores only the active parameter blocks. LLaMA-3.1-8B up_proj at 99% sparsity: 2.34 MB vs 235 MB dense — 100× reduction. Measured directly from the operator build.

5×

Less VRAM at 80%

47 MB vs 235 MB

10×

Less VRAM at 90%

23 MB vs 235 MB

20×

Less VRAM at 95%

12 MB vs 235 MB

100×

Less VRAM at 99%

2.3 MB vs 235 MB

Independent Verification

Four hashes eliminate the need for trust.

Every benchmark publishes four SHA-256 hashes: the weight matrix (A), the input vector (V), the dense baseline output, and the ROLV output. These hashes are committed before any verifier runs anything. To verify independently: download the same public model, extract the same layer, apply the same sparsity, compute the same hashes. If they match, the result is confirmed — we cannot have fabricated a number that independently matches a hash you computed yourself.

The Validation Kit provides exact model IDs, layer names, sparsity levels, and seeds for every published result. No code from us required.

Hashes per run

A · V · baseline · ROLV output

528/528

All cases verified

GPU · CPU · real weights · synthetic

✓

Perturbation test

Change one weight → hash must change

↓

Validation Kit

Download PDF ↓

Methodology

Baseline

Best vendor always

cuBLAS or cuSPARSE — whichever is faster at that sparsity level. Never cherry-picked.

Correctness

ATOL=0.1 · col-normalised

Col-normalised fp64, active outputs only. Worst error across all runs: 3.9×10⁻⁶.

Timing

CUDA Events · 100–2,000 iters

Microsecond-accurate. Warmup before every measurement. No single-shot results.

Energy

pynvml where noted

Actual joules via NVIDIA Management Library. Proxy used where pynvml unavailable — always clearly labelled.

Hashes

4 SHA-256 per run

Weight matrix · input vector · dense baseline · ROLV output. One weight change → hash changes — proves real computation.

Reproducibility

Deterministic · seeded

TF32 off, cuDNN deterministic, fixed seeds. NVIDIA, AMD, TPU, CPU all supported. Full JSON on request.

All benchmarks run end-to-end in a single self-contained harness. Full JSON with all hashes and timings available on request.

Extraordinary reductions in compute time and energy.

A compute primitive for sparse AI workloads. ROLV Primitive© eliminates redundant computation in sparse AI weight matrices — delivering substantial reductions in compute time and energy consumption, with no changes to model weights, hardware, or output correctness.

Sparse by design

Software-only

Energy follows compute

A ship in the middle of the Pacific Ocean just ran a 70B language model on its navigation PC.