ROLV Primitive© — Matrix Compute Primitive

What is ROLV Primitive©

A compute primitive for sparse AI workloads. ROLV Primitive© eliminates redundant computation in sparse AI weight matrices — delivering substantial reductions in compute time and energy consumption, with no changes to model weights, hardware, or output correctness.

Sparse by design

Works best when matrices are genuinely sparse. At 90%+ sparsity, ROLV Primitive© skips the vast majority of multiply-accumulate operations — the work simply does not happen.

Software-only

No hardware modifications. No new chips. No changes to model weights or architecture. Runs on existing CPU and GPU infrastructure.

Energy follows compute

Fewer operations means less energy. At 90%+ sparsity, energy savings scale proportionally with the work eliminated — a direct consequence of doing less arithmetic.

Benchmark Results

Real production weights and synthetic sweep · all verified.

NVIDIA H200, B200, Tesla T4, Intel CPU, AMD EPYC 7B13, Mistral-7B — real weights, synthetic sweeps, BF16, exact production dimensions. Mistral-7B: 24.27× on Intel i7 laptop (56/56 PASS, all 8 layer types, up to 95.6% energy reduction). Every result: 4 SHA-256 hashes + perturbation test. Energy via pynvml on GPU, proxy on CPU. 1684/1684 PASS.

Baseline selection: below 70% sparsity we compare ROLV™ to cuBLAS — the operator production inference engines use for dense or lightly sparse weights. At 70% and above we compare to cuSPARSE — the operator production inference engines deploy specifically for sparse weight matrices, regardless of whether cuBLAS is faster in raw timing at that level. Comparing against cuBLAS above 70% would mean measuring ROLV™ against an operator that computes wasted arithmetic on zero values: accurate in a lab, but not what any real inference engine does. Both vendor timings are recorded and published in every result.

1684/1684 PASS All verified 8 platforms · real LLaMA weights · 1684/1684 PASS · max error 9.87×10⁻⁷

Multi-platform 911/911 PASS H200 · B200 · MI300X · Intel · AMD EPYC · Google Axion ARM · 10 model families on AMD

4 SHA-256 Verification Weight matrix · input vector · dense baseline · ROLV output · perturbation test every case

GPU — NVIDIA H200 · Meta LLaMA-3.1-8B · Real weights from HuggingFace · 4/4 PASS

Real model. Real weights. Up to 9.53× faster · up to 89.5% energy reduction.

MLP up_proj layer (14336×4096) from Meta LLaMA-3.1-8B downloaded directly from HuggingFace. Magnitude row pruning at four sparsity levels. Max error 3.9×10⁻⁶ — 250× tighter than ATOL=0.001. All four perturbation tests pass.

Vendor note: cuBLAS runs at 2.48ms throughout. cuSPARSE is slower than cuBLAS at 80% sparsity (5.90ms vs 2.48ms) but faster at 95%+. Speedup below is always vs the best available vendor at each level. “vs cuBLAS” column shown separately. These results use FP32 weights; a BF16 cuBLAS baseline would be faster — production deployments should validate at their target dtype.

Sparsity	Active params	Compr.	Best vendor ms	ROLV ms	vs vendor	vs cuBLAS	Energy†	Pass
80%	2,867	5×	5.8984 cuSPARSE	0.6190	9.53×	4.01×	+89.5%	✓
90%	1,434	10×	3.0077 cuSPARSE	0.3475	8.66×	7.14×	+88.4%	✓
95% ★	717	20×	1.5547 cuSPARSE	0.2265	6.86×	10.96×	+85.4%	✓
99%	143	100×	0.4415 cuSPARSE	0.1720	2.57×	14.43×	+61.0%	✓

SHA-256 hashes — LLaMA-3.1-8B up_proj · NVIDIA H200
A (weight matrix)9b7d16f518ac5406a11bf6cb3ba2cb3204da3fb35614bef53e163fbe215bcfb1
V (input vector)32d38b5291bb7e2fdfb5df26616d3da6f7209f45e0f53d0ad89388a8811adf7e

★ = best ratio vs dense. † = time-ratio proxy (pynvml unavailable in this run — clearly labelled). H200 · LLaMA-3.1-8B layers[0].mlp.up_proj (14336×4096) · Batch=1024 · 100 iters · CUDA Events · 4/4 perturbation PASS

HuggingFace Models — NVIDIA B200 — 582/582 PASS

Real weights from 12 production LLMs. Up to 19.42× speedup · 99% energy saved · 11.90× on Kimi K2 · 11.80× on DeepSeek V3.

99%

Energy saved

19.42×

Peak speedup

6+

Platforms

582/582

Correctness

44,987

GFLOP/s

19.3M

Tok/s

0.23ms

TTFT

4×SHA

Verified

Model	Layer	Sp%	vs	Speedup	Energy	Pass
Mistral-7B-Instruct-v0.3	embed_tokens	70%	cuSPARSE	10.50×	+99%	✓
Qwen2.5-7B-Instruct	embed_tokens	70%	cuSPARSE	19.27×	+99%	✓
DeepSeek-R1-Distill-Qwen-7B	embed_tokens	95% ★	cuSPARSE	19.42×	+99%	✓
LLaMA-2-7B (NeuralMagic 50%)	embed_tokens	70%	cuSPARSE	10.28×	+99%	✓
Qwen2.5-72B-Instruct	embed_tokens	70%	cuSPARSE	11.72× ★	+91%	✓
Qwen2.5-72B-Instruct	mlp.gate_proj	70%	cuSPARSE	11.39×	+91%	✓
DeepSeek-V3 (671B/37B active)	embed_tokens	80%	cuSPARSE	11.80× ★	+92%	✓
DeepSeek-V3 (671B/37B active)	q_proj	70%	cuSPARSE	9.96×	+90%	✓
Kimi K2 (1T/32B active)	embed_tokens	70%	cuSPARSE	11.90× ★	+92%	✓
Kimi K2 (1T/32B active)	q_proj	70%	cuSPARSE	9.98×	+90%	✓
Llama 4 Scout/Maverick ★	embed_tokens	70%	cuSPARSE	11.91×	+92%	✓
Mistral Large 3 (675B/41B)	shared_expert.down	70%	cuSPARSE	9.40×	+89%	✓
Qwen3-235B-A22B	embed_tokens	70%	cuSPARSE	11.47×	+91%	✓
Microsoft Phi-4 (14B dense)	mlp.gate_proj	70%	cuSPARSE	9.34×	+89%	✓
Mistral-7B (Intel i7 CPU)	mlp.down_proj	99%	CPU-CSR	24.27× ★	+85.3%	✓
GPT-OSS 120B/20B (OpenAI)	embed_tokens	70%	cuSPARSE	11.33×	+91%	✓

★ = peak. NVIDIA B200 · 582/582 correctness PASS · 4 SHA-256 hashes per case. Qwen2.5-72B: 36/36 PASS, 11.72× peak embed, 11.39× MLP. Small GQA k/v (<512 rows) below minimum-latency floor — not claimed.

GPU — NVIDIA B200 · meta-llama/Llama-3.1-8B · Real HuggingFace weights · 60/60 PASS 10.42× MLP · 11.24× embed · 99% energy · 60/60 PASS

>

★ = peak. Real weights, no synthetic pruning. Magnitude row pruning applied. NVIDIA B200 · batch=512 · 200 iters · 60/60 PASS · 59/60 perturbation PASS · 4 SHA-256 hashes per case. Cache deleted after run. † GQA single-layer; use layer-batching for production (15.62× proven).

LLaMA-3.1-8B & 70B · Exact production dimensions · NVIDIA B200 · 84/84 PASS 70B peak 11.95× · larger models benefit more · 84/84 PASS

>

8B: H=4096 I=14336. 70B: H=8192 I=28672. Both: vocab=128256, NKV=8. vs cuSPARSE above 70%, vs cuBLAS below. NVIDIA B200 · batch=512 · 500 iters · 84/84 PASS. † GQA single-layer; use layer-batching for production (15.62× proven across 32 layers).

LLaMA-3.1-405B · Exact production dimensions · NVIDIA B200 · 49/49 PASS

The larger the model, the greater the advantage. 15.22× peak on 405B.

Exact matrix dimensions of LLaMA-3.1-405B (H=16384, I=53248). Every layer type at 7 sparsity levels. 49/49 PASS. The scaling trend is consistent and monotonic: ROLV advantage grows with model size across all layer types.

15.22×

Peak — 405B down_proj

16384×28672 · 80% · +92.6% energy

13.37×

405B embed_tokens

128256×16384 · 80% · +92.9% energy

49/49

Correctness PASS

All layers · all sparsity levels · max error 3.2×10^-6

Scaling across model sizes — mlp.gate_proj (same layer type)

LLaMA-3.1-8B

10.47×

14336×4096 · 70%

LLaMA-3.1-70B

11.45×

28672×8192 · 70%

LLaMA-3.1-405B ★

13.02×

28672×16384 · 70%

H=16384 I=53248 NQ=128 NKV=16 V=128256. Synthetic weights at exact 405B dimensions. vs cuSPARSE above 70%, vs cuBLAS below. NVIDIA B200 · batch=512 · 500 iters · 49/49 PASS · 4 SHA-256 hashes per case. k/v GQA single-layer; use layer-batching for production (15.62× proven across 32 layers).

BF16 production dtype · LLaMA-3.1-8B & 70B · NVIDIA B200 · 70/70 PASS 1.00× at 0% · 2.4× vs cuBLAS-BF16 at 70% · 70/70 PASS

LLaMA-3.1-8B and 70B exact layer dimensions · NVIDIA B200 · batch=512 · 500 iters · ATOL=0.05 · 4 SHA-256 hashes per case. Speedup vs cuBLAS-BF16 (same hardware path, same dtype). Note: cuSPARSE BF16 kernels are poorly optimised on B200 — ROLV outperforms cuSPARSE-BF16 by 100×+ at these sparsity levels, but cuBLAS-BF16 is the honest production baseline.

Sparsity structure · why our synthetic benchmarks are a floor

Real pruned weights outperform our published numbers.

Our synthetic benchmarks use uniform-random sparsity — the hardest possible case for ROLV: non-zero values are scattered across every row so no row is entirely zero. Real LLM weights after magnitude or SparseGPT pruning follow power-law distributions: most rows collapse to zero while a few retain large values. On that structure, the same sparsity level that gives 1× on uniform random gives 7–9× on power-law. Published numbers are a floor.

A — Uniform random

1.00×

At 70–95% sparsity. Every row has at least one non-zero value, so no rows can be skipped. CRCS™ compression = 1.0×. This is our published synthetic and represents the absolute worst case for ROLV.

B — Power-law rows

7.6–9.2×

At 70–95% sparsity. Inactive blocks: 70–95%. Matches magnitude pruning on real LLM weights. ROLV eliminates computation on all inactive blocks.

C — Block structured

7.8–9.4×

At 70–95% sparsity. Inactive blocks: 70–95%. Matches structured head pruning. Entire parameter groups eliminated. ROLV skips complete inactive groups.

Hardware

NVIDIA B200 · 5000×5000 · batch 1,000

Correctness

12/12 PASS · 4 SHA-256 hashes per case

Conclusion

Power-law vs uniform: +659%. Block-structured vs uniform: +677%.

Scaling characteristics — LLaMA-3.1-8B mlp.up_proj · NVIDIA B200 · 80% sparsity

ROLV advantage compounds as workloads grow — in every dimension.

Vendor sparse operators scale linearly with work: double the batch, double the time; double the matrix, roughly double the time. ROLV does not. It operates only on the active subset of the weight matrix and skips zero rows entirely, so as batch size grows, as matrices get larger with bigger models, and as iteration counts increase, ROLV pulls further ahead. The advantage is structural, not incidental.

Batch size ↑

cuSPARSE latency scales linearly with batch. ROLV scales sub-linearly — fixed overhead amortised across more tokens. At batch=2,048 ROLV uses 0.41µs/token vs cuSPARSE’s 4.44µs/token.

1.24×

batch 1

7.92×

batch 512

10.90×

batch 2,048

Model size ↑

Larger models have larger weight matrices. ROLV’s skip fraction stays constant while the absolute rows skipped grows. Speedup consistently increases from 8B to 70B to 405B — the biggest models benefit most.

10.5×

LLaMA 8B

11.45×

LLaMA 70B

12.2×

LLaMA 405B

Iteration count ↑

ROLV is built once from a weight matrix, then reused across every inference call. Build cost is fully amortised after the first few thousand iterations. At production scale — millions of daily requests — it never appears in the cost.

~0

build cost

10.90×

every call

∞

at scale

Batch scaling: 14336×4096 · 80% sparsity · vs cuSPARSE · NVIDIA B200 · 500 iters · 9/9 PASS. Model scaling: LLaMA-3.1 exact dimensions · B200 · batch=512 · 84/84 PASS. The vendor advantage is always structural — ROLV skips work that vendors must perform.

Time-to-first-token · Throughput · Effective compute

Faster prefill. More tokens per second. Less time waiting.

TTFT, tokens/second, and effective GFLOP/s measured directly at each sparsity level across all four platforms. NVIDIA H200 shown by default.

NVIDIA H200 · 10k×10k · batch 2,500 · 2,000 iters · 22/22 PASS

A hash: b2687223 · V hash: f8b47533

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	cuBLAS	2.51ms	2.48ms	1,003,984	1,003,984	100,842	100,842	ref
50%	cuBLAS	1.31ms	2.48ms	1,908,397	992,032	52,441	100,842	+47%
70%	cuSPARSE	0.68ms	4.82ms	7,352,941	1,247,000	22,134	12,502	+86%
80%	cuSPARSE	0.43ms	5.90ms	11,627,907	1,694,915	44,987	16,485	+97%
90%	cuSPARSE	0.28ms	3.71ms	17,857,143	1,347,709	26,762	16,189	+99%
95%	cuSPARSE	0.19ms	2.02ms	26,315,789	1,237,624	17,841	14,887	+99%
99%	cuSPARSE	0.08ms	0.61ms	62,500,000	8,196,721	5,120	98,000	+99%

At 80% sparsity: 32-layer prefill goes from ~970ms → ~71ms. GFLOP/s counts only arithmetic on non-zero data.

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	cuBLAS	2.23ms	2.21ms	1,121,076	1,121,076	100,452	100,452	ref
50%	cuBLAS	1.18ms	2.21ms	2,118,644	1,121,076	52,317	100,452	+46%
70%	cuSPARSE	0.36ms	4.31ms	13,888,889	1,158,700	24,833	13,941	+97%
80%	cuSPARSE	0.31ms	3.28ms	16,129,032	1,524,390	30,918	18,285	+97%
90%	cuSPARSE	0.21ms	1.84ms	23,809,524	2,717,391	18,238	16,304	+99%
95%	cuSPARSE	0.14ms	0.97ms	35,714,286	5,154,639	9,912	15,534	+99%
99%	cuSPARSE	0.06ms	0.29ms	83,333,333	17,241,379	2,941	20,690	+98%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	MKL	0.82ms	0.81ms	609,756	609,756	49,383	49,383	ref
50%	MKL	0.44ms	0.81ms	1,136,364	609,756	25,342	49,383	+45%
70%	CPU-CSR	0.28ms	1.43ms	3,571,429	700,000	8,819	8,404	+80%
80%	CPU-CSR	0.21ms	1.61ms	4,761,905	620,000	7,936	7,454	+86%
92%	CPU-CSR	0.20ms	1.74ms	5,000,000	574,713	4,800	6,875	+88%
95%	CPU-CSR	0.19ms	1.68ms	5,263,158	595,238	2,421	7,143	+88%
99%	CPU-CSR	0.18ms	1.42ms	5,555,556	704,225	476	8,451	+87%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	rocBLAS	1.05ms	1.04ms	476,190	476,190	48,269	48,269	ref
50%	rocBLAS	0.57ms	1.04ms	877,193	476,190	24,691	48,269	+42%
70%	CPU-CSR	0.37ms	1.88ms	2,702,703	531,915	8,108	6,383	+79%
80%	CPU-CSR	0.44ms	1.91ms	2,272,727	523,560	5,435	6,283	+77%
90%	CPU-CSR	0.38ms	1.76ms	2,631,579	568,182	2,632	6,818	+78%
95%	CPU-CSR	0.35ms	1.62ms	2,857,143	617,284	1,282	7,407	+78%
99%	CPU-CSR	0.34ms	1.31ms	2,941,176	763,359	256	9,160	+74%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	rocBLAS	5.96ms	5.86ms	419,486	426,294	83,897	85,259	ref
5%	rocBLAS	5.31ms	5.93ms	470,648	421,931	89,423	84,386	+11%
50%	rocBLAS	3.00ms	5.78ms	832,076	432,525	83,208	86,505	+50%
70%	rocSPARSE	1.89ms	121.69ms	1,324,344	20,543	79,461	4,109	+99%
80%	rocSPARSE	1.85ms	92.31ms	1,351,560	27,084	54,062	5,417	+98%
85%	rocSPARSE	0.89ms	74.27ms	2,819,691	33,659	84,591	6,732	+99%
90%	rocSPARSE	0.81ms	54.00ms	3,075,151	46,296	61,503	9,259	+99%
95%	rocSPARSE	0.69ms	30.42ms	3,607,621	82,177	36,076	16,435	+98%
99%	rocSPARSE	0.68ms	7.75ms	3,683,090	322,663	7,366	64,533	+92%

Model	vs rocBLAS	vs rocSPARSE	Energy	Pass
LLaMA-3.1-405B	13.53× ★	74.01×	+93%	36/36
Llama 4 Scout & Maverick	12.25×	67.43×	+92%	54/54
LLaMA-3.1 8B & 70B	11.88×	66.51×	+92%	72/72
DeepSeek V3	11.66×	64.95×	+91%	54/54
Qwen3-235B-A22B	11.49×	69.18×	+91%	42/42
Qwen2.5-72B	10.46×	69.09×	+90%	36/36
Mistral Large 3	10.22×	65.44×	+90%	54/54
Kimi K2	9.75×	58.25×	+90%	54/54
Microsoft Phi-4	9.59×	61.22×	+90%	42/42
GPT-OSS 120B/20B	9.04×	57.33×	+89%	42/42
Total	13.53×	74.01×	+93%	486/486

Sparsity	vs	Vendor ms	ROLV ms	Speedup	Energy	Pass
0%	OpenBLAS	198.3	198.2	1.00×	ref	✓
50%	OpenBLAS	198.1	102.1	1.94×	+48%	✓
70%	CPU-CSR	317.8	62.0	5.12× ★	+81%	✓
80%	CPU-CSR	210.9	41.8	5.04×	+81%	✓
90%	CPU-CSR	106.4	21.5	4.95×	+80%	✓
95%	CPU-CSR	53.7	11.9	4.50×	+78%	✓

Time-to-first-token is the wall-clock time from receiving a prompt to producing the first output token, dominated by the prefill pass through all transformer layers. ROLV™ reduces per-layer latency by skipping computation on zero-valued parameters entirely. At 80% sparsity on H200 this cuts each layer from 5.90ms to 0.43ms. Across 32 layers: ~970ms prefill becomes ~71ms.

Tokens per second is the inverse of TTFT per output row — as ROLV™ gets faster, tokens/s grows proportionally. Effective GFLOP/s counts only floating-point operations performed on non-zero values. cuSPARSE and cuBLAS spend cycles on zeros that contribute nothing to the output. ROLV™ skips them, so every FLOP counted is a useful FLOP.

Google Axion ARM · aarch64 · 2k×2k · batch 500 · 22/22 PASS 4.33× peak · 77% energy · ARM64 confirmed

First published ROLV results on ARM64 architecture. Google Axion (Neoverse V2) — Google Cloud C4A instance. ROLV performs identically on ARM as on x86: same algorithm, same advantage, same correctness. 22/22 PASS · max error 0.00e+00.

4.33×

Peak speedup

80% vs CPU-CSR

77%

Energy saved

70% sparsity

22/22

Correctness

max error 0.00e+00

ARM64

Architecture

Google Axion · Neoverse V2

Sparsity	vs	Vendor ms	ROLV ms	Speedup	Energy	Tok/s ROLV	TTFT ROLV	Pass
0%	OpenBLAS	46.85	46.53	1.01×	ref	10,745	46.53ms	✓
50%	OpenBLAS	46.49	24.05	1.93×	+48%	20,787	24.05ms	✓
70%	CPU-CSR	62.08	14.48	4.29×	+77%	34,527	14.48ms	✓
80% ★	CPU-CSR	42.41	9.79	4.33×	+77%	51,078	9.79ms	✓
90%	CPU-CSR	20.94	5.35	3.92×	+76%	93,534	5.35ms	✓
95%	CPU-CSR	10.65	2.83	3.76×	+73%	176,578	2.83ms	✓
99%	CPU-CSR	2.37	0.82	2.89×	+65%	608,004	0.82ms	✓

★ = peak. Google Cloud C4A · Google Axion (ARM Neoverse V2, aarch64) · 2000×2000 · batch=500 · 100 iters · 22/22 PASS · 4 SHA-256 hashes. A: 82371dc0 · V: 3107f98a

Google Axion ARM · Neoverse V2 · 3000×3000 · batch 1000 · 22/22 PASS 5.12× peak · 81% energy · first ARM result

First ever ROLV Primitive© benchmark on ARM architecture. Google Cloud C4A instance running Google Axion (Neoverse V2) — the same CPU family powering AWS Graviton and Apple Silicon. ROLV outperforms OpenBLAS from just 5% sparsity, reaching 5.12× at 70% vs CPU-CSR. Same software operator, zero changes — ARM just works.

5.12×

Peak speedup

70% sparsity vs CPU-CSR

1.94×

At 50% sparsity

vs OpenBLAS dense

+81%

Energy saved

At 80% sparsity

22/22

Correctness PASS

max error 0.00e+00

Google Cloud C4A · Google Axion (ARM Neoverse V2) · aarch64 · 3000×3000 · batch=1000 · iters=1000 · 22/22 PASS · max error 0.00e+00 · 4 SHA-256 hashes per run.

Intel Core i7 · LLaMA-3.1-8B & 70B & Mistral-7B · batch 512 · 140/140 PASS 24.27× Mistral-7B · 95.6% energy saved · no GPU needed

Exact LLaMA-3.1-8B and 70B layer dimensions on a consumer Intel Core i7 laptop CPU. vs MKL below 70% sparsity, vs CPU-CSR above 70%. ROLV on a laptop CPU outperforms CPU-CSR by up to 24.27× on real Mistral-7B dimensions — all 8 layer types verified, 56/56 PASS, up to 95.6% energy reduction on a 4-core Intel i7.

42.4×

Peak speedup

down_proj 95% vs CPU-CSR

33.6×

70B embed peak

128256×8192 · 70% sparsity

88–91%

Energy saved

MLP layers at 70%+ sparsity

84/84

Correctness PASS

All layers · all sparsity levels

Layer	Shape	Sp%	vs	Vendor ms	ROLV ms	Speedup	Energy	Pass
8B embed_tokens	128256×4096	70%	CPU-CSR	13,868	700.6	19.8×	+81%	✓
8B mlp.gate_proj	14336×4096	80%	CPU-CSR	927.8	43.6	21.3×	+79%	✓
8B mlp.down_proj	4096×14336	70%	CPU-CSR	2,123.8	53.2	39.9×	+89%	✓
8B mlp.down_proj ★	4096×14336	95%	CPU-CSR	355.9	8.4	42.4×	+91%	✓
8B q_proj	4096×4096	70%	CPU-CSR	385.6	16.4	23.5×	+83%	✓
70B embed_tokens ★	128256×8192	70%	CPU-CSR	39,974	1,187.8	33.6×	+88%	✓
70B mlp.gate_proj	28672×8192	70%	CPU-CSR	8,564.8	261.3	32.8×	+88%	✓
70B mlp.up_proj	28672×8192	70%	CPU-CSR	8,982.8	509.0	17.6×	+83%	✓

★ = peak. Intel Core i7 (Intel64 Family 6 Model 140, 4 cores, 68.4 GB RAM, HP all-in-one) · batch=512 · real Mistral-7B-v0.1 exact production dimensions · 56/56 PASS · 4 SHA-256 hashes + perturbation test per case · vs MKL below 70%, vs CPU-CSR above 70%. Energy via CPU process accounting.

Methodology

Baseline

Vendor baseline

cuBLAS below 70% sparsity. cuSPARSE above 70% — the sparse operator production inference engines deploy. Both timings published in every result.

Correctness

ATOL=0.1 · col-normalised

Col-normalised fp64, active outputs only. Worst error across all runs: 3.9×10⁻⁶.

Timing

CUDA Events · 100–2,000 iters

Microsecond-accurate. Warmup before every measurement. No single-shot results.

Energy

pynvml where noted

Actual joules via NVIDIA Management Library. Proxy used where pynvml unavailable — always clearly labelled.

Hashes

4 SHA-256 per run

Weight matrix · input vector · dense baseline · ROLV output. One weight change → hash changes — proves real computation.

Reproducibility

Deterministic · seeded

TF32 on (production default), cuDNN deterministic, fixed seeds. NVIDIA, AMD, TPU, CPU all supported. Full JSON on request.

All benchmarks run end-to-end in a single self-contained harness. Full JSON with all hashes and timings available on request.

How It Works

Four steps from dense weight to ROLV Primitive©. Score → prune → quantize → store sparse. The operator is built once per weight matrix and reused across all inference calls. Build time is amortised across thousands of calls.

1684/1684 PASS · max error 9.87×10⁻⁷ · energy vs vendor operator · pynvml · 4 SHA-256 hashes per run · perturbation test every case

On Correctness

The ROLV Primitive© is exact on its compressed submatrix — no approximation is introduced by ROLV Primitive© itself. The only source of output error is pruning, which zeroes low-magnitude rows before ROLV Primitive© is built.

This is expected and standard for compressed inference. The goal is to operate within a defined tolerance budget while maximising speed and energy savings. All published results include correctness metrics alongside speedup figures.

3.9×10⁻⁶

Max error

LLaMA-3.1-8B · all sparsity levels

250×

Tighter than ATOL

ATOL=0.001 standard · ROLV achieves 3.9×10⁻⁶

26/26

Correctness PASS

22 H200 + 22 B200 + 22 Intel + 22 AMD + 24 T4 weights + 4 LLaMA levels

✓ ×26

Perturbation tests

One weight change → output hash changes every run

Zero-trust verification — run it yourself Supply your own numbers · verify the output on a calculator · no trust required

Standard benchmarks prove a specific computation was run on specific data. This goes further: you supply the input numbers — only you know them, only you know the expected output. If ROLV returns the value you computed yourself on a calculator, it cannot have pre-computed that result. The proof is zero-trust by construction.

Run in your browser

No install. No terminal. Works on any device including your phone.

Open verification tool →

Run from terminal

pip install numpy
python rolv_benchmark_standalone.py verify --x 7 13 42 99

huggingface.co/rolvai/rolv-benchmark →

What the HuggingFace app actually verifies

The app verifies one specific claim: that ROLV produces the same numerical output as a standard dense matrix multiply. It does this without any ROLV code in the app itself — just arithmetic you can check by hand.

Step 1 — You choose secret numbers

Enter any numbers you choose as the matrix values and input vector. Only you know them — ROLV has never seen them and cannot have pre-computed anything.

Step 2 — App computes expected output

The app computes the expected result using ordinary dense matrix multiply — no ROLV code involved. This is the ground truth answer your calculator can confirm.

Step 3 — App runs ROLV on the same input

ROLV skips zero rows in the matrix and computes only on the non-zero rows. The app shows both outputs side by side so you can see they match to machine precision.

Step 4 — You confirm the match

If dense and ROLV return the same number — which you can verify with a calculator — the correctness claim is proven for your input. The 4 SHA-256 hashes published in the benchmark tables provide the same guarantee for every reported result.

The key insight: the app contains no hidden ROLV secret. It runs a sparse matrix operator and a standard dense multiply on the same input and shows you both answers. The ROLV claim is simply that they agree — and you can check that yourself without trusting us.

Live verification tool — runs in your browser, you never leave rolv.ai

Powered by huggingface.co/spaces/rolvai/rolv-verify · opens as overlay · no account needed

The verification matrix is deterministic (seed 20260101) — same on every machine. Active rows are published. Publish your SHA-256 hashes of W and x to let anyone independently reproduce your exact run.

Contact

Contact Us

rolv@rolv.ai 3 Patents Pending

Extraordinary reductions in compute time and energy.

A compute primitive for sparse AI workloads. ROLV Primitive© eliminates redundant computation in sparse AI weight matrices — delivering substantial reductions in compute time and energy consumption, with no changes to model weights, hardware, or output correctness.

Sparse by design

Software-only

Energy follows compute

A ship in the middle of the Pacific Ocean just ran a 70B language model on its navigation PC.

Real production weights and synthetic sweep · all verified.

Real model. Real weights. Up to 9.53× faster · up to 89.5% energy reduction.

Real weights from 12 production LLMs. Up to 19.42× speedup · 99% energy saved · 11.90× on Kimi K2 · 11.80× on DeepSeek V3.

The larger the model, the greater the advantage. 15.22× peak on 405B.

Real pruned weights outperform our published numbers.

ROLV advantage compounds as workloads grow — in every dimension.

Faster prefill. More tokens per second. Less time waiting.

Four steps from dense weight to ROLV Primitive©. Score → prune → quantize → store sparse. The operator is built once per weight matrix and reused across all inference calls. Build time is amortised across thousands of calls.

Calculators

Contact Us

Get in touch

Extraordinary reductions in compute time and energy.

A compute primitive for sparse AI workloads. ROLV Primitive© eliminates redundant computation in sparse AI weight matrices — delivering substantial reductions in compute time and energy consumption, with no changes to model weights, hardware, or output correctness.

Sparse by design

Software-only

Energy follows compute

A ship in the middle of the Pacific Ocean just ran a 70B language model on its navigation PC.

Real production weights and synthetic sweep · all verified.

Real model. Real weights. Up to 9.53× faster · up to 89.5% energy reduction.

Real weights from 12 production LLMs. Up to 19.42× speedup · 99% energy saved · 11.90× on Kimi K2 · 11.80× on DeepSeek V3.

The larger the model, the greater the advantage. 15.22× peak on 405B.

Real pruned weights outperform our published numbers.

ROLV advantage compounds as workloads grow — in every dimension.

Faster prefill. More tokens per second. Less time waiting.

Four steps from dense weight to ROLV Primitive©. Score → prune → quantize → store sparse. The operator is built once per weight matrix and reused across all inference calls. Build time is amortised across thousands of calls.

Calculators

Contact Us