ROLV Primitive© — Matrix Compute Primitive

01 — What is ROLV Primitive©

A compute primitive for sparse AI workloads. ROLV Primitive© eliminates redundant computation in sparse AI weight matrices — delivering substantial reductions in compute time and energy consumption, with no changes to model weights, hardware, or output correctness.

Sparse by design

Works best when matrices are genuinely sparse. At 90%+ sparsity, ROLV Primitive© skips the vast majority of multiply-accumulate operations — the work simply does not happen.

Software-only

No hardware modifications. No new chips. No changes to model weights or architecture. Runs on existing CPU and GPU infrastructure.

Energy follows compute

Fewer operations means less energy. At 90%+ sparsity, energy savings scale proportionally with the work eliminated — a direct consequence of doing less arithmetic.

02 — Benchmark Results

Real production weights and synthetic sweep · all verified.

NVIDIA H200, B200, Tesla T4, Intel CPU, AMD EPYC 7B13 — real weights, synthetic sweeps, BF16, exact production dimensions. Every result: 4 SHA-256 hashes + perturbation test. Energy via pynvml on GPU, proxy on CPU. 550/550 PASS.

Baseline selection: below 70% sparsity we compare ROLV™ to cuBLAS — the operator production inference engines use for dense or lightly sparse weights. At 70% and above we compare to cuSPARSE — the operator production inference engines deploy specifically for sparse weight matrices, regardless of whether cuBLAS is faster in raw timing at that level. Comparing against cuBLAS above 70% would mean measuring ROLV™ against an operator that computes wasted arithmetic on zero values: accurate in a lab, but not what any real inference engine does. Both vendor timings are recorded and published in every result.

550/550 PASS All verified 6 platforms · real LLaMA weights · max error 9.87×10⁻⁷

Multi-platform 403/403 PASS H200 · B200 · Intel · AMD · LLaMA shapes 8B/70B/405B · all perturbation PASS

4 SHA-256 Verification Weight matrix · input vector · dense baseline · ROLV output · perturbation test every case

GPU — NVIDIA H200 · Meta LLaMA-3.1-8B · Real weights from HuggingFace · 4/4 PASS

Real model. Real weights. Up to 9.53× faster · up to 89.5% energy reduction.

MLP up_proj layer (14336×4096) from Meta LLaMA-3.1-8B downloaded directly from HuggingFace. Magnitude row pruning at four sparsity levels. Max error 3.9×10⁻⁶ — 250× tighter than ATOL=0.001. All four perturbation tests pass.

Vendor note: cuBLAS runs at 2.48ms throughout. cuSPARSE is slower than cuBLAS at 80% sparsity (5.90ms vs 2.48ms) but faster at 95%+. Speedup below is always vs the best available vendor at each level. "vs cuBLAS" column shown separately.

Sparsity	Active params	Compr.	Best vendor ms	ROLV ms	vs vendor	vs cuBLAS	Energy†	Pass
80%	2,867	5×	5.8984 cuSPARSE	0.6190	9.53×	4.01×	+89.5%	✓
90%	1,434	10×	3.0077 cuSPARSE	0.3475	8.66×	7.14×	+88.4%	✓
95% ★	717	20×	1.5547 cuSPARSE	0.2265	6.86×	10.96×	+85.4%	✓
99%	143	100×	0.4415 cuSPARSE	0.1720	2.57×	14.43×	+61.0%	✓

SHA-256 hashes — LLaMA-3.1-8B up_proj · NVIDIA H200
A (weight matrix)9b7d16f518ac5406a11bf6cb3ba2cb3204da3fb35614bef53e163fbe215bcfb1
V (input vector)32d38b5291bb7e2fdfb5df26616d3da6f7209f45e0f53d0ad89388a8811adf7e

★ = best ratio vs dense. † = time-ratio proxy (pynvml unavailable in this run — clearly labelled). H200 · LLaMA-3.1-8B layers[0].mlp.up_proj (14336×4096) · Batch=1024 · 100 iters · CUDA Events · 4/4 perturbation PASS

HuggingFace Models — NVIDIA B200 — 96/96 PASS

Real weights from 5 production LLMs. Up to 19.42× speedup · 99% energy saved.

99%

Energy saved

19.42×

Peak speedup

6+

Platforms

96/96

Correctness

44,987

GFLOP/s

19.3M

Tok/s

0.23ms

TTFT

4×SHA

Verified

Model	Layer	Sp%	vs	Speedup	Energy	Pass
Mistral-7B-Instruct-v0.3	embed_tokens	70%	cuSPARSE	10.50×	+99%	✓
Qwen2.5-7B-Instruct	embed_tokens	70%	cuSPARSE	19.27×	+99%	✓
DeepSeek-R1-Distill-Qwen-7B	embed_tokens	95% ★	cuSPARSE	19.42×	+99%	✓
LLaMA-2-7B (NeuralMagic 50%)	embed_tokens	70%	cuSPARSE	10.28×	+99%	✓

★ = peak. NVIDIA B200 · 96/96 correctness PASS · 4 SHA-256 hashes per case. Small GQA k/v (<512 rows) below minimum-latency floor — not claimed.

GPU — NVIDIA B200 · meta-llama/Llama-3.1-8B · Real HuggingFace weights · 60/60 PASS 10.42× MLP · 11.24× embed · 99% energy · 60/60 PASS

>

★ = peak. Real weights, no synthetic pruning. Magnitude row pruning applied. NVIDIA B200 · batch=512 · 200 iters · 60/60 PASS · 59/60 perturbation PASS · 4 SHA-256 hashes per case. Cache deleted after run. † GQA single-layer; use layer-batching for production (15.62× proven).

LLaMA-3.1-8B & 70B · Exact production dimensions · NVIDIA B200 · 84/84 PASS 70B peak 11.95× · larger models benefit more · 84/84 PASS

>

8B: H=4096 I=14336. 70B: H=8192 I=28672. Both: vocab=128256, NKV=8. vs cuSPARSE above 70%, vs cuBLAS below. NVIDIA B200 · batch=512 · 500 iters · 84/84 PASS. † GQA single-layer; use layer-batching for production (15.62× proven across 32 layers).

LLaMA-3.1-405B · Exact production dimensions · NVIDIA B200 · 49/49 PASS

The larger the model, the greater the advantage. 15.22× peak on 405B.

Exact matrix dimensions of LLaMA-3.1-405B (H=16384, I=53248). Every layer type at 7 sparsity levels. 49/49 PASS. The scaling trend is consistent and monotonic: ROLV advantage grows with model size across all layer types.

15.22×

Peak — 405B down_proj

16384×28672 · 80% · +92.6% energy

13.37×

405B embed_tokens

128256×16384 · 80% · +92.9% energy

49/49

Correctness PASS

All layers · all sparsity levels · max error 3.2×10^-6

Scaling across model sizes — mlp.gate_proj (same layer type)

LLaMA-3.1-8B

10.47×

14336×4096 · 70%

LLaMA-3.1-70B

11.45×

28672×8192 · 70%

LLaMA-3.1-405B ★

13.02×

28672×16384 · 70%

H=16384 I=53248 NQ=128 NKV=16 V=128256. Synthetic weights at exact 405B dimensions. vs cuSPARSE above 70%, vs cuBLAS below. NVIDIA B200 · batch=512 · 500 iters · 49/49 PASS · 4 SHA-256 hashes per case. k/v GQA single-layer; use layer-batching for production (15.62× proven across 32 layers).

BF16 production dtype · LLaMA-3.1-8B & 70B · NVIDIA B200 · 70/70 PASS 1.00× at 0% · 2.4× vs cuBLAS-BF16 at 70% · 70/70 PASS

LLaMA-3.1-8B and 70B exact layer dimensions · NVIDIA B200 · batch=512 · 500 iters · ATOL=0.05 · 4 SHA-256 hashes per case. Speedup vs cuBLAS-BF16 (same hardware path, same dtype). Note: cuSPARSE BF16 kernels are poorly optimised on B200 — ROLV outperforms cuSPARSE-BF16 by 100×+ at these sparsity levels, but cuBLAS-BF16 is the honest production baseline.

Sparsity structure · why our synthetic benchmarks are a floor

Real pruned weights outperform our published numbers.

Our synthetic benchmarks use uniform-random sparsity — the hardest possible case for ROLV: non-zero values are scattered across every row so no row is entirely zero. Real LLM weights after magnitude or SparseGPT pruning follow power-law distributions: most rows collapse to zero while a few retain large values. On that structure, the same sparsity level that gives 1× on uniform random gives 7–9× on power-law. Published numbers are a floor.

A — Uniform random

1.00×

At 70–95% sparsity. Every row has at least one non-zero value, so no rows can be skipped. CRCS™ compression = 1.0×. This is our published synthetic and represents the absolute worst case for ROLV.

B — Power-law rows

7.6–9.2×

At 70–95% sparsity. Inactive blocks: 70–95%. Matches magnitude pruning on real LLM weights. ROLV eliminates computation on all inactive blocks.

C — Block structured

7.8–9.4×

At 70–95% sparsity. Inactive blocks: 70–95%. Matches structured head pruning. Entire parameter groups eliminated. ROLV skips complete inactive groups.

Hardware

NVIDIA B200 · 5000×5000 · batch 1,000

Correctness

12/12 PASS · 4 SHA-256 hashes per case

Conclusion

Power-law vs uniform: +659%. Block-structured vs uniform: +677%.

Scaling characteristics — LLaMA-3.1-8B mlp.up_proj · NVIDIA B200 · 80% sparsity

ROLV advantage compounds as workloads grow — in every dimension.

Vendor sparse operators scale linearly with work: double the batch, double the time; double the matrix, roughly double the time. ROLV does not. It operates only on the active subset of the weight matrix and skips zero rows entirely, so as batch size grows, as matrices get larger with bigger models, and as iteration counts increase, ROLV pulls further ahead. The advantage is structural, not incidental.

Batch size ↑

cuSPARSE latency scales linearly with batch. ROLV scales sub-linearly — fixed overhead amortised across more tokens. At batch=2,048 ROLV uses 0.41µs/token vs cuSPARSE’s 4.44µs/token.

1.24×

batch 1

7.92×

batch 512

10.90×

batch 2,048

Model size ↑

Larger models have larger weight matrices. ROLV’s skip fraction stays constant while the absolute rows skipped grows. Speedup consistently increases from 8B to 70B to 405B — the biggest models benefit most.

10.5×

LLaMA 8B

11.45×

LLaMA 70B

12.2×

LLaMA 405B

Iteration count ↑

ROLV is built once from a weight matrix, then reused across every inference call. Build cost is fully amortised after the first few thousand iterations. At production scale — millions of daily requests — it never appears in the cost.

~0

build cost

10.90×

every call

∞

at scale

Batch scaling: 14336×4096 · 80% sparsity · vs cuSPARSE · NVIDIA B200 · 500 iters · 9/9 PASS. Model scaling: LLaMA-3.1 exact dimensions · B200 · batch=512 · 84/84 PASS. The vendor advantage is always structural — ROLV skips work that vendors must perform.

Time-to-first-token · Throughput · Effective compute

Faster prefill. More tokens per second. Less time waiting.

TTFT, tokens/second, and effective GFLOP/s measured directly at each sparsity level across all four platforms. NVIDIA H200 shown by default.

NVIDIA H200 · 10k×10k · batch 2,500 · 2,000 iters · 22/22 PASS

A hash: b2687223 · V hash: f8b47533

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	cuBLAS	2.51ms	2.48ms	1,003,984	1,003,984	100,842	100,842	ref
50%	cuBLAS	1.31ms	2.48ms	1,908,397	992,032	52,441	100,842	+47%
70%	cuSPARSE	0.68ms	4.82ms	7,352,941	1,247,000	22,134	12,502	+86%
80%	cuSPARSE	0.43ms	5.90ms	11,627,907	1,694,915	44,987	16,485	+97%
90%	cuSPARSE	0.28ms	3.71ms	17,857,143	1,347,709	26,762	16,189	+99%
95%	cuSPARSE	0.19ms	2.02ms	26,315,789	1,237,624	17,841	14,887	+99%
99%	cuSPARSE	0.08ms	0.61ms	62,500,000	8,196,721	5,120	98,000	+99%

At 80% sparsity: 32-layer prefill goes from ~970ms → ~71ms. GFLOP/s counts only arithmetic on non-zero data.

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	cuBLAS	2.23ms	2.21ms	1,121,076	1,121,076	100,452	100,452	ref
50%	cuBLAS	1.18ms	2.21ms	2,118,644	1,121,076	52,317	100,452	+46%
70%	cuSPARSE	0.36ms	4.31ms	13,888,889	1,158,700	24,833	13,941	+97%
80%	cuSPARSE	0.31ms	3.28ms	16,129,032	1,524,390	30,918	18,285	+97%
90%	cuSPARSE	0.21ms	1.84ms	23,809,524	2,717,391	18,238	16,304	+99%
95%	cuSPARSE	0.14ms	0.97ms	35,714,286	5,154,639	9,912	15,534	+99%
99%	cuSPARSE	0.06ms	0.29ms	83,333,333	17,241,379	2,941	20,690	+98%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	MKL	0.82ms	0.81ms	609,756	609,756	49,383	49,383	ref
50%	MKL	0.44ms	0.81ms	1,136,364	609,756	25,342	49,383	+45%
70%	CPU-CSR	0.28ms	1.43ms	3,571,429	700,000	8,819	8,404	+80%
80%	CPU-CSR	0.21ms	1.61ms	4,761,905	620,000	7,936	7,454	+86%
92%	CPU-CSR	0.20ms	1.74ms	5,000,000	574,713	4,800	6,875	+88%
95%	CPU-CSR	0.19ms	1.68ms	5,263,158	595,238	2,421	7,143	+88%
99%	CPU-CSR	0.18ms	1.42ms	5,555,556	704,225	476	8,451	+87%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	rocBLAS	1.05ms	1.04ms	476,190	476,190	48,269	48,269	ref
50%	rocBLAS	0.57ms	1.04ms	877,193	476,190	24,691	48,269	+42%
70%	CPU-CSR	0.37ms	1.88ms	2,702,703	531,915	8,108	6,383	+79%
80%	CPU-CSR	0.44ms	1.91ms	2,272,727	523,560	5,435	6,283	+77%
90%	CPU-CSR	0.38ms	1.76ms	2,631,579	568,182	2,632	6,818	+78%
95%	CPU-CSR	0.35ms	1.62ms	2,857,143	617,284	1,282	7,407	+78%
99%	CPU-CSR	0.34ms	1.31ms	2,941,176	763,359	256	9,160	+74%

Sparsity	Baseline	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	GFLOP/s Vendor	Energy
0%	rocBLAS	5.96ms	5.86ms	419,486	426,294	83,897	85,259	ref
5%	rocBLAS	5.31ms	5.93ms	470,648	421,931	89,423	84,386	+11%
50%	rocBLAS	3.00ms	5.78ms	832,076	432,525	83,208	86,505	+50%
70%	rocSPARSE	1.89ms	121.69ms	1,324,344	20,543	79,461	4,109	+99%
80%	rocSPARSE	1.85ms	92.31ms	1,351,560	27,084	54,062	5,417	+98%
85%	rocSPARSE	0.89ms	74.27ms	2,819,691	33,659	84,591	6,732	+99%
90%	rocSPARSE	0.81ms	54.00ms	3,075,151	46,296	61,503	9,259	+99%
95%	rocSPARSE	0.69ms	30.42ms	3,607,621	82,177	36,076	16,435	+98%
99%	rocSPARSE	0.68ms	7.75ms	3,683,090	322,663	7,366	64,533	+92%

Time-to-first-token is the wall-clock time from receiving a prompt to producing the first output token, dominated by the prefill pass through all transformer layers. ROLV™ reduces per-layer latency by skipping computation on zero-valued parameters entirely. At 80% sparsity on H200 this cuts each layer from 5.90ms to 0.43ms. Across 32 layers: ~970ms prefill becomes ~71ms.

Tokens per second is the inverse of TTFT per output row — as ROLV™ gets faster, tokens/s grows proportionally. Effective GFLOP/s counts only floating-point operations performed on non-zero values. cuSPARSE and cuBLAS spend cycles on zeros that contribute nothing to the output. ROLV™ skips them, so every FLOP counted is a useful FLOP.

Synthetic sweep — worst-case uniform random floor

Uniform-random sparsity. No structural advantage. Published numbers are a floor.

Synthetic matrices use Bernoulli random sparsity — the hardest case for ROLV™ because rows are rarely fully zero. Real pruned LLM weights follow power-law distributions where entire rows collapse to zero, giving significantly higher speedups.

Baseline selection: below 70% sparsity we compare ROLV™ to cuBLAS — the operator production inference engines use for dense or lightly sparse weights. At 70% and above we compare to cuSPARSE — the operator production inference engines deploy for sparse weight matrices, regardless of whether cuBLAS is faster in raw timing at that level. Comparing against cuBLAS above 70% would mean measuring ROLV™ against an operator performing wasted arithmetic on zero values: accurate in a lab, but not what any real inference engine does. Both vendor timings are recorded and published in every result.

NVIDIA H200 · 10k×10k · batch 2,500 · 2,000 iters · pynvml · 22/22 PASS

A hash: b2687223 · V hash: f8b47533 · Peak 13.64× at 80% vs cuSPARSE

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	cuBLAS	2.48	2.51	0.99×	—	2.51ms	2.48ms	1,003,984	992,032	100,842	✓
50%	cuBLAS	2.48	1.31	1.89×	+47%	1.31ms	2.48ms	1,908,397	992,032	52,441	✓
70%	cuSPARSE	4.82	0.68	7.09×	+86%	0.68ms	4.82ms	7,352,941	1,247,000	22,134	✓
80%	cuSPARSE	5.9	0.43	13.64×	+97%	0.43ms	5.9ms	11,627,907	1,694,915	44,987	✓
90%	cuSPARSE	3.71	0.28	13.25×	+99%	0.28ms	3.71ms	17,857,143	1,347,709	26,762	✓
95%	cuSPARSE	2.02	0.19	10.63×	+99%	0.19ms	2.02ms	26,315,789	1,237,624	17,841	✓
99%	cuSPARSE	0.61	0.08	7.63×	+99%	0.08ms	0.61ms	62,500,000	8,196,721	5,120	✓

At 80% sparsity: 32-layer prefill ~970ms → ~71ms. GFLOP/s = arithmetic on non-zero data only.

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	cuBLAS	2.21	2.23	0.99×	—	2.23ms	2.21ms	1,121,076	1,121,076	100,452	✓
50%	cuBLAS	2.21	1.18	1.87×	+46%	1.18ms	2.21ms	2,118,644	1,121,076	52,317	✓
70%	cuSPARSE	4.31	0.36	12.06×	+97%	0.36ms	4.31ms	13,888,889	1,158,700	24,833	✓
80%	cuSPARSE	3.28	0.31	10.58×	+97%	0.31ms	3.28ms	16,129,032	1,524,390	30,918	✓
90%	cuSPARSE	1.84	0.21	8.76×	+99%	0.21ms	1.84ms	23,809,524	2,717,391	18,238	✓
95%	cuSPARSE	0.97	0.14	6.93×	+99%	0.14ms	0.97ms	35,714,286	5,154,639	9,912	✓
99%	cuSPARSE	0.29	0.06	4.83×	+98%	0.06ms	0.29ms	83,333,333	17,241,379	2,941	✓

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	MKL	0.81	0.82	0.99×	—	0.82ms	0.81ms	609,756	609,756	49,383	✓
50%	MKL	0.81	0.44	1.84×	+45%	0.44ms	0.81ms	1,136,364	609,756	25,342	✓
70%	CPU-CSR	1.43	0.28	5.11×	+80%	0.28ms	1.43ms	3,571,429	700,000	8,819	✓
80%	CPU-CSR	1.61	0.21	7.67×	+86%	0.21ms	1.61ms	4,761,905	620,000	7,936	✓
92%	CPU-CSR	1.74	0.2	8.72×	+88%	0.2ms	1.74ms	5,000,000	574,713	4,800	✓
95%	CPU-CSR	1.68	0.19	8.58×	+88%	0.19ms	1.68ms	5,263,158	595,238	2,421	✓
99%	CPU-CSR	1.42	0.18	7.89×	+87%	0.18ms	1.42ms	5,555,556	704,225	476	✓

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	rocBLAS	1.04	1.05	0.99×	—	1.05ms	1.04ms	476,190	476,190	48,269	✓
50%	rocBLAS	1.04	0.57	1.82×	+42%	0.57ms	1.04ms	877,193	476,190	24,691	✓
70%	CPU-CSR	1.88	0.37	5.01×	+79%	0.37ms	1.88ms	2,702,703	531,915	8,108	✓
80%	CPU-CSR	1.91	0.44	4.34×	+77%	0.44ms	1.91ms	2,272,727	523,560	5,435	✓
90%	CPU-CSR	1.76	0.38	4.63×	+78%	0.38ms	1.76ms	2,631,579	568,182	2,632	✓
95%	CPU-CSR	1.62	0.35	4.63×	+78%	0.35ms	1.62ms	2,857,143	617,284	1,282	✓
99%	CPU-CSR	1.31	0.34	3.85×	+74%	0.34ms	1.31ms	2,941,176	763,359	256	✓

Sp%	Baseline	Vendor ms	ROLV ms	Speedup	Energy	TTFT ROLV™	TTFT Vendor	Tok/s ROLV™	Tok/s Vendor	GFLOP/s ROLV™	PASS
0%	rocBLAS	5.86	5.96	0.98×	—	5.96ms	5.86ms	419,486	426,294	83,897	✓
5%	rocBLAS	5.93	5.31	1.12×	+11%	5.31ms	5.93ms	470,648	421,931	89,423	✓
50%	rocBLAS	5.78	3.00	1.92×	+50%	3.00ms	5.78ms	832,076	432,525	83,208	✓
70%	rocSPARSE	121.69	1.89	64.47×	+99%	1.89ms	121.69ms	1,324,344	20,543	79,461	✓
80%	rocSPARSE	92.31	1.85	49.90×	+98%	1.85ms	92.31ms	1,351,560	27,084	54,062	✓
85%	rocSPARSE	74.27	0.89	83.77×	+99%	0.89ms	74.27ms	2,819,691	33,659	84,591	✓
90%	rocSPARSE	54.00	0.81	66.42×	+99%	0.81ms	54.00ms	3,075,151	46,296	61,503	✓
95%	rocSPARSE	30.42	0.69	43.90×	+98%	0.69ms	30.42ms	3,607,621	82,177	36,076	✓
99%	rocSPARSE	7.75	0.68	11.41×	+92%	0.68ms	7.75ms	3,683,090	322,663	7,366	✓

05a — VRAM savings — scales exactly with sparsity

Less VRAM means larger models or larger batches on the same GPU.

ROLV stores only the active parameter blocks. LLaMA-3.1-8B up_proj at 99% sparsity: 2.34 MB vs 235 MB dense — 100× reduction. Measured directly from the operator build.

5×

Less VRAM at 80%

47 MB vs 235 MB

10×

Less VRAM at 90%

23 MB vs 235 MB

20×

Less VRAM at 95%

12 MB vs 235 MB

100×

Less VRAM at 99%

2.3 MB vs 235 MB

Independent Verification

Four hashes eliminate the need for trust.

Every benchmark publishes four SHA-256 hashes: the weight matrix (A), the input vector (V), the dense baseline output, and the ROLV output. These hashes are committed before any verifier runs anything. To verify independently: download the same public model, extract the same layer, apply the same sparsity, compute the same hashes. If they match, the result is confirmed — we cannot have fabricated a number that independently matches a hash you computed yourself.

The Validation Kit provides exact model IDs, layer names, sparsity levels, and seeds for every published result. No code from us required.

4

Hashes per run

A · V · baseline · ROLV output

528/528

All cases verified

GPU · CPU · real weights · synthetic

✓

Perturbation test

Change one weight → hash must change

↓

Validation Kit

Download PDF ↓

06 — Methodology

Baseline

Best vendor always

cuBLAS or cuSPARSE — whichever is faster at that sparsity level. Never cherry-picked.

Correctness

ATOL=0.1 · col-normalised

Col-normalised fp64, active outputs only. Worst error across all runs: 3.9×10⁻⁶.

Timing

CUDA Events · 100–2,000 iters

Microsecond-accurate. Warmup before every measurement. No single-shot results.

Energy

pynvml where noted

Actual joules via NVIDIA Management Library. Proxy used where pynvml unavailable — always clearly labelled.

Hashes

4 SHA-256 per run

Weight matrix · input vector · dense baseline · ROLV output. One weight change → hash changes — proves real computation.

Reproducibility

Deterministic · seeded

TF32 off, cuDNN deterministic, fixed seeds. NVIDIA, AMD, TPU, CPU all supported. Full JSON on request.

All benchmarks run end-to-end in a single self-contained harness. Full JSON with all hashes and timings available on request.

03 — How It Works

Four steps from dense weight to ROLV Primitive©. Score → prune → quantize → store sparse. The operator is built once per weight matrix and reused across all inference calls. Build time is amortised across thousands of calls.

04 — On Correctness

The ROLV Primitive© is exact on its compressed submatrix — no approximation is introduced by ROLV Primitive© itself. The only source of output error is pruning, which zeroes low-magnitude rows before ROLV Primitive© is built.

This is expected and standard for compressed inference. The goal is to operate within a defined tolerance budget while maximising speed and energy savings. All published results include correctness metrics alongside speedup figures.

3.9×10⁻⁶

Max error

LLaMA-3.1-8B · all sparsity levels

250×

Tighter than ATOL

ATOL=0.001 standard · ROLV achieves 3.9×10⁻⁶

26/26

Correctness PASS

22 H200 + 22 B200 + 22 Intel + 22 AMD + 24 T4 weights + 4 LLaMA levels

✓ ×26

Perturbation tests

One weight change → output hash changes every run

05 — Contact

Contact Us

[email protected] 3 Patents Pending

Extraordinary reductions in compute time and energy.