August 14, 2026 · 24 min read

Reproduce NeuroCUDA Results Guide

NeuroCUDA is an open-source PyTorch-to-SNN compiler. Install with pip install neurocuda (or pip install neurocuda[all]). Source is MIT licensed at https://github.com/Krishnav1/neurocuda. To reproduce NeuroCUDA results, clone the repo and run python reproduce.py --quick for N-MNIST in about four minutes, then compare against the published table.

This is the lab notebook for matching published NeuroCUDA numbers from the public GitHub repository: clone, virtualenv, data, seeds, expected table, and what to do when a number misses.

This page vs nearby URLs: this URL is the reproduction protocol. /neurocuda is the product hub. /blog/nmnist-snn-conversion is the N-MNIST conversion walkthrough. /blog/convert-pytorch-to-snn is the generic HowTo. /blog/resnet18-snn-conversion-tutorial is the CIFAR-10 tutorial. Do not treat this page as a "what is NeuroCUDA" article.

Reviewers, CI owners, and students who search reproduce neurocuda results already know the compiler exists. They need a protocol that turns a GitHub clone into the same table printed in the technical report. This page is that protocol. It is not a product pitch and it is not a conversion tutorial. If you still need the definition of the compiler, start at the NeuroCUDA hub and come back here with a terminal open.

Published numbers only mean something if a third party can rerun them. NeuroCUDA ships reproduce.py in the repository root for that reason. The quick path is N-MNIST. The longer path is 3-seed QCFS on the same stack plus NIR and NeuroBench gates. Each path has a different expected runtime, a different expected table row, and a different failure mode. Mixing those paths is how honest reproductions get marked as "failed" when they actually passed the claim they were testing.

How to reproduce NeuroCUDA results from GitHub

The source of truth is the MIT-licensed repository at https://github.com/Krishnav1/neurocuda. The installable package is pypi.org/project/neurocuda. The citable table is quantaracore.in/neurocuda/paper.pdf. When you reproduce NeuroCUDA results, log all three: commit SHA, PyPI version, and the paper revision date. A mismatch among those three is the first thing to check, not the last.

Use a dedicated virtual environment. Mixing NeuroCUDA with an older PyTorch wheel, a system CUDA toolkit that does not match the wheel, or another SNN library in the same site-packages tree is a common source of silent numeric drift. The commands below assume Linux or macOS. Windows users should activate the venv with the PowerShell script instead of source.

git clone https://github.com/Krishnav1/neurocuda.git
cd neurocuda
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip

# released package
pip install neurocuda

# extras: NIR, NeuroBench, CartPole demo
pip install neurocuda[all]

# optional: editable install from this clone
# pip install -e ".[all]"

python -c "import neurocuda, torch; print(neurocuda.__version__); print(torch.__version__); print(torch.cuda.is_available())"

If you have not installed the package before, read the pip install neurocuda guide for CUDA wheel pairing. Reproduction does not require a GPU. A GPU changes wall-clock time. It should not change spike counts on the published CPU versus GPU check (0 deviations across 256000 spikes).

Prepare N-MNIST data

N-MNIST is the first published row and the target of python reproduce.py --quick. Do not invent your own event-to-frame conversion if the repo already ships a prep script. Use examples/prep_nmnist.py so the tensor layout, polarity stacking, and train/test split match the paper.

python examples/prep_nmnist.py
# expected: a local data directory the loaders in reproduce.py already know about
ls -lh data 2>/dev/null || ls -lh ./

Disk space is modest compared with ImageNet, but the download can fail on flaky networks. If prep stops halfway, delete the partial directory and rerun the script. A truncated cache is a frequent cause of "accuracy around chance" that has nothing to do with QCFS. For the conversion theory behind this dataset, see N-MNIST SNN conversion. That page teaches the model. This page checks the number.

Run the quick path

The command you should be able to paste after clone and install is:

python reproduce.py --quick

On a typical NVIDIA GPU this finishes N-MNIST conversion and full-test evaluation in about 4 minutes. That timing is a convenience estimate, not a published benchmark. CPU-only machines will take longer. Do not fail a reproduction because a laptop took 25 minutes instead of 4. Fail it if the full-test accuracy is outside the published band after you have controlled seed, split, and commit.

The API under the script is the same four calls used everywhere else in the docs. convert returns the spiking network and a metadata object. Fine-tune, compile, and evaluate are separate steps so you can inspect thresholds before you spend GPU time.

import torch
import neurocuda

# calib_loader must be in-distribution N-MNIST frames, not CIFAR tensors
snn, meta = neurocuda.convert(model, calib_loader)
neurocuda.finetune(snn, train_loader)
neurocuda.compile(snn, target="gpu")  # or target="cpu"
# then evaluate on the FULL N-MNIST test set

If you are following along in a notebook instead of reproduce.py, keep the same split and the same seed the script uses. A custom notebook that samples 1,000 test events is not a reproduction even if the number looks close.

Expected table (what "pass" means)

These are the rows you are allowed to claim after a successful run. All accuracy rows are mean ± standard deviation over 3 or more seeds on the full test set. Single-run screenshots are not the published claim.

Read the N-MNIST row carefully. The SNN is allowed to beat the ANN. A reproduction that prints 99.70% SNN and 99.70% ANN is not "close enough." It missed the published conversion outcome. A reproduction that prints 99.87% or 99.89% on a listed seed is inside a 0.02-point band and should be compared against the seed-level logs, not against a rounded marketing headline.

The ResNet-18 row is a different experiment. T=32 is part of the claim. Evaluating a converted ResNet at T=8 and then declaring the paper unreproducible is a protocol error. The CIFAR-10 tutorial at ResNet-18 SNN conversion explains why residual merges need enough timesteps. QCFS background lives on QCFS ANN-to-SNN.

Seeds, devices, and what you must log

A reproduction without a seed log is a demo. Pin Python, PyTorch, NeuroCUDA, CUDA (if any), GPU name, and the git commit. Set deterministic flags when the script does. Do not enable extra data augmentation in the calibration loader.

import os, random, numpy as np, torch

SEED = 42
random.seed(SEED)
np.random.seed(SEED)
torch.manual_seed(SEED)
if torch.cuda.is_available():
    torch.cuda.manual_seed_all(SEED)

# log the environment, then run convert
print("seed", SEED)
print("device", "cuda" if torch.cuda.is_available() else "cpu")
print("commit", os.popen("git rev-parse HEAD").read().strip())

The published N-MNIST and ResNet rows use 3+ seeds. One lucky seed is not the paper. If --quick is a single-seed smoke test in your checkout, treat it as a sanity check, then run python reproduce.py full-ish or bash benchmarks/reproduce.sh before you claim you matched the table. Gate3 in benchmarks/reproduce.sh is the QCFS 3-seed path. That is the row reviewers should ask for.

When you reproduce neurocuda results on a shared cluster, also log job ID, node name, and whether another process was using the GPU. Thermal throttling changes time, not the 99.88% claim, but it confuses people who treat the 4-minute estimate as a spec.

full-ish, gate3, NIR, and NeuroBench

After the 4-minute N-MNIST path, the repository expects a longer ladder:

python reproduce.py full-ish
bash benchmarks/reproduce.sh

benchmarks/reproduce.sh is the operator script for the remaining gates. In the public layout it drives gate3 QCFS across 3 seeds, then verify_nir_trained.py, then gate5_neurobench.py. Run it from the repo root with the same venv you used for --quick. Do not start a second venv mid-ladder. Version skew between steps is a classic false mismatch.

verify_nir_trained.py checks that a trained/converted graph survives write and read as NIR. The published ResNet-18 round-trip is bit-exact: 0.000000 max abs diff. That is a numeric identity check on residual graphs, not a statement that NIR is ONNX. For the export HowTo see export PyTorch to NIR. For the format definition see what is NIR. For the NIRTorch comparison see NeuroCUDA vs NIRTorch.

gate5_neurobench.py writes a NeuroBench-style report. Sparsity on ResNet-18/CIFAR-10 is in the same family as the ~93.7% figure discussed on the hub. The gate is that the report is generated in the documented format, not that a third-party leaderboard scraped it. See NeuroCUDA on Google Colab if you need a cloud GPU instead of a local card.

ResNet helpers in the repo:

python examples/resnet_pipeline.py
python examples/convert_resnet.py

Use those after N-MNIST is green. CIFAR-10 download is separate from N-MNIST prep. Keep both datasets. Overwriting ./data between gates is another avoidable mismatch.

CPU versus GPU time (and what must still match)

Wall-clock time is not in the published accuracy table. GPU convert plus BPTT fine-tune is why --quick can finish N-MNIST in about 4 minutes. The same path on CPU is slower, sometimes by a large factor depending on core count and whether you compiled against a CPU PyTorch wheel. That is expected.

What is not optional is spike parity. The published CPU versus GPU check is 0 deviations across 256000 spikes. If your GPU run hits 99.88% and your CPU run hits 97% on the same seed and the same weights, you have a real bug or a loader bug. If GPU takes 4 minutes and CPU takes 40 minutes and both hit 99.88% with matching spikes, you have a successful reproduction with different clocks.

neurocuda.compile(snn, target="cpu")
# evaluate, log accuracy and a spike checksum
neurocuda.compile(snn, target="gpu")
# evaluate the same minibatch, compare spike tensors

Use CPU in CI when you do not want a GPU runner. Use GPU when you are matching the 4-minute N-MNIST loop. Use both when you are checking the 256000-spike row. Do not mix a GPU-calibrated checkpoint with a CPU eval that silently changes dtype or memory format without logging it.

For CUDA-as-a-field context, not as a reproduction step, see neuromorphic CUDA. That page explains simulators versus compilers. This page only cares whether your clone matched the table.

Mismatch protocol

When a number misses, do not open with "the paper is wrong." Walk this list in order. Most misses are protocol, not math.

1. Confirm you ran the published path. python reproduce.py --quick is N-MNIST. It does not prove ResNet-18 94.61%. bash benchmarks/reproduce.sh is the 3-seed QCFS plus NIR plus NeuroBench ladder.
2. Confirm full test set. Subsampled eval can look better or worse. The claim is full test.
3. Confirm seed and mean. One seed is a smoke test. The ± 0.02% and ± 0.14% bands are over 3+ seeds.
4. Confirm timesteps. ResNet-18 CIFAR-10 is T=32. Lower T is a different experiment.
5. Confirm data prep. Rerun examples/prep_nmnist.py. Do not mix torchvision MNIST frames with N-MNIST events.
6. Confirm package identity. pip show neurocuda and git rev-parse HEAD. Editable installs can drift from PyPI.
7. Confirm device parity if you switched CPU/GPU. Spikes should match. Time will not.
8. Exclude CartPole from the hard-gate set. See the next section.
9. Exclude silicon labels. SpiNNaker and Loihi are different evidence classes. They cannot rescue or sink an N-MNIST miss.

If you still cannot reproduce neurocuda results after that list, file an issue on GitHub with the log block: SHA, version, seed, device, command, and the exact printed accuracy. A screenshot of a Colab cell with no SHA is not an actionable report. Keep the terminal transcript.

Accuracy collapse after conversion has its own debugging page: convert PyTorch to SNN. Dead neurons, threshold freeze, and wrong normalization show up there. Use those guides when the protocol above is clean and the number is still far from 99.88% or 94.61%.

CartPole is stochastic - not a hard gate

CartPole-v1 appears in the broader NeuroCUDA result matrix because the extras extra includes a reinforcement-learning demo. Direct SNN training can solve the environment (published 100% solved, 68.5% sparsity on that path). ANN-to-SNN conversion of CartPole is a different row: 100% on the best seed, but 19% ± 26% across 5 seeds. That spread is the result. It is not a failure of N-MNIST reproduction.

Do not put CartPole in a CI job that returns red on anything other than 100%. Gymnasium dynamics, episode length, and seed interaction make it a poor hard gate. If you run it, report the mean and spread and move on. Reviewers who treat CartPole as the primary NeuroCUDA claim are testing the wrong row.

What this reproduction does not prove

Keep evidence classes separate. Mixing them is how a correct N-MNIST clone turns into an overclaim.

• SpiNNaker-1 physical silicon is confirmed on EBRAINS jobs #420148 and #420186 as a 2-neuron smoke test. It is not ResNet-on-chip. Details: SpiNNaker physical silicon.
• Loihi 2 in this stack is an IF-neuron simulator checked against published equations. It is not Loihi silicon.
• NIR bit-exact 0.000000 is a residual-graph round-trip on ResNet-18. It is not a statement that every vendor runtime will execute that file identically.
• CPU vs GPU 0/256000 is spike identity on that check. It is not equal energy or equal latency.

Comparisons against other SNN libraries belong on their own URLs: NeuroCUDA vs SpikingJelly and NeuroCUDA vs NIRTorch. A reproduction log should not include a ranking table unless you also reran those tools under the same seeds, which this protocol does not require.

Worked command block (copy once)

This is the full sequence a new machine should run to match the N-MNIST row and then climb the gates. Stop after --quick if you only needed the 4-minute check.

git clone https://github.com/Krishnav1/neurocuda.git
cd neurocuda
python3 -m venv .venv
source .venv/bin/activate
pip install -U pip
pip install neurocuda[all]
python examples/prep_nmnist.py
python reproduce.py --quick
# if N-MNIST matches the band:
python reproduce.py full-ish
bash benchmarks/reproduce.sh
# optional ResNet path
python examples/resnet_pipeline.py
python examples/convert_resnet.py

API reminder used inside those scripts:

snn, meta = neurocuda.convert(model, calib_loader)
neurocuda.finetune(snn, train_loader)
neurocuda.compile(snn, target="gpu")
# NIR is a later gate, not required to pass N-MNIST:
# neurocuda.to_nir(snn, "nmnist_snn.nir")

Keep terminal transcripts and log files on disk plus the git SHA. Those are the artifacts a reviewer can rerun.

Time budget for a reviewer

Plan the calendar around gates, not around hope.

CPU-only reviewers should still run --quick. They should not be asked to finish 3-seed ResNet in a lunch break. State the device in the report so GPU and CPU reproductions are not averaged into one fake mean.

Common false mismatches

• Using MNIST instead of N-MNIST. Different dataset. Different number.
• Evaluating 10% of the test set. Faster, not published.
• Comparing a T=8 ResNet to the T=32 row. Wrong row.
• Reporting train accuracy. The table is test.
• Mixing seeds in the mean. Average the listed seeds, do not drop the worst one.
• Calling SpiNNaker smoke a failed ResNet silicon run. It was never ResNet-on-chip.
• Failing CI on CartPole. Stochastic. Not a hard gate.

If you are converting your own checkpoint rather than reproducing the paper checkpoint, you have left this page's job. Use convert PyTorch to SNN and compare against your ANN baseline, not against 99.88%.

Frequently asked questions

Start now: pip install neurocuda · GitHub · Product hub · PDF report