June 28, 2026 8 min read

QCFS Threshold (Lambda) Not Learning During ANN-to-SNN Conversion: Root Cause

If your QCFS clipping threshold freezes near its initial value and never moves during calibration, the method isn't broken - your optimizer configuration is. Here's the exact gradient-flow mechanism and the fix.

What QCFS is supposed to do

Quantization Clip-Floor-Shift (Bu et al., 2023) is a calibration method for converting a trained ANN into a spiking neural network. It replaces each ReLU activation with a function that clips, floors, and shifts the activation into discrete quantization levels, controlled by a learnable per-layer threshold parameter, typically written as λ (lambda) or θ (theta):

QCFS(x, λ, L) = λ · clip(floor((x / λ) · L + 0.5) / L, 0, 1)

The threshold λ is meant to learn the right firing threshold for each layer's converted spiking neurons, so the network's spike rate over T timesteps approximates the original ReLU's continuous output. When this works, the converted SNN should recover most of the original ANN's accuracy with a small number of timesteps. When it doesn't work, λ sits at (or very near) its initial value - often 1.0, or whatever default the implementation initializes it to - for the entire calibration run, and the converted network never properly calibrates per-layer.

Why this looks like a method failure but isn't

It is tempting, after a calibration run where λ doesn't move, to conclude that QCFS as a method doesn't generalize, or that learnable thresholds don't help. That conclusion is wrong, and the giveaway is in the gradient, not the result. If you log gradient norms separately for λ parameters versus convolutional weight parameters during the first few calibration steps, you will typically see λ's gradient norm sitting one to two orders of magnitude below the weights'. That's not a sign the method has no learning signal for λ - it's a sign the learning signal exists but is small, and a shared learning rate tuned for weights is far too conservative for it.

Where the small gradient comes from

λ only enters the QCFS function through the clip and floor operations. The floor function's gradient is zero almost everywhere by construction (it's piecewise constant), so in practice QCFS implementations use a surrogate gradient through the floor, typically approximated as 1 within the active range and 0 outside the clip bounds. That means λ's gradient contribution comes almost entirely from activations sitting near the clip boundary - a much smaller fraction of the total activation distribution than the activations that contribute gradient to the layer's weights through the main matrix multiply. Fewer activations contributing gradient means a smaller aggregate gradient magnitude for λ, by construction of the function, not as an implementation accident.

The fix: separate parameter groups, separate learning rate

The standard fix, consistent with how the original QCFS paper and most working reimplementations handle this, is to never let λ share an optimizer learning rate with the convolutional/linear weights:

lambda_params = [p for n, p in model.named_parameters() if 'lambda' in n]
weight_params = [p for n, p in model.named_parameters() if 'lambda' not in n]

optimizer = torch.optim.SGD([
    {'params': weight_params, 'lr': 0.01},
    {'params': lambda_params, 'lr': 0.5},   # 10-50x higher
])

The exact multiplier depends on the network depth and the specific activation statistics of the pretrained ANN, but 10-50x the base learning rate is a reasonable starting search range. The right value is the one where gradient-norm logging shows λ actually moving by a meaningful fraction of its initial value within the first epoch, not the one that "feels aggressive."

Two more contributing causes worth ruling out

Initialization far from the right scale

If λ is initialized to a constant like 1.0 for every layer regardless of that layer's actual activation range, a small gradient has to travel a long distance to reach the right value. Initializing λ per-layer from the ANN's actual activation statistics (e.g. the 99.7th percentile of that layer's pre-activation outputs on a calibration batch) gives the gradient a much shorter distance to travel, which compounds with the learning-rate fix rather than replacing it.

Early layers calibrating before later layers have stabilized

In a deep network, layer 1's optimal λ depends on what layer 2 onward does with its output, and vice versa. If you don't see λ moving specifically in layers 1-2 while later layers calibrate normally, check whether your calibration schedule trains all layers jointly from step one, versus a staged or layer-wise warmup - joint training from a poor initialization can leave early layers stuck in a local optimum that a staged calibration avoids.

How to confirm you actually have this bug

What a properly calibrating QCFS run looks like

Lambda values that converge to layer-specific numbers reflecting that layer's actual activation scale are the signature of correct calibration. A value that ends within rounding distance of its initialization, especially in early layers, is the signature of this bug - not evidence that learnable QCFS thresholds don't help.

Frequently asked questions