layernorm

std.nn.layernorm · Level L1

Layer normalisation of one feature vector, with scale γ, shift β and a small ε. Calls variance.

(x − x̄)/√(σ² + ε)·γ + β

Signature

layernorm(x: f64[n], gamma: f64[n], beta: f64[n], eps: f64[]) → f64[n]

Structure

The function as NOVA stores it: one box per input, operation and output, and arrows that carry values. A double border marks another library function this one runs — called once, or by Scan once per element; select it to open that function.

xf64[n]epsf64[]gammaf64[n]betaf64[n]MeanmuvariancevSubtractdAddveSqrtsdDividexnMultiplyscaledAddyyf64[n]
  • input
  • operation
  • constant
  • call
  • output

Verification

  • Signature proven by NOVA’s shape solver, for every size.
  • Agrees with the reference (x - x.mean()) / np.sqrt(x.var() + eps) * gamma + beta to 80 digits (100-digit arithmetic), on all 40 test cases.
  • All 159 float64 results inside the running error bound; the closest uses 97% of it.
  • Interpreter and NumPy backend return bit-identical results.
Accuracy in detail
correctly rounded (the float64 nearest the exact value)
74%
bit-equal to the NumPy formula in float64
100%
largest error, in units in the last place
30

Large ulp counts appear where a result is tiny next to the numbers it is computed from (after cancellation, for example), so one unit in the last place is tiny too; the absolute error is still inside the bound. Results within their own error of zero are not counted.

Identity

Calls
variance
Called by
—
sha256:94f490284675a5947627a8c808a2e6b373897eab270767319f58e142aa6b2936

The semantic hash of the graph. It changes when the program changes, and never when only its documentation does.

Control handle

Symbol
Ω:std.nn.layernorm · Ω:layernorm
Pins
sha256:24ad99a697a59b98ff18c56e16df9eb74d12d2532bddc7b27ca349ea7b13e246this graph and the 1 it reaches through calls
Evidence
sha256:82022d7e20175d724ba0e77585e7209f6b5d395f7bbc5f291004713c21c7298cthe hash of its verification record
Needs
no capability: a pure function

Through NOVA’s control layer, the symbol launches this function only while the program still matches what it pins: a change to this graph, or to any graph it reaches, needs a migration first.