causal_attention

std.nn.causal_attention · Level L2

Causal (decoder) self-attention: row i attends only to rows j ≤ i. The mask compares positions from Iota (j > i is the future); future scores are replaced by the row minimum before the stable softmax, so nothing can overflow, and their weights are set to zero after it.

softmax over j ≤ i of (Q Kᵀ/√d)ᵢⱼ, then · V

Signature

causal_attention(Q: f64[n, d], K: f64[n, d], V: f64[n, d]) → f64[n, d]

Structure

The function as NOVA stores it: one box per input, operation and output, and arrows that carry values. A double border marks another library function this one runs — called once, or by Scan once per element; select it to open that function.

Qf64[n, d]Kf64[n, d]Vf64[n, d]SizedTransposeKTSqrtrdMatMulQKDivideSIotarowIotacolReduceMinlowGreaterfutureWhereSmReduceMaxmSubtractz0.0ExpeWhereemReduceSumtotalDivideAMatMulOOf64[n, d]
  • input
  • operation
  • constant
  • call
  • output

Verification

  • Signature proven by NOVA’s shape solver, for every size.
  • Agrees with the reference for each row i: softmax((Q @ K.T)[i, :i+1] / sqrt(d)) @ V[:i+1] to 80 digits (100-digit arithmetic), on all 40 test cases.
  • All 643 float64 results inside the running error bound; the closest uses 10% of it.
  • Interpreter and NumPy backend return bit-identical results.
Accuracy in detail
correctly rounded (the float64 nearest the exact value)
56%
bit-equal to the NumPy formula in float64
77%
largest error, in units in the last place
87

Large ulp counts appear where a result is tiny next to the numbers it is computed from (after cancellation, for example), so one unit in the last place is tiny too; the absolute error is still inside the bound. Results within their own error of zero are not counted.

Identity

Calls
—
Called by
—
sha256:59d015d39695c7167bcbbecf68f41331d9cee191db5703ea63bd3a3c3e9bce82

The semantic hash of the graph. It changes when the program changes, and never when only its documentation does.

Control handle

Symbol
Ω:std.nn.causal_attention · Ω:causal_attention
Pins
sha256:fd95223488a5457000a0e6c305399407a57e978254668cf906db9c9c193f9ab7this graph alone
Evidence
sha256:0bb7dccd6a235186a7432f13bed00d678166b2fa1f7c8816cdcd39f9ac0b0525the hash of its verification record
Needs
no capability: a pure function

Through NOVA’s control layer, the symbol launches this function only while the program still matches what it pins: a change to this graph, or to any graph it reaches, needs a migration first.