gelu_tanh

std.elementwise.gelu_tanh · Level L0

GELU, tanh approximation, as used in GPT-style networks.

½·x·(1 + tanh(c·(x + 0.044715·x³))), c = √(2/π)

Signature

gelu_tanh(x: f64[n]) → f64[n]

Structure

The function as NOVA stores it: one box per input, operation and output, and arrows that carry values. A double border marks another library function this one runs — called once, or by Scan once per element; select it to open that function.

xf64[n]0.50.044715cubex3MultiplyhxMultiplykx30.797885AddinnerMultiplyarg1.0TanhthAddonepMultiplyyyf64[n]
  • input
  • operation
  • constant
  • call
  • output

Verification

  • Signature proven by NOVA’s shape solver, for every size.
  • Agrees with the reference 0.5 * x * (1 + np.tanh(c * (x + 0.044715 * x**3))) to 80 digits (100-digit arithmetic), on all 40 test cases.
  • All 162 float64 results inside the running error bound; the closest uses 73% of it.
  • Interpreter and NumPy backend return bit-identical results.
Accuracy in detail
correctly rounded (the float64 nearest the exact value)
65%
bit-equal to the NumPy formula in float64
100%
largest error, in units in the last place
1.8e+9

Large ulp counts appear where a result is tiny next to the numbers it is computed from (after cancellation, for example), so one unit in the last place is tiny too; the absolute error is still inside the bound. Results within their own error of zero are not counted.

Note

c is the float64 value nearest √(2/π); the reference uses the same constant.

Identity

Calls
cube
Called by
—
sha256:3c2cdf2da16959ecd913253994d8a490b2710d09d69fa1ddb6becf33158f1573

The semantic hash of the graph. It changes when the program changes, and never when only its documentation does.

Control handle

Symbol
Ω:std.elementwise.gelu_tanh · Ω:gelu_tanh
Pins
sha256:d6ad254247d6e158e9a9ab7f011a6b59166bd6ed4c4deee1e65272f52b227247this graph and the 1 it reaches through calls
Evidence
sha256:bb03bd4e957c011c1977d5303bca35f4ad401e97ba6c0c61a0e883af19697d6dthe hash of its verification record
Needs
no capability: a pure function

Through NOVA’s control layer, the symbol launches this function only while the program still matches what it pins: a change to this graph, or to any graph it reaches, needs a migration first.