EML-NOVA · Primitives
What the graphs are made of.
Every library function is composed from these primitives. The last column counts the functions that use each one directly.
| Primitive | What it does | Inputs | Differentiable | Used by |
|---|---|---|---|---|
| Constant | A literal value inside the graph. | 0 | yes | 71 |
| Add | Elementwise sum, NumPy-style broadcasting. | 2 | yes | 52 |
| Subtract | Elementwise difference, broadcasting. | 2 | yes | 63 |
| Multiply | Elementwise product, broadcasting. | 2 | yes | 77 |
| Divide | Elementwise quotient, broadcasting. | 2 | yes | 33 |
| Negate | Elementwise negation. | 1 | yes | 16 |
| MatMul | Matrix product over the last two axes; leading axes broadcast. | 2 | yes | 20 |
| Transpose | Permute the axes. | 1 | yes | 14 |
| Reshape | Same elements, new shape; one −1 is inferred. | 1 | yes | 23 |
| ReduceSum | Sum along one axis, or over everything. | 1 | yes | 29 |
| Mean | Mean along one axis, or over everything. | 1 | yes | 18 |
| Relu | max(x, 0), elementwise. | 1 | yes | 6 |
| Sigmoid | 1 / (1 + e⁻ˣ), elementwise. | 1 | yes | 1 |
| Tanh | Hyperbolic tangent, elementwise. | 1 | yes | 3 |
| Softmax | Normalised exponentials along one axis. | 1 | yes | 1 |
| Identity | Pass a value through unchanged. | 1 | yes | — |
| StopGradient | Identity going forward, zero gradient going back. | 1 | yes | — |
| If | Choose between two values by a scalar condition. | 3 | not yet | — |
| BoundedLoop | A loop with a fixed bound; for now its body is one elementwise operation. | 1 | not yet | — |
| Call | Call another graph of the same module. This is how the library composes; since Round 13.1, differentiation follows the call into the callee. | any | yes | 45 |
| Sqrt | Square root, elementwise; correctly rounded (IEEE 754). | 1 | yes | 11 |
| Exp | Exponential, elementwise. | 1 | yes | 9 |
| Log | Natural logarithm, elementwise. | 1 | yes | 12 |
| Maximum | Exact elementwise maximum, broadcasting; ties send the gradient to the first operand. | 2 | yes | 8 |
| Minimum | Exact elementwise minimum, broadcasting; ties send the gradient to the first operand. | 2 | yes | 3 |
| Less | Elementwise a < b as a 0/1 float mask; zero gradient. | 2 | yes | 4 |
| Greater | Elementwise a > b as a 0/1 float mask; zero gradient. | 2 | yes | 11 |
| Equal | Elementwise a = b as a 0/1 float mask; zero gradient. | 2 | yes | 11 |
| Where | Select a where the condition is non-zero, else b; all three broadcast. | 3 | yes | 8 |
| ReduceMax | Largest value along one axis, or overall; ties share the gradient equally. | 1 | yes | 8 |
| ReduceMin | Smallest value along one axis, or overall; ties share the gradient equally. | 1 | yes | 3 |
| Size | The element count of a tensor, or the length of one axis, as a value; zero gradient. | 1 | yes | 9 |
| Slice | A contiguous window along one axis; negative bounds count from the end. A window outside the axis is refused, not clamped. | 1 | yes | 22 |
| Concat | Join tensors along one axis. | any | yes | 12 |
| Iota | The positions 0, 1, …, n−1 along one axis, as values shaped to broadcast; zero gradient. | 1 | yes | 12 |
| Sort | Stable sort along one axis, ascending or descending; the gradient follows the permutation back. | 1 | yes | 3 |
| ArgMax | Position of the first largest value, as a value; zero gradient. | 1 | yes | 1 |
| ArgMin | Position of the first smallest value, as a value; zero gradient. | 1 | yes | 1 |
| CumSum | Running sum along one axis, from the start or from the end. | 1 | yes | 3 |
| CumMax | Running maximum along one axis; ties keep the earlier position. | 1 | yes | 1 |
| CumMin | Running minimum along one axis; ties keep the earlier position. | 1 | yes | — |
| Scan | A loop whose body is another graph: carry a value through a sequence, one step per element, and keep every step. Differentiable: the gradient runs the loop backwards. | any | yes | 7 |
| Cast | Convert between float64 and int64, exactly: a value that is not a whole number, or lies beyond ±2⁵³, is refused, never rounded; zero gradient. | 1 | yes | 5 |
| Gather | Pick slices along an axis by integer indices (np.take); an index outside the axis is refused. The gradient is scattered back. | 2 | yes | 4 |
| GatherAlong | Pick one element per position along an axis (np.take_along_axis): per-row labels, orders from another array. | 2 | yes | 1 |
| ScatterAdd | Add slices into a base at integer indices along an axis; repeated indices accumulate (np.add.at). | 3 | yes | 3 |
| ArgSort | The stable sorting permutation, as positions (floats, like ArgMax); Cast them to index with them. Zero gradient. | 1 | yes | 3 |
| While | A loop that runs a body graph while a condition graph says go on, up to a stated maximum number of steps; exceeding it is an error, not a truncated answer. Differentiable: the gradient replays the steps taken. | any | yes | 4 |
| Solve | Solve A·x = b by Gaussian elimination with partial pivoting; a singular matrix is refused. | 2 | yes | 3 |
| Cholesky | The lower-triangular L with L·Lᵀ = A for a symmetric positive-definite A (reads A's lower triangle). | 1 | yes | 5 |
| TriangularSolve | Forward or back substitution with a triangular matrix. | 2 | yes | 4 |
| EighValues | The eigenvalues of a symmetric matrix, ascending. | 1 | yes | 2 |
| EighVectors | The unit eigenvectors of a symmetric matrix, each signed so its largest component is positive. | 1 | yes | 1 |
| QROrthogonal | The orthonormal factor Q of A = Q·R (Householder QR) for a matrix with at least as many rows as columns. | 1 | yes | 3 |
| QRTriangular | The upper-triangular factor R of A = Q·R, with a positive diagonal; a matrix without full column rank is refused. | 1 | yes | 2 |
| SVDValues | The singular values of a matrix, largest first. | 1 | yes | 6 |
| SVDLeftVectors | The left singular vectors, as columns, each pair signed by its right vector. | 1 | yes | 3 |
| SVDRightVectors | The right singular vectors, as columns, each signed so its largest component is positive. | 1 | yes | 4 |
| FloorDiv | ⌊a/b⌋, rounded toward −∞: exact for integers, the floor of the exact quotient for floats; a zero divisor is refused. | 2 | yes | 3 |
| Mod | a − b·⌊a/b⌋, the remainder with the divisor's sign. | 2 | yes | 9 |
| FFT | The discrete Fourier transform of a complex vector stored as [2, n] (real parts, imaginary parts), unnormalized. | 1 | yes | 4 |
| IFFT | The inverse transform, with its 1/n, so IFFT(FFT(z)) = z. | 1 | yes | 3 |
| Erf | The error function, erf x = (2/√π)·∫₀ˣ e^(−t²) dt. | 1 | yes | — |
| Erfc | The complementary error function 1 − erf x, computed directly so the tail keeps its relative accuracy. | 1 | yes | 1 |
| LogGamma | log|Γ(x)|; the poles at 0, −1, −2, … are refused. Its gradient is the digamma function. | 1 | yes | 5 |
| Expm1 | eˣ − 1, accurate near 0 where exp(x) − 1 loses every digit. | 1 | yes | 1 |
| Log1p | log(1 + x), accurate near 0 where log(1 + x) would round 1 + x first. | 1 | yes | 4 |
| Ndtri | The standard normal quantile Φ⁻¹(p), for 0 < p < 1; computed by NOVA (Acklam, refined by Halley steps). | 1 | yes | 2 |
| GammaInc | The regularized lower incomplete gamma function P(a, x); differentiable in x and in a (∂P/∂a is computed by NOVA, through the same series and continued fraction). | 2 | yes | 3 |
| BetaInc | The regularized incomplete beta function I_x(a, b); differentiable in x, a and b (∂I/∂a and ∂I/∂b are computed by NOVA, through the same continued fraction). | 3 | yes | 3 |
| GammaIncInv | The inverse of P(a, x) in x: the x with P(a, x) = p, for 0 ≤ p < 1; computed by NOVA (Newton on the log tail); differentiable in p and in a (by implicit differentiation). | 2 | yes | 2 |
| BetaIncInv | The inverse of I_x(a, b) in x: the x with I_x(a, b) = p; computed by NOVA (Newton on the log tail, in logit x); differentiable in p, a and b (by implicit differentiation). | 3 | yes | 2 |