SLERP Math: Formula, Boundaries, and Fallback

SLERP mathematical specification for EMEP. Defines the spherical linear interpolation formula, shape expectations, boundary cases near theta = 0 and theta = pi, and fallback to lerp. Reference: Shoemake 1985.

SLERP (Spherical Linear Interpolation) interpolates between two parameter vectors along the shortest arc on a hypersphere. This page defines the formula, boundary cases, and fallback rules used by the TensorEngine.

Formula

Given two parameter vectors A and B in R^d, and interpolation parameter t in [0, 1]:

theta = arccos( (A dot B) / (||A||_2 * ||B||_2) )

SLERP(A, B, t) = (sin((1 - t) * theta) / sin(theta)) * A
               + (sin(t * theta) / sin(theta)) * B

Normalization is applied per parameter tensor, not globally across the model.

Symbol Table

Symbol	Meaning
A, B	Parameter vectors (flattened tensors)
t	Interpolation parameter in [0, 1]
theta	Angle between A and B in radians
		.		_2	L2 norm
dot	Dot product
sin	Sine function
arccos	Inverse cosine

Shape Expectations

SLERP operates on flattened 1-D vectors. For a weight tensor of shape (H, I), both A and B are reshaped to length H * I before interpolation. The result is reshaped back to (H, I).

Input Shape	Flattened Length	Output Shape
(H, I)	H * I	(H, I)
(V, H)	V * H	(V, H)
(H,)	H	(H,)

Boundary Cases

Theta Near Zero

When A and B are nearly parallel, theta approaches 0 and sin(theta) approaches 0. Division by sin(theta) becomes numerically unstable.

Condition	Action
theta < 1e-6	Fallback to linear interpolation (lerp)

Fallback formula:

SLERP(A, B, t) = (1 - t) * A + t * B

Theta Near Pi

When A and B are nearly opposite, theta approaches pi. The shortest path is ambiguous: both arcs have equal length.

Condition	Action
	theta - pi	< 1e-6	Fallback to lerp

Engineering assumption: in practice, parameter vectors from fine-tuned variants of the same base model rarely diverge to theta near pi. This case is logged but not specially handled.

Numerical Stability

All dot products and norms are computed in fp64. The result is cast to fp32 before the final weighted sum. This prevents loss of precision when A and B are nearly orthogonal.

Example

For two 3-D vectors:

A = [1, 0, 0]
B = [0, 1, 0]
t = 0.5

theta = arccos(0) = pi / 2

SLERP(A, B, 0.5) = (sin(pi/4) / sin(pi/2)) * A + (sin(pi/4) / sin(pi/2)) * B
                 = (sqrt(2)/2) * A + (sqrt(2)/2) * B
                 = [0.707, 0.707, 0]

Cross-Links

• Merge Math for shared notation and weight shapes.
• Merge Strategies for when SLERP is selected.
• Tensor Operations for the TensorEngine implementation.