Tensor Shape Validation Algorithm and Rules

Tensor shape validation specification for EMEP. Iterates over state_dict, compares shapes and dtypes per parameter, reports first mismatch, and references broadcasting rules from Tensor Math.

Tensor shape validation is the final gate before a merge can proceed. This page specifies the algorithm that iterates over state_dict entries, compares shapes and dtypes, and reports the first fatal mismatch.

Validation Algorithm

Dtype Compatibility Rules

Dtype A	Dtype B	Compatible?	Action if CONDITIONALLY_COMPATIBLE
fp32	fp32	Yes	None
fp32	bf16	CONDITIONALLY_COMPATIBLE	Cast to fp32 or bf16
fp32	fp16	CONDITIONALLY_COMPATIBLE	Cast to fp32 or fp16
bf16	bf16	Yes	None
bf16	fp16	CONDITIONALLY_COMPATIBLE	Cast to fp32
fp16	fp16	Yes	None
int8	int8	Yes	None
int8	fp16	CONDITIONALLY_COMPATIBLE	Dequantize or quantize
int4	int4	Yes	None
int4	any	CONDITIONALLY_COMPATIBLE	Dequantize

Broadcasting Rules

When shapes do not match exactly, the TensorEngine applies broadcasting rules defined in Tensor Math. Broadcasting is only permitted when the shapes are compatible per elementwise rules: dimensions must be equal or one of them must be 1. Broadcasting across parameter keys (e.g., different layer counts) is not supported and produces INCOMPATIBLE.

Mismatch Report Format

The first mismatch terminates validation and produces a structured report:

{
  "status": "INCOMPATIBLE",
  "key": "model.layers.0.self_attn.q_proj.weight",
  "expected_shape": [4096, 4096],
  "actual_shape": [4096, 5120],
  "expected_dtype": "float32",
  "actual_dtype": "float32"
}

Cross-Links

• Model Compatibility for the full decision tree.
• Architecture Compatibility for architecture checks.
• Tensor Math for broadcasting and elementwise rules.
• Merge Engine for how validation feeds into merging.