Architecture Compatibility Algorithm and Checks - EMEP

Architecture compatibility verifies that two models share the same structural blueprint. This page specifies the algorithm, the exact checks, and the conditions under which a cross-family merge can proceed.

Check Dimensions

The ModelCompatibilityAnalyzer evaluates architecture across these dimensions:

Dimension	What It Checks
Model family	Llama, Mistral, Qwen, Gemma, etc.
Model type	base, instruct, chat, code
Layer count	Number of transformer layers
Hidden size	Dimension per layer
Intermediate size	MLP expansion dimension
Attention heads	Count and grouping
KV heads	GQA/MQA head count
Embedding size	Vocab embedding dimension
RoPE base	Rotary base frequency
RoPE theta	Scaling factor
RoPE scaling	None, linear, NTK, YaRN
Context length	Maximum supported context

Compatibility Decision Flow

Cross-Family Merge Rule

Cross-family merges are INCOMPATIBLE unless the MergeStrategy is explicitly Franken-Merge and adapters are declared. This is an engineering assumption: structural differences between families (attention variants, MLP layouts, normalization placement) require hand-designed adapter layers that cannot be generated automatically.

CONDITIONALLY_COMPATIBLE Cases

Case	Required Action
RoPE mismatch	Strategy must interpolate or override RoPE config
Context mismatch	Strategy must clamp or scale context
Franken-Merge with adapters	Strategy must provide and validate adapter weights

INCOMPATIBLE Cases

Case	Reason
Layer count mismatch	Tensor shapes diverge at every layer boundary
Hidden size mismatch	Attention and MLP shapes incompatible
Attention/KV heads mismatch	GQA grouping invalid
Embedding size mismatch	Output projection shape invalid
Cross-family without adapters	Structural layout incompatible

Cross-Links

Model Compatibility for the full decision tree.
Tokenizer Compatibility for the tokenizer branch.
Tensor Shape Validation for the tensor branch.
Merge Strategies for Franken-Merge details.