> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# EMEP Model Merge Tests and Golden Outputs

> Tensor correctness tests per MergeStrategy in EMEP with golden outputs and tolerance policies for SLERP, TIES, DARE, Task Arithmetic, and Franken-Merge.

Model merge tests verify that every MergeStrategy produces mathematically correct and deterministic outputs. Each strategy has a golden reference test and a defined tolerance policy.

## Test Structure

For each strategy, the test suite:

1. Loads a fixed pair of small reference models.
2. Executes the merge with fixed random seeds and fixed parameters.
3. Compares output tensors against a precomputed golden reference.
4. Reports PASS if all tensors are within tolerance, FAIL otherwise.

## Strategies and Tolerances

| Strategy                              | Golden Reference | Tolerance     | Notes                                |
| ------------------------------------- | ---------------- | ------------- | ------------------------------------ |
| SLERP (Shoemake 1985)                 | Precomputed fp32 | 1e-5 relative | Spherical interpolation of weights   |
| TIES (Yadav et al. 2023)              | Precomputed fp32 | 1e-5 relative | Trim, elect, sign consensus          |
| DARE (Yu et al. 2023)                 | Precomputed fp32 | 1e-5 relative | Drop and rescale with fixed seed     |
| Task Arithmetic (Ilharco et al. 2022) | Precomputed fp32 | 1e-5 relative | Linear combination with task vectors |
| Franken-Merge                         | Precomputed fp32 | 1e-5 relative | Layer-wise stitching                 |

## Tolerance Policy

Tolerance is relative to the maximum absolute value of the reference tensor:

```text theme={null}
|output - reference| / max(|reference|) < tolerance
```

For fp16 and bf16 merges, tolerance is relaxed to 1e-3 relative to account for reduced precision. fp8 and int8 merges are tested separately with wider tolerances defined in [Numerical Testing](/testing/numerical-testing).

## Golden Output Generation

Golden references are generated once using fp32 arithmetic on a fixed CPU backend. The generation script is versioned in ArtifactStore. Regeneration requires a manual trigger and a code review. Golden references are stored as safetensors files with SHA-256 checksums.

## Edge Cases

The test suite covers these edge cases for every strategy:

* Merging identical models (output must equal input).
* Merging models with zero delta (output must equal input).
* Single-layer models.
* Models with tied embeddings.
* Models with different vocabularies (must fail compatibility before merge).
* Empty parameter sets (must error gracefully).

## Regression Detection

Any change to TensorEngine, MergeEngine, or a MergeStrategy implementation must pass the full merge test suite. A failure blocks the merge request. The CI job uploads diff tensors to ArtifactStore for inspection.

## Integration Points

* **TensorEngine**: merge tests exercise the full tensor operation stack.
* **Numerical Testing**: tolerance thresholds are shared with the numerical test suite.
* **CI**: merge tests run on every commit and nightly.
* **ArtifactStore**: stores golden references, test outputs, and diff artifacts.