> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# TIES Math: Trim, Elect Sign, and Disjoint Merge

> TIES mathematical specification for EMEP. Defines the three-step trim-by-magnitude, elect-sign, and disjoint-merge formulas with density parameter and rescaling. Reference: Yadav et al. 2023.

TIES (Trim, Elect Sign, and Merge) reduces interference in multi-model merging by keeping only the most significant parameter changes and resolving sign conflicts. This page defines the three-step math used by the TensorEngine.

## Overview

TIES operates on task vectors tau\_i = theta\_ft\_i - theta\_pre. The three steps are:

1. **Trim:** Keep only the top-d fraction of parameters by magnitude in each task vector.
2. **Elect Sign:** Resolve sign conflicts across task vectors by majority vote.
3. **Disjoint Merge:** Merge only parameters that agree with the elected sign.

## Step 1: Trim by Magnitude

Given a task vector tau and density parameter d in (0, 1]:

```text theme={null}
M_d(tau) = mask where |tau_j| >= threshold_d

threshold_d = d-th percentile of |tau|

tau_trimmed = tau circle_dot M_d(tau)
```

| Symbol      | Meaning                                           |        |                     |
| ----------- | ------------------------------------------------- | ------ | ------------------- |
| tau         | Task vector                                       |        |                     |
| d           | Density parameter, fraction of parameters to keep |        |                     |
| M\_d(tau)   | Binary mask: 1 if                                 | tau\_j | is in top d, else 0 |
| circle\_dot | Element-wise (Hadamard) product                   |        |                     |

Parameters with magnitude below the d-th percentile are zeroed out.

## Step 2: Elect Sign

Given N trimmed task vectors tau\_trimmed\_i, elect a sign for each parameter position j:

```text theme={null}
s_j = sign( sum_i tau_trimmed_i_j )
```

| Symbol  | Meaning                                   |
| ------- | ----------------------------------------- |
| s\_j    | Elected sign for position j: +1, -1, or 0 |
| sign(x) | +1 if x > 0, -1 if x \< 0, 0 if x = 0     |

If the sum is exactly zero, the elected sign is 0 and the parameter is excluded from merging.

## Step 3: Disjoint Merge

Merge only parameters where the elected sign matches the task vector's sign:

```text theme={null}
tau_merged_j = sum_i [ tau_trimmed_i_j if sign(tau_trimmed_i_j) == s_j else 0 ]
```

The final merged model is:

```text theme={null}
theta_merged = theta_pre + tau_merged
```

## Rescaling

After disjoint merge, the magnitude of tau\_merged may differ from the average magnitude of the source task vectors. Rescaling is optional and controlled by a config flag:

```text theme={null}
tau_rescaled = tau_merged * ( mean_i ||tau_trimmed_i||_2 / ||tau_merged||_2 )
```

If ||tau\_merged||\_2 is zero (all parameters excluded), the merge fails and produces INVALID.

## Parameters

| Parameter      | Symbol | Range               | Default    |
| -------------- | ------ | ------------------- | ---------- |
| density        | d      | (0, 1]              | 0.6        |
| sign consensus | -      | "majority" or "all" | "majority" |
| rescale        | -      | bool                | false      |

## Failure Modes

| Condition                            | Outcome                         |
| ------------------------------------ | ------------------------------- |
| d = 0                                | All parameters trimmed; INVALID |
| All signs conflict at every position | tau\_merged = 0; INVALID        |
| Rescaling with zero norm             | INVALID                         |

## Cross-Links

* [Merge Math](/math/merge-math) for shared notation and task vector definitions.
* [Merge Strategies](/merge/merge-strategies) for when TIES is selected.
* [DARE Math](/math/dare-math) for combining TIES with DARE sparsification.
* [Tensor Operations](/merge/tensor-operations) for the TensorEngine implementation.