> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# Fitness Engine and Candidate Scoring

> Specification for the FitnessEngine in EMEP, covering scalar single-objective aggregation and multi-objective vector fitness computation.

The FitnessEngine converts evaluation metrics into comparable fitness values for the evolutionary loop. It supports both single-objective scalar aggregation and multi-objective vector fitness. This page specifies the aggregation formulas, weighting schemes, and the critical isolation rule: fitness is computed only from the Optimization Set.

## Fitness Computation Flow

```mermaid theme={null}
flowchart TD
    START([START]) --> INPUT[Evaluation Metrics]
    INPUT --> SPLIT{Split Source?}
    SPLIT -->|Optimization Set| PROC[Process Metrics]
    SPLIT -->|Validation Set| BLOCK[Blocked]
    SPLIT -->|Hidden Test Set| BLOCK
    BLOCK --> END1([END - Not Used])
    PROC --> MODE{Fitness Mode?}
    MODE -->|Single| SCALAR[Scalar Aggregation]
    MODE -->|Multi| VECTOR[Vector Fitness]
    SCALAR --> OUTPUT[Return Fitness Value]
    VECTOR --> OUTPUT
    OUTPUT --> END2([END])
```

## Single-Objective Scalar Aggregation

The FitnessEngine combines multiple metrics into a single scalar value using weighted summation or target-relative scoring.

### Weighted Sum

```text theme={null}
fitness = sum_i(w_i * normalized_metric_i)
sum(w_i) = 1.0
```

Weights are configurable per experiment. Default weights balance accuracy, efficiency, and safety:

| Metric Category | Default Weight | Normalization                   |
| --------------- | -------------- | ------------------------------- |
| Accuracy        | 0.5            | Best on Optimization Set        |
| Efficiency      | 0.2            | Inverse latency, max 1.0        |
| Safety          | 0.2            | Pass rate on safety suite       |
| Robustness      | 0.1            | Variance across prompt variants |

### Target-Relative Scoring

Each metric is scored relative to a target value:

```text theme={null}
score_i = metric_i / target_i
score_i = min(score_i, 1.0)  # cap at 100%
fitness = sum(w_i * score_i)
```

Target values are set per benchmark or inherited from baseline models. Target-relative scoring makes fitness interpretable: 1.0 means all targets met.

## Multi-Objective Vector Fitness

For multi-objective optimization, fitness is a vector rather than a scalar. The EvolutionEngine passes this vector to the non-dominated sorting algorithm (NSGA-II, Deb et al. 2002).

```text theme={null}
fitness_vector = [accuracy, efficiency, safety, robustness]
```

Each component is independently normalized. The Pareto front contains candidates where no other candidate dominates on all objectives.

## Example Aggregation Formulas

**Standard Accuracy-Weighted Fitness**

```text theme={null}
fitness = 0.6 * normalize(accuracy) + 0.25 * normalize(1 / latency) + 0.15 * normalize(safety_pass_rate)
```

**Balanced Multi-Objective Vector**

```text theme={null}
fitness_vector = [
  normalize(perplexity),      # lower is better, inverted
  normalize(exact_match),     # higher is better
  normalize(1 / memory_gb),    # efficiency
  normalize(refusal_rate)     # safety
]
```

**Target-Relative for Instruction Following**

```text theme={null}
targets = {exact_match: 0.75, rouge_l: 0.70, safety: 0.95}
scores = {k: min(v / targets[k], 1.0) for k, v in metrics.items()}
fitness = mean(scores.values())
```

## Metric Normalization

All metrics are normalized to \[0, 1] before aggregation:

* For metrics where higher is better: `norm = (value - min) / (max - min)`
* For metrics where lower is better: `norm = 1 - (value - min) / (max - min)`
* If max equals min: `norm = 0.5`

Min and max are computed from the current population's Optimization Set results. This makes fitness relative to the current search progress.

## Isolation Rule

<Warning>
  Fitness is computed ONLY from the Optimization Set. The Validation Set and Hidden Test Set are never used during fitness calculation, ranking, or selection. This rule is enforced at the data layer. Any attempt to compute fitness from non-optimization splits triggers an error and halts the experiment.
</Warning>

## Integration with EvolutionEngine

The FitnessEngine exposes:

```text theme={null}
FitnessEngine.compute_scalar(metrics, weights) -> float
FitnessEngine.compute_vector(metrics) -> List[float]
FitnessEngine.normalize(metrics, population_stats) -> Dict[str, float]
```

All results are logged to the ExperimentTracker with the full metric dictionary and computed fitness values.