Fitness Engine and Candidate Scoring

Specification for the FitnessEngine in EMEP, covering scalar single-objective aggregation and multi-objective vector fitness computation.

The FitnessEngine converts evaluation metrics into comparable fitness values for the evolutionary loop. It supports both single-objective scalar aggregation and multi-objective vector fitness. This page specifies the aggregation formulas, weighting schemes, and the critical isolation rule: fitness is computed only from the Optimization Set.

Fitness Computation Flow

Single-Objective Scalar Aggregation

The FitnessEngine combines multiple metrics into a single scalar value using weighted summation or target-relative scoring.

Weighted Sum

fitness = sum_i(w_i * normalized_metric_i)
sum(w_i) = 1.0

Weights are configurable per experiment. Default weights balance accuracy, efficiency, and safety:

Metric Category	Default Weight	Normalization
Accuracy	0.5	Best on Optimization Set
Efficiency	0.2	Inverse latency, max 1.0
Safety	0.2	Pass rate on safety suite
Robustness	0.1	Variance across prompt variants

Target-Relative Scoring

Each metric is scored relative to a target value:

score_i = metric_i / target_i
score_i = min(score_i, 1.0)  # cap at 100%
fitness = sum(w_i * score_i)

Target values are set per benchmark or inherited from baseline models. Target-relative scoring makes fitness interpretable: 1.0 means all targets met.

Multi-Objective Vector Fitness

For multi-objective optimization, fitness is a vector rather than a scalar. The EvolutionEngine passes this vector to the non-dominated sorting algorithm (NSGA-II, Deb et al. 2002).

fitness_vector = [accuracy, efficiency, safety, robustness]

Each component is independently normalized. The Pareto front contains candidates where no other candidate dominates on all objectives.

Example Aggregation Formulas

Standard Accuracy-Weighted Fitness

fitness = 0.6 * normalize(accuracy) + 0.25 * normalize(1 / latency) + 0.15 * normalize(safety_pass_rate)

Balanced Multi-Objective Vector

fitness_vector = [
  normalize(perplexity),      # lower is better, inverted
  normalize(exact_match),     # higher is better
  normalize(1 / memory_gb),    # efficiency
  normalize(refusal_rate)     # safety
]

Target-Relative for Instruction Following

targets = {exact_match: 0.75, rouge_l: 0.70, safety: 0.95}
scores = {k: min(v / targets[k], 1.0) for k, v in metrics.items()}
fitness = mean(scores.values())

Metric Normalization

All metrics are normalized to [0, 1] before aggregation:

• For metrics where higher is better: norm = (value - min) / (max - min)
• For metrics where lower is better: norm = 1 - (value - min) / (max - min)
• If max equals min: norm = 0.5

Min and max are computed from the current population’s Optimization Set results. This makes fitness relative to the current search progress.

Isolation Rule

Fitness is computed ONLY from the Optimization Set. The Validation Set and Hidden Test Set are never used during fitness calculation, ranking, or selection. This rule is enforced at the data layer. Any attempt to compute fitness from non-optimization splits triggers an error and halts the experiment.

Integration with EvolutionEngine

The FitnessEngine exposes:

FitnessEngine.compute_scalar(metrics, weights) -> float
FitnessEngine.compute_vector(metrics) -> List[float]
FitnessEngine.normalize(metrics, population_stats) -> Dict[str, float]

All results are logged to the ExperimentTracker with the full metric dictionary and computed fitness values.