> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# EMEP Integration Test Plan and Scenarios

> End-to-end integration test scenarios for EMEP covering register, merge, evaluate, and promote workflows.

Integration tests verify that EMEP components work together across realistic workflows. Tests use real backends where possible and fall back to high-fidelity mocks only for unavailable GPU or storage resources.

## Scenario: Register to Promote

```mermaid theme={null}
flowchart TD
    START(["START"]) --> REGISTER["Register Model A"]
    REGISTER --> REGISTER2["Register Model B"]
    REGISTER2 --> COMPAT["Check Compatibility"]
    COMPAT -->|"COMPATIBLE"| MERGE["Execute Merge"]
    COMPAT -->|"INCOMPATIBLE"| ABORT["Abort + Log"]
    MERGE --> VALIDATE["Merge Validation"]
    VALIDATE -->|"PASS"| EVAL["Evaluate Candidate"]
    VALIDATE -->|"FAIL"| ABORT
    EVAL -->|"PASS"| PROMOTE["Promote to RELEASED"]
    EVAL -->|"FAIL / REGRESSION"| ABORT
    PROMOTE --> END(["END"])
    ABORT --> END
```

This scenario exercises ModelRegistry, ModelCompatibilityAnalyzer, MergeEngine, MergeValidation, EvaluationEngine, BenchmarkEngine, and DeploymentManager in sequence.

## Scenario: Evolution Run

1. Seed population with 4 registered models.
2. Start EvolutionEngine with NSGA-II (Deb et al. 2002).
3. CandidateGenerator produces offspring via crossover and mutation.
4. MergeEngine executes merges for each offspring.
5. EvaluationEngine runs benchmarks on each candidate.
6. FitnessEngine computes multi-objective fitness.
7. Selection reduces population to configured size.
8. Loop until max generations or convergence.
9. Archive final population in ExperimentTracker.

## Scenario: Offline Pack and Deploy

1. Register a model with `emep model register`.
2. Run `emep offline pack` to produce a self-contained package.
3. Transfer package to air-gapped environment.
4. Load package via DeploymentManager.
5. Validate inference with InferenceBackend.
6. Promote to DEPLOYED.

## Test Data

Integration tests use a curated set of small models and datasets stored in ArtifactStore. Model sizes are kept small to fit CI runners. Datasets use the Optimization Set split only; the Hidden Test Set is never used in integration tests.

## Backend Matrix

| Backend        | Used In                           | Mock Fallback             |
| -------------- | --------------------------------- | ------------------------- |
| vLLM           | BenchmarkEngine, InferenceBackend | Mock with fixed latencies |
| Kubernetes GPU | GPU Orchestration                 | Kind + CPU simulation     |
| S3             | ArtifactStore                     | Local MinIO               |
| Postgres       | ModelRegistry, ExperimentTracker  | SQLite                    |

## Failure Injection in Integration Tests

Selected integration tests inject controlled failures:

* Kill MergeEngine mid-merge and verify ArtifactStore cleanup.
* Drop network to ModelRegistry and verify retry with exponential backoff.
* Corrupt a checkpoint file and verify ModelLoader error handling.

See [Failure Injection](/testing/failure-injection) for the full catalog.

## Integration Points

* **CI**: integration tests run on every commit and nightly.
* **ExperimentTracker**: each scenario is logged as an experiment.
* **ArtifactStore**: stores test outputs, logs, and failure artifacts.
* **Performance Testing**: selected scenarios also collect latency and memory metrics. See [Performance Testing](/testing/performance-testing).