> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# EMEP Performance Testing and Metrics

> Latency, throughput, memory, tokens per second, and TTFT performance tests for EMEP with cross-links to deployment validation.

Performance testing measures how EMEP components behave under load. Tests cover inference latency, throughput, memory consumption, tokens per second, and time to first token (TTFT). Results are compared against thresholds defined in [Performance Validation](/deployment/performance-validation).

## Test Categories

### Latency

Measure end-to-end latency for:

* Model registration and validation.
* Compatibility analysis.
* Merge execution.
* Benchmark evaluation.
* Candidate promotion.

Latency is reported as p50, p95, and p99.

### Throughput

Measure jobs completed per minute for:

* Concurrent merge jobs.
* Concurrent benchmark runs.
* Concurrent evolution generations.

Throughput tests use a fixed GPU pool and measure saturation behavior.

### Memory

Measure peak memory usage for:

* Model loading (full precision and quantized).
* Merge execution (intermediate tensors).
* Benchmark inference (batch size sweep).

Memory is reported in GiB and as a fraction of available GPU memory.

### Tokens Per Second

Measure generation throughput for InferenceBackend across batch sizes and sequence lengths. Report prefill and decode phases separately.

### Time to First Token (TTFT)

Measure latency from request submission to first generated token. TTFT is critical for interactive use cases.

## Test Matrix

| Metric     | Batch Size  | Sequence Length | Dtype     | Backend |
| ---------- | ----------- | --------------- | --------- | ------- |
| Latency    | 1           | 512             | bf16      | vLLM    |
| Throughput | 1, 4, 8, 16 | 512             | bf16      | vLLM    |
| Memory     | 1, 4, 8     | 512, 2048       | bf16, fp8 | vLLM    |
| Tokens/sec | 1, 4, 8     | 512, 2048       | bf16      | vLLM    |
| TTFT       | 1           | 512, 2048       | bf16      | vLLM    |

## Load Testing

Load tests simulate multiple concurrent users:

1. Ramp up requests over 60 seconds.
2. Hold peak load for 300 seconds.
3. Ramp down over 60 seconds.
4. Measure error rate, latency percentiles, and resource utilization.

## Baselines and Thresholds

Performance baselines are established on a reference hardware configuration. Deviations greater than 10% relative trigger investigation. Thresholds are documented in [Performance Validation](/deployment/performance-validation).

## Integration Points

* **GPU Orchestration**: performance tests validate GPU scheduling and memory limits. See [GPU Orchestration](/architecture/gpu-orchestration).
* **Inference Backend**: tests exercise InferenceBackend under load. See [Inference](/deployment/inference).
* **Deployment**: performance results feed into deployment decisions. See [Performance Validation](/deployment/performance-validation).
* **CI**: performance tests run weekly and on release candidates.
* **Monitoring**: metrics are exported to the observability stack. See [Monitoring](/operations/monitoring).