> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# Performance Validation: Latency and Throughput Gates

> PerformanceValidation measures latency, throughput, memory, tokens per second, and TTFT. Statistical protocol, regression gate, and flowchart specified.

PerformanceValidation measures the runtime behavior of a deployed model. This page specifies the metrics, the measurement protocol, the statistical criteria for acceptance, and the regression gate that prevents degraded models from reaching production.

## Metrics

| Metric     | Definition                             | Unit         |
| ---------- | -------------------------------------- | ------------ |
| Latency    | End-to-end request duration            | milliseconds |
| Throughput | Total tokens generated per second      | tokens/sec   |
| Memory     | Peak GPU or RAM usage during inference | GiB          |
| Tokens/sec | Generation speed per request           | tokens/sec   |
| TTFT       | Time to first token                    | milliseconds |

All metrics are collected under controlled load conditions to ensure comparability across runs.

## Measurement Flowchart

The validation pipeline runs benchmarks, collects samples, computes statistics, and applies the regression gate.

```mermaid theme={null}
flowchart TD
    START([START]) --> INPUT[INPUT: Model, Backend, Load Profile, Baseline]
    INPUT --> WARMUP[Warmup: 10 requests discard]
    WARMUP --> BENCHMARK[Benchmark: Minimum N samples]
    BENCHMARK --> STATS[Compute mean, std, bootstrap CI]
    STATS --> REGRESS{Regression vs Baseline?}
    REGRESS -->|YES| REJECT[Reject: Flag Regression]
    REGRESS -->|NO| ACCEPT[Accept: Pass Gate]
    REJECT --> END_FAIL([END: FAIL])
    ACCEPT --> END_PASS([END: PASS])
```

## Statistical Protocol

PerformanceValidation uses a bootstrap confidence interval protocol to reduce noise and false positives.

1. **Minimum sample size**: 100 requests per load profile. (Engineering assumption)
2. **Warmup**: 10 requests executed and discarded before measurement.
3. **Bootstrap**: 10,000 resamples with replacement to compute a 95% CI for the mean of each metric.
4. **Comparison**: The new model passes if the upper bound of its 95% CI is below the baseline mean plus the regression threshold.

| Metric     | Regression Threshold | Direction       |
| ---------- | -------------------- | --------------- |
| Latency    | +10%                 | Higher is worse |
| Throughput | -10%                 | Lower is worse  |
| Memory     | +10%                 | Higher is worse |
| Tokens/sec | -10%                 | Lower is worse  |
| TTFT       | +10%                 | Higher is worse |

<Note>
  Threshold values are engineering assumptions. Final values will be calibrated in OQ-006 after reference benchmark runs.
</Note>

## Regression Gate

The regression gate compares the new candidate against the current production baseline. If any metric exceeds its regression threshold, the candidate fails and cannot proceed to canary or production.

* Failure is recorded in [ExperimentTracker](/tracking/experiment-tracking) with full sample distributions.
* The candidate status is set to REGRESSION.
* [DeploymentManager](/deployment/deployment-specification) blocks promotion.

## Load Profiles

| Profile  | Concurrent Requests | Request Length | Use Case                    |
| -------- | ------------------- | -------------- | --------------------------- |
| Single   | 1                   | 512 tokens     | Latency baseline, TTFT      |
| Moderate | 8                   | 1024 tokens    | Throughput, memory pressure |
| Heavy    | 32                  | 2048 tokens    | Saturation, stability       |

Every candidate is tested against all three profiles. Results are stored in [ArtifactStore](/tracking/artifact-registry) and linked to the model lineage.

## Integration

PerformanceValidation is invoked by [DeploymentManager](/deployment/deployment-specification) during staging and by [Offline Deployment](/deployment/offline-deployment) during acceptance testing. It reads models from [ArtifactStore](/tracking/artifact-registry) and reports results to [Observability](/architecture/observability).