> ## Documentation Index
> Fetch the complete documentation index at: https://doc.blueapi.ir/llms.txt
> Use this file to discover all available pages before exploring further.

# Backup Strategy and Restore Procedures

> Backup targets, schedules, retention policies, and restore drill procedures for model artifacts, experiment databases, datasets, metrics, and signing keys.

EMEP state spans model weights, experiment metadata, datasets, evaluation metrics, and cryptographic keys. Losing any of these breaks reproducibility or blocks deployment. This page defines what to back up, how often, for how long, and how to restore.

## Backup Targets

| Target            | Contents                                      | Storage Class                     |
| ----------------- | --------------------------------------------- | --------------------------------- |
| Model artifacts   | Trained and merged model weights, checkpoints | Hot for 30 days, then cold tier   |
| Experiment DB     | Experiment state, parameters, lineage         | Hot, replicated                   |
| Dataset store     | Training and evaluation datasets              | Hot for active, cold for archived |
| Metric store      | Evaluation results, benchmark scores          | Hot, time-series optimized        |
| Registry metadata | Model lifecycle, compatibility records        | Hot, replicated                   |
| Signing keys      | Root and intermediate key backups (offline)   | Offline HSM + physical vault      |

## Schedule

| Target            | Frequency                    | Method                     |
| ----------------- | ---------------------------- | -------------------------- |
| Model artifacts   | Continuous (write-on-ingest) | Replicated object store    |
| Experiment DB     | Hourly                       | Incremental snapshot       |
| Dataset store     | Daily (delta)                | Incremental sync           |
| Metric store      | Continuous                   | Time-series replication    |
| Registry metadata | Hourly                       | Transaction log shipping   |
| Signing keys      | On change                    | Manual HSM export ceremony |

## Retention

| Target            | Retention                          | Rationale                         |
| ----------------- | ---------------------------------- | --------------------------------- |
| Model artifacts   | 90 days hot, 2 years cold          | Compliance + reproducibility      |
| Experiment DB     | 2 years                            | Audit trail                       |
| Dataset store     | Per dataset policy; minimum 1 year | License obligations               |
| Metric store      | 1 year                             | Trend analysis                    |
| Registry metadata | 7 years                            | Enterprise audit requirement      |
| Signing keys      | Permanent (archived)               | Historical signature verification |

## Restore Drill Flow

```mermaid theme={null}
flowchart LR
    START([Restore Drill]) --> SELECT[Select Target + Point-in-Time]
    SELECT --> LOCATE[Locate Backup]
    LOCATE --> VERIFY[Verify Integrity]
    VERIFY --> VALID{Valid?}
    VALID -->|NO| ESCALATE[Escalate: Corrupt Backup]
    VALID -->|YES| RESTORE[Restore to Isolated Environment]
    RESTORE --> TEST[Run Validation Tests]
    TEST --> PASS{Pass?}
    PASS -->|NO| DIAGNOSE[Diagnose: Data or Process Issue]
    PASS -->|YES| RECORD[Record Drill Result]
    RECORD --> END([End])
```

Restore drills run quarterly on a non-production environment. A failed drill triggers a backup integrity investigation and a re-run within 7 days.

## Signing Key Restore

Signing key backup uses Shamir secret sharing across multiple physical locations. Restoring a key requires a quorum of shares and an HSM import ceremony. The restored key is verified by signing a test artifact and validating the signature against the public key on record.

## Cross-Links

* [Disaster Recovery](/operations/disaster-recovery)
* [Storage Operations](/operations/storage-operations)
* [Artifact Signing](/security/artifact-signing)