Backup Strategy and Restore Procedures - EMEP

EMEP state spans model weights, experiment metadata, datasets, evaluation metrics, and cryptographic keys. Losing any of these breaks reproducibility or blocks deployment. This page defines what to back up, how often, for how long, and how to restore.

Backup Targets

Target	Contents	Storage Class
Model artifacts	Trained and merged model weights, checkpoints	Hot for 30 days, then cold tier
Experiment DB	Experiment state, parameters, lineage	Hot, replicated
Dataset store	Training and evaluation datasets	Hot for active, cold for archived
Metric store	Evaluation results, benchmark scores	Hot, time-series optimized
Registry metadata	Model lifecycle, compatibility records	Hot, replicated
Signing keys	Root and intermediate key backups (offline)	Offline HSM + physical vault

Schedule

Target	Frequency	Method
Model artifacts	Continuous (write-on-ingest)	Replicated object store
Experiment DB	Hourly	Incremental snapshot
Dataset store	Daily (delta)	Incremental sync
Metric store	Continuous	Time-series replication
Registry metadata	Hourly	Transaction log shipping
Signing keys	On change	Manual HSM export ceremony

Retention

Target	Retention	Rationale
Model artifacts	90 days hot, 2 years cold	Compliance + reproducibility
Experiment DB	2 years	Audit trail
Dataset store	Per dataset policy; minimum 1 year	License obligations
Metric store	1 year	Trend analysis
Registry metadata	7 years	Enterprise audit requirement
Signing keys	Permanent (archived)	Historical signature verification

Restore Drill Flow

Restore drills run quarterly on a non-production environment. A failed drill triggers a backup integrity investigation and a re-run within 7 days.

Signing Key Restore

Signing key backup uses Shamir secret sharing across multiple physical locations. Restoring a key requires a quorum of shares and an HSM import ceremony. The restored key is verified by signing a test artifact and validating the signature against the public key on record.

Cross-Links