Storage Operations: Cleanup, Retention, and Tiering

Artifact cleanup, retention enforcement, cold tier migration, capacity thresholds, and the automated storage lifecycle flow for model weights and experiment data.

Model weights, checkpoints, evaluation artifacts, and datasets consume terabytes of object storage. Without automated lifecycle management, costs grow and performance degrades. This page defines the storage operations that keep the artifact store efficient and within budget.

Lifecycle Flow

Retention Rules

Artifact Type	Hot Tier	Cold Tier	Archive	Delete
Model weights (RELEASED)	30 days	1 year	2 years	After archive expiry
Model weights (DEPRECATED)	7 days	90 days	1 year	After archive expiry
Merge candidates (PASS)	14 days	90 days	1 year	After archive expiry
Merge candidates (FAIL)	7 days	30 days	N/A	After cold expiry
Checkpoints	Duration of experiment	30 days	90 days	After archive expiry
Evaluation artifacts	30 days	90 days	1 year	After archive expiry
Dataset copies	Per dataset policy	Per policy	Per policy	Per policy
Logs and metrics	7 days	30 days	1 year	After archive expiry

Capacity Thresholds

Threshold	Action
70%	Alert P4: capacity trending
80%	Alert P3: trigger accelerated cleanup of FAIL candidates
90%	Alert P2: block new large uploads; emergency cleanup of deprecated models
95%	Alert P1: read-only mode for non-critical operations

Cleanup Procedures

Automated Cleanup

A scheduled job scans artifacts daily. It classifies each artifact by:

• Lifecycle state (REGISTERED, CANDIDATE, EVALUATED, RELEASED, DEPRECATED, ARCHIVED)
• Last access timestamp
• Size and tiering cost

The job applies the retention table and generates a manifest of actions. The manifest is reviewed by policy: P4 and below actions are automatic; P3 and above require operator confirmation.

Manual Cleanup

Operators may trigger emergency cleanup via API or CLI. Emergency cleanup targets:

• FAIL candidates older than 7 days
• DEPRECATED models older than 30 days
• Orphaned checkpoints with no parent experiment

All deletions are logged with artifact ID, size, reason, and operator identity. Deleted artifacts are soft-deleted for 7 days before physical removal.

Cold Tier Migration

Migration to cold storage is asynchronous and verifies integrity before deleting the hot copy. The registry metadata is updated with the new tier location. A failed migration retries once and alerts if it fails again.

Cross-Links

• Backup Strategy
• Monitoring
• Alerting