ExperimentTracker Event Stream and Responsibilities

Specification for the ExperimentTracker in EMEP, covering event stream semantics, immutable append-only logging, and integration with the evolution and evaluation pipeline.

The ExperimentTracker is the central audit and state log for all EMEP experiments. It records every significant event with full provenance, timestamps, and context. This page specifies the tracker responsibilities, event stream semantics, and integration points.

Responsibilities

The ExperimentTracker manages:

• Experiment lifecycle state transitions
• Candidate generation, validation, evaluation, and status changes
• Benchmark execution records
• Resource usage (GPU, memory, time)
• Configuration snapshots
• Error and failure states

It does not store model weights or dataset samples. Those are handled by the ArtifactStore and DatasetRegistry.

Event Stream Semantics

The tracker implements an immutable append-only event stream. Every event is a JSON record with the following fields:

Field	Type	Description
event_id	UUID	Unique identifier
experiment_id	string	Parent experiment
timestamp	ISO8601	Event time
event_type	string	Category: lifecycle, candidate, benchmark, error, resource
payload	object	Event-specific data
parent_event_id	UUID	For causal chains

Events are never modified or deleted. Corrections are appended as new events referencing the original. This guarantees a complete, tamper-evident history.

Event Types

Lifecycle Events

• experiment.created
• experiment.preparing
• experiment.running
• experiment.evaluating
• experiment.completed
• experiment.failed
• experiment.cancelled
• experiment.archived

Candidate Events

• candidate.generated
• candidate.validated
• candidate.evaluated
• candidate.status_changed

Benchmark Events

• benchmark.started
• benchmark.sample_completed
• benchmark.finished

Error Events

• error.validation_failed
• error.evaluation_failed
• error.merge_failed
• error.resource_exhausted

Resource Events

• resource.gpu_allocated
• resource.gpu_released
• resource.memory_peak

Append-Only Guarantee

The tracker storage layer enforces append-only semantics at the API level. There are no update or delete operations. Event streams are partitioned by experiment_id for query efficiency.

Query Interface

The ExperimentTracker exposes:

Tracker.append(event)
Tracker.get_stream(experiment_id, filters) -> List[Event]
Tracker.get_latest(experiment_id, event_type) -> Event
Tracker.get_experiment_summary(experiment_id) -> Summary

Queries return events in timestamp order. Filter options include event type, time range, and candidate ID.

Integration

Component	Events Logged
EvolutionEngine	Generation start, candidate generation, selection, termination
EvaluationEngine	Benchmark start, sample completion, score aggregation
ModelCompatibilityAnalyzer	Validation results, compatibility states
MergeEngine	Merge start, completion, failure
FitnessEngine	Fitness computation per candidate
ArtifactStore	Artifact upload, hash verification

Retention

Event streams are retained according to experiment state:

• COMPLETED: retained indefinitely
• FAILED: retained for 90 days, then archived to cold storage
• CANCELLED: retained for 30 days, then archived
• ARCHIVED: moved to cold storage immediately

Cold storage uses the same content-addressed system as the ArtifactStore. Event streams can be restored for audit or reproduction.

The ExperimentTracker is the source of truth for experiment state. Other components may cache state for performance, but the tracker record is authoritative in any conflict.