Complete Experiment Record Schema

Specification for the full experiment record schema in EMEP, including all fields for reproducibility, provenance, and result storage with JSON example.

The experiment record is the complete snapshot of an EMEP experiment. It combines configuration, execution context, results, and provenance into a single queryable document. This page specifies every field, its semantics, and a full JSON example.

Field Definitions

Field	Type	Required	Description
experiment_id	string	Yes	Unique identifier
experiment_name	string	Yes	Human-readable name
status	string	Yes	CREATED, PREPARING, RUNNING, EVALUATING, COMPLETED, FAILED, CANCELLED, ARCHIVED
created_at	ISO8601	Yes	Creation timestamp
completed_at	ISO8601	No	Completion timestamp
model_id	string	No	Final model ID if experiment produced one
parent_models	List[string]	Yes	Model IDs used as parents
model_revisions	Dict[string, string]	Yes	Model ID to version hash mapping
merge_method	string	Yes	SLERP, TIES, DARE, Task Arithmetic, Franken-Merge
merge_parameters	object	Yes	Genome parameters used
dataset_version	string	Yes	DatasetRegistry version tag
benchmark_version	string	Yes	BenchmarkCatalog version tag
gpu_type	string	Yes	GPU hardware identifier
gpu_count	int	Yes	Number of GPUs allocated
cuda_version	string	Yes	CUDA runtime version
pytorch_version	string	Yes	PyTorch version
framework_versions	Dict[string, string]	Yes	Other dependency versions
git_commit	string	Yes	EMEP repository commit hash
random_seed	int	Yes	Master random seed
seeds	List[int]	No	Per-benchmark seeds
timestamp	ISO8601	Yes	Record timestamp
model_hash	string	No	SHA-256 of final model weights
artifact_hash	string	No	SHA-256 of artifact archive
score	float	No	Final aggregate score
scores	Dict[string, float]	No	Per-benchmark scores
latency_ms	float	No	Mean inference latency
throughput	float	No	Tokens per second
memory_usage_gb	float	No	Peak memory usage
failure_state	object	No	Error details if status is FAILED
metadata	object	No	Arbitrary additional data

JSON Example

{
  "experiment_id": "exp_2024_001",
  "experiment_name": "Llama 7B Persian Merge v3",
  "status": "COMPLETED",
  "created_at": "2024-01-15T09:00:00Z",
  "completed_at": "2024-01-15T14:30:00Z",
  "model_id": "model_llama_7b_persian_v3",
  "parent_models": [
    "model_llama_7b_base_v3",
    "model_llama_7b_instruct_v2"
  ],
  "model_revisions": {
    "model_llama_7b_base_v3": "sha256:a1b2c3...",
    "model_llama_7b_instruct_v2": "sha256:d4e5f6..."
  },
  "merge_method": "TIES",
  "merge_parameters": {
    "alpha": [0.6, 0.4],
    "density": 0.7,
    "per_layer_alpha": {}
  },
  "dataset_version": "datasets_v2024.1",
  "benchmark_version": "benchmarks_v2024.2",
  "gpu_type": "NVIDIA A100",
  "gpu_count": 4,
  "cuda_version": "12.1",
  "pytorch_version": "2.1.0",
  "framework_versions": {
    "transformers": "4.36.0",
    "datasets": "2.14.0",
    "accelerate": "0.25.0"
  },
  "git_commit": "abc123def456",
  "random_seed": 42,
  "seeds": [42, 123, 456],
  "timestamp": "2024-01-15T14:30:00Z",
  "model_hash": "sha256:789abc...",
  "artifact_hash": "sha256:012def...",
  "score": 0.8234,
  "scores": {
    "persian_reading_comp_v1": 0.81,
    "persian_cloze_v1": 0.79,
    "english_mmlu_v1": 0.85,
    "safety_v1": 0.92
  },
  "latency_ms": 45.2,
  "throughput": 1250.0,
  "memory_usage_gb": 28.4,
  "failure_state": null,
  "metadata": {
    "user": "researcher_1",
    "project": "persian_llm",
    "cost_estimate_usd": 120.0
  }
}

Schema Versioning

The experiment record schema is versioned independently of EMEP releases. Schema version is stored in the schema_version field. Backward compatibility is maintained: new fields are optional, removed fields are deprecated but accepted.

Query Patterns

Common queries against experiment records:

• Find all experiments using a specific parent model
• Compare scores across merge methods
• Filter by GPU type and memory usage
• Identify experiments with failure states
• Reproduce an experiment from its full record

Integration

The ExperimentTracker writes the full record at experiment completion. The record is stored in the ArtifactStore as a content-addressed object. The ModelRegistry references the experiment record from the model’s provenance chain.

The experiment record contains all information needed to reproduce an experiment except the model weights themselves. Weights are referenced by model_hash and stored in the ArtifactStore.