Optional RAG Architecture: Downstream Retrieval Pipeline

Retrieval-Augmented Generation pipeline for EMEP. Ingestion, embeddings, retrieval, reranking, and citation. Downstream and optional.

Retrieval-Augmented Generation (RAG) is a downstream, optional capability for EMEP. It does not modify model weights and does not replace model creation, merging, or evolutionary optimization. This page specifies the full RAG pipeline, its components, and its relationship to the core platform.

RAG is downstream and optional. It does not modify weights. It does not replace model creation or merging. Core EMEP functionality does not depend on RAG. See Project Constitution for the scope boundary.

Pipeline Overview

The RAG pipeline augments model inference by retrieving relevant documents and injecting them into the prompt context. The pipeline is independent of MergeEngine, EvolutionEngine, and EvaluationEngine.

Pipeline Stages

Ingestion

Documents enter the pipeline from DatasetRegistry or external connectors. Supported formats include plain text, Markdown, PDF, and HTML. Ingestion extracts text and preserves structural metadata such as headings, tables, and lists.

Chunking

Documents are split into chunks for embedding. Chunking strategies include:

• Fixed token count with overlap
• Semantic boundaries (paragraphs, sections)
• Hierarchical chunking (parent document + child chunks)

Chunk size and overlap are configurable per dataset.

Embeddings

Each chunk is converted to a dense vector using an embedding model. The embedding model is loaded via InferenceBackend and runs independently of the generation model. Embeddings are stored with chunk metadata and provenance links.

Vector Store

Embeddings are indexed in a vector store supporting approximate nearest neighbor search. The store is queryable by vector similarity and by metadata filters (source, date, document type).

Retrieval

At inference time, the user query is embedded and used to retrieve the top-K most similar chunks. K is configurable per use case. Retrieved chunks are passed to the reranking stage.

Reranking

A cross-encoder or lightweight reranker reorders retrieved chunks by relevance to the specific query. Reranking improves precision over pure vector similarity.

Context Assembly

Selected chunks are assembled into a context string respecting the generation model’s context window. Assembly strategies include:

• Concatenation with separator tokens
• Summarization if total chunk length exceeds the window
• Priority ordering by reranker score

Citation

Every generated claim that relies on retrieved context is annotated with a citation linking back to the source chunk and document. Citations are exposed in the API response for verification and audit.

Knowledge Updates

New documents can be ingested and embedded without retraining or merging the generation model. Knowledge updates are versioned and tracked in ExperimentTracker. Rollback to a previous knowledge version is supported.

Integration Boundaries

Component	RAG Interaction	Core EMEP Independence
InferenceBackend	Loads embedding and generation models	Core inference does not require RAG
DatasetRegistry	Provides document sources	Core datasets are separate
ExperimentTracker	Logs RAG experiments	Core experiments do not depend on RAG
ModelRegistry	May register embedding models	Core model lifecycle is unchanged

Future Scope

RAG may be extended with:

• Hybrid search (vector + keyword)
• Multi-modal retrieval (image, audio)
• Feedback loops for chunk relevance scoring

These extensions are future goals and not part of the initial core.