PEFT Overview for EMEP Fine-Tuning

Parameter-Efficient Fine-Tuning methods in EMEP including LoRA, prefix-tuning, prompt-tuning, and trade-offs versus full fine-tuning.

Parameter-Efficient Fine-Tuning (PEFT) reduces trainable parameters by updating small adapter modules instead of full weights. EMEP uses PEFT to produce specialist inputs for MergeEngine without the cost of full fine-tuning. The PEFT survey (Han et al. 2024) catalogs the adapter families EMEP supports.

Adapter Families

EMEP implements three adapter families. Each adds parameters to a frozen base model.

LoRA

LoRA (Hu et al. 2021) decomposes weight updates into low-rank matrices A and B. For a weight matrix W, the update is delta W = A B^T with rank r. Only A and B are trained. LoRA is the default PEFT method in EMEP because it merges cleanly into base weights and has low memory overhead.

See LoRA for math, hyperparameters, and the training flowchart.

Prefix-Tuning

Prefix-tuning prepends trainable prefix vectors to keys and values in attention layers. The base model remains frozen. Prefix-tuning works well for classification and generation tasks but is less flexible for merging than LoRA because prefix vectors do not compose with base weights in the same way.

Prompt-Tuning

Prompt-tuning trains soft prompt embeddings at the input layer. It is the most parameter-efficient of the three but also the most task-specific. Prompt-tuned models may require task-specific prompts at inference time, which complicates deployment in InferenceBackend.

Trade-Offs

Method	Trainable Params	Mergeability	Forgetting Risk	Inference Cost
Full FT	100%	N/A (replaces weights)	High	Same as base
LoRA	less than 1% (typical)	High (mergeable)	Low (partial)	Same as base after merge
Prefix-tuning	about 0.1%	Low	Low	Same as base + prefix length
Prompt-tuning	about 0.01%	Very low	Very low	Same as base + prompt length

When to Prefer PEFT Over Full FT

Prefer PEFT when:

• The base model is large and GPU memory is constrained.
• You need multiple specialist variants of the same base model.
• The target task is close to the base model’s pretraining distribution.
• You intend to merge the fine-tuned model with others in MergeEngine.

Use full fine-tuning only when adapters fail to reach target accuracy or when the task distribution diverges significantly from pretraining.

Integration with MergeEngine

LoRA adapters can be merged into base weights before entering MergeEngine, or kept as separate adapter artifacts. MergeEngine treats a merged LoRA model as a standard model with VALIDATED status. Prefix-tuned and prompt-tuned models require conversion or are treated as separate model variants with limited merge compatibility.

Unsupported Claim

Prefix-tuning and prompt-tuning are not guaranteed to be compatible with all MergeStrategy implementations. Compatibility is determined at runtime by ModelCompatibilityAnalyzer.