数据与研究

simpo-training

Name: simpo-training
Author: orchestra-research

orchestra-research/ai-research-skills

Simple Preference Optimization for LLM alignment. Reference-free alternative to DPO with better performance (+6.4 points on AlpacaEval 2.0). No reference model needed, more efficient than DPO. Use for preference alignment when want simpler, faster training than DPO/PPO.

966 次安装
1.4万星标
收录于 2026.01.21

安装命令

$ npx skills add https://github.com/orchestra-research/ai-research-skills --skill simpo-training

SKILL.md

SimPO - Simple Preference Optimization

Quick start

SimPO is a reference-free preference optimization method that outperforms DPO without needing a reference model.

Installation:

# Create environment
conda create -n simpo python=3.10 && conda activate simpo

# Install PyTorch 2.2.2
# Visit: https://pytorch.org/get-started/locally/

# Install alignment-handbook
git clone https://github.com/huggingface/alignment-handbook.git
cd alignment-handbook
python -m pip install .

SKILL.md

SimPO - Simple Preference Optimization

Quick start

相关 Skills