DPO: Direct Preference Optimization
Learns directly from pairs of human preferences, skipping the separate reward model and reinforcement learning of RLHF.
Why it matters
Simple and stable, it became one of the most widely used alignment methods for open models; Llama 3 used it in post-training.
Built on
Led to
Source: arxiv.org ↗