Direct Preference Optimization (DPO): Your Language Model is Secretly a Reward Model Published on May 29, 2023 Updated on Jan 15, 2026 Reinforcement Learning Reinforcement Learning One minuteContents TL;DRMotivations & InnovationsApproachModelTraining RecipeData RecipeExperiments arXiv 2305.18290 TL;DRMotivations & InnovationsApproachModelTraining RecipeData RecipeExperimentsRelated PostsInstructGPT: Training language models to follow instructions with human feedback 2022-03-04PPO: Proximal Policy Optimization Algorithms 2017-07-20
