TRL, short for Transformers Reinforcement Learning, is a Hugging Face library for post-training transformer language models. It provides trainers for Supervised Fine-Tuning, Group Relative Policy Optimization, Direct Preference Optimization, reward modeling and more, with experimental online and offline methods and knowledge distillation trainers. It is integrated with the Hugging Face transformers library and documented on the Hugging Face site, with getting started, conceptual and how-to guides. As an open-source library it has no subscription.
TRL targets machine learning engineers, applied researchers and advanced students who already work in Python and want to align or specialize language models. The documentation walks through dataset formats, training logs, reducing memory use, speeding up training and distributing training across devices. Beginners can learn post-training with the smol course the maintainers link, while experts can explore experimental trainers. Anyone using Hugging Face models, datasets and Hub will find the workflow familiar.
A team can fine-tune an assistant model on curated conversations with SFTTrainer, then apply DPO using preference pairs to shape its behavior. Researchers train reasoning models with GRPO, drawing on posts such as the Open-R1 reproduction of DeepSeek-R1, and vision language teams align multimodal models. Engineers cut memory use with PEFT adapters, DeepSpeed and Liger Kernel integrations and scale runs across GPUs. Co-located vLLM generation speeds up online reinforcement learning, and the examples section offers community tutorials. Blog posts also cover Ulysses sequence parallelism for million-token contexts and weight syncing for very large models.
