Improving how LLMs learn and respond
We’re also advancing how LLMs train, learn, and respond to users, improving performance and efficiency on several fronts.
With larger context windows, LLMs can now learn from potentially thousands of examples at once — known as many-shot in-context learning (ICL). This process boosts model performance on tasks like math, translation, and reasoning, but often requires high-quality, human-generated data. To make training more cost-effective, we explore methods to adapt many-shot ICL that reduce reliance on manually curated data. There is so much data available for training language models, the main constraint for teams building them becomes the available compute. We address an important question: with a fixed compute budget, how do you choose the right model size to achieve the best results?
Another innovative approach, which we call Time-Reversed Language Models (TRLM), explores pretraining and finetuning an LLM to work in reverse. When given traditional LLM responses as input, a TRLM generates queries that might have produced those responses. When paired with a traditional LLM, this method not only helps ensure responses follow user instructions better, but also improves the generation of citations for summarized text, and enhances safety filters against harmful content.
Curating high-quality data is vital for training large AI models, but manual curation is difficult at scale. To address this, our Joint Example Selection (JEST) algorithm optimizes training by identifying the most learnable data within larger batches, enabling up to 13× fewer training rounds and 10× less computation, outperforming state-of-the-art multimodal pretraining baselines.
Planning tasks are another challenge for AI, particularly in stochastic environments, where outcomes are influenced by randomness or uncertainty. Researchers use various inference types for planning, but there’s no consistent approach. We demonstrate that planning itself can be viewed as a distinct type of probabilistic inference and propose a framework for ranking different inference techniques based on their planning effectiveness.