Staff-Recommended Papers for NLP/LLM Course

This is a curated list of papers recommended by the course staff. Feel free to explore!

2025

• Recursive Language Models - Zhang et al. 2025.
• Learning to Reason in 4D: Dynamic Spatial Understanding for Vision Language Models - Zhou et al. 2025.
• Latent Collaboration in Multi-Agent Systems - Zou et al. 2025.
• Equivalence of Context and Parameter Updates in Modern Transformer Blocks - Goldwaser et al. 2025.
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics - Balestriero et al. 2025.
• Diffusion Language Models are Super Data Learners - Ni et al. 2025.
• Detecting Data Contamination in LLMs via In-Context Learning - Zawalski et al. 2025.
• The Art of Scaling Reinforcement Learning Compute for LLMs - Khatri et al. 2025.
• Diffusion Language Models Know the Answer Before Decoding - Li et al. 2025.
• AICrypto: Evaluating Cryptography Capabilities of Large Language Models - Wang et al. 2025.
• Spurious Rewards: Rethinking Training Signals in RLVR - Shao et al. 2025.
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning - Wang et al. 2025.
• Superposition Yields Robust Neural Scaling - Liu et al. 2025.
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free - Qiu et al. 2025.
• Absolute Zero: Reinforced Self-play Reasoning with Zero Data - Zhao et al.
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example - Wang et al. 2025.
• TTRL: Test-Time Reinforcement Learning - Zuo et al. 2025.
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? - Yue et al. 2025.
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning - Wang et al. 2025.
• AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence - Liu et al. 2025.
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention - Yuan et al. 2025.
• Large Language Diffusion Models - Nie et al. 2025.
• Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions - Kim et al. 2025.
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training - Chu et al. 2025.
• Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps - Ma et al. 2025.

2024

• Training Large Language Models to Reason in a Continuous Latent Space - Hao et al. 2024.
• AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making - Huang et al. 2024.
• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models - Fang et al. 2024.
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters - Snell et al. 2024.
• Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process - Ye et al. 2024.
• From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attacks - Zhang et al. 2024.
• Refusal in Language Models Is Mediated by a Single Direction - Arditi et al. 2024.
• AI Sandbagging: Language Models can Strategically Underperform on Evaluations - Weij et al. 2024.
• Language Models Resist Alignment: Evidence From Data Compression - Ji et al. 2024.
• Safety Alignment Should Be Made More Than Just a Few Tokens Deep - Qi et al. 2024.
• Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo - Zhao et al. 2024.
• Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study - Xu et al. 2024.
• Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards - Hwang et al. 2024.
• PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models - Meng et al. 2024.
• MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs - Yu et al. 2024.
• The Unreasonable Ineffectiveness of the Deeper Layers - Gromov et al. 2024.
• RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval - Wen et al. 2024.
• Massive Activations in Large Language Models - Sun et al. 2024.
• No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices - Pang et al. 2024.
• Debating with More Persuasive LLMs Leads to More Truthful Answers - Khan et al. 2024.
• TOFU: A Task of Fictitious Unlearning for LLMs - Maini et al. 2024.

2023

• The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation - Xu et al. 2023.
• Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game - Xu et al. 2023.
• Controlled Decoding from Language Models - Mudgal et al. 2023.
• Detecting Pretraining Data from Large Language Models - Shi et al. 2023.
• EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities - Li et al. 2023.
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! - Qi et al. 2023.
• YaRN: Efficient Context Window Extension of Large Language Models - Peng et al. 2023.
• ProAgent: Building Proactive Cooperative Agents with Large Language Models - Zhang et al. 2023.
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model - Rafailov et al. 2023.
• Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective - Feng et al. 2023.
• Can Language Models Solve Graph Problems in Natural Language? - Wang et al. 2023.
• Visual Instruction Tuning - Liu et al. 2023.
• Generative Agents: Interactive Simulacra of Human Behavior - Park et al. 2023.
• MarioGPT: Open-Ended Text2Level Generation through Large Language Models - Sudhakaran et al. 2023.

2022

• Demonstrate-Search-Predict: Composing Retrieval and Language Models for Knowledge-Intensive NLP - Khattab et al. 2022.
• On the Blind Spots of Model-Based Evaluation Metrics for Text Generation - He et al. 2022.
• Fast Inference from Transformers via Speculative Decoding - Leviathan et al. 2022.
• Mass-Editing Memory in a Transformer - Meng et al. 2022.

2021

• Towards a Unified View of Parameter-Efficient Transfer Learning - He et al. 2021.
• RoFormer: Enhanced Transformer with Rotary Position Embedding - Su et al. 2021.

2020

• Transformer Feed-Forward Layers Are Key-Value Memories - Geva et al. 2020.
• A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation - Nadeem et al. 2020.
• On Layer Normalization in the Transformer Architecture - Xiong et al. 2020.
• Scaling Laws for Neural Language Models - Kaplan et al. 2020.