Staff-Recommended Papers for NLP/LLM Course

This is a curated list of papers recommended by the course staff. Feel free to explore!

2025

• AICrypto: Evaluating Cryptography Capabilities of Large Language Models - Wang et al. 2025.
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? - Yue et al. 2025.
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention - Yuan et al. 2025.
• Large Language Diffusion Models - Nie et al. 2025.

2024

• AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making - Huang et al. 2024.
• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models - Fang et al. 2024.
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters - Snell et al. 2024.
• Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process - Ye et al. 2024.
• From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attacks - Zhang et al. 2024.
• Refusal in Language Models Is Mediated by a Single Direction - Arditi et al. 2024.
• Safety Alignment Should Be Made More Than Just a Few Tokens Deep - Qi et al. 2024.
• Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo - Zhao et al. 2024.
• Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study - Xu et al. 2024.
• Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards - Hwang et al. 2024.
• PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models - Meng et al. 2024.
• MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs - Yu et al. 2024.
• RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval - Wen et al. 2024.
• Debating with More Persuasive LLMs Leads to More Truthful Answers - Khan et al. 2024.
• TOFU: A Task of Fictitious Unlearning for LLMs - Maini et al. 2024.

2023

• The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation - Xu et al. 2023.
• Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game - Xu et al. 2023.
• Controlled Decoding from Language Models - Mudgal et al. 2023.
• Detecting Pretraining Data from Large Language Models - Shi et al. 2023.
• EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities - Li et al. 2023.
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! - Qi et al. 2023.
• YaRN: Efficient Context Window Extension of Large Language Models - Peng et al. 2023.
• ProAgent: Building Proactive Cooperative Agents with Large Language Models - Zhang et al. 2023.
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model - Rafailov et al. 2023.
• Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective - Feng et al. 2023.
• Can Language Models Solve Graph Problems in Natural Language? - Wang et al. 2023.
• Visual Instruction Tuning - Liu et al. 2023.
• Generative Agents: Interactive Simulacra of Human Behavior - Park et al. 2023.
• MarioGPT: Open-Ended Text2Level Generation through Large Language Models - Sudhakaran et al. 2023.

2022

• Demonstrate-Search-Predict: Composing Retrieval and Language Models for Knowledge-Intensive NLP - Khattab et al. 2022.
• On the Blind Spots of Model-Based Evaluation Metrics for Text Generation - He et al. 2022.
• Fast Inference from Transformers via Speculative Decoding - Leviathan et al. 2022.

2021

• RoFormer: Enhanced Transformer with Rotary Position Embedding - Su et al. 2021.

2020

• A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation - Nadeem et al. 2020.
• On Layer Normalization in the Transformer Architecture - Xiong et al. 2020.
• Scaling Laws for Neural Language Models - Kaplan et al. 2020.