Staff-Recommended Materials for NLP/LLM Course

This is a curated list of materials recommended by the course staff. Feel free to explore!

Papers

2026

• Stealing Reasoning Traces from Proprietary LLM APIs - Panfilov et al. 2026.
• Data Difficulty and the Generalization–Extrapolation Tradeoff in LLM Fine-Tuning - Liu et al. 2026.
• PostTrainBench: Can LLM Agents Automate LLM Post-Training? - Rank et al. 2026.
• The Flexibility Trap: Rethinking the Value of Arbitrary Order in Diffusion Language Models - Ni et al. 2026.

2025

• Recursive Language Models - Zhang et al. 2025.
• Learning to Reason in 4D: Dynamic Spatial Understanding for Vision Language Models - Zhou et al. 2025.
• Latent Collaboration in Multi-Agent Systems - Zou et al. 2025.
• Equivalence of Context and Parameter Updates in Modern Transformer Blocks - Goldwaser et al. 2025.
• LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics - Balestriero et al. 2025.
• Diffusion Language Models are Super Data Learners - Ni et al. 2025.
• Detecting Data Contamination in LLMs via In-Context Learning - Zawalski et al. 2025.
• Scaling Latent Reasoning via Looped Language Models - Zhu et al. 2025.
• The Art of Scaling Reinforcement Learning Compute for LLMs - Khatri et al. 2025.
• Fantastic Pretraining Optimizers and Where to Find Them - Wen et al. 2025.
• Diffusion Language Models Know the Answer Before Decoding - Li et al. 2025.
• AICrypto: Evaluating Cryptography Capabilities of Large Language Models - Wang et al. 2025.
• Spurious Rewards: Rethinking Training Signals in RLVR - Shao et al. 2025.
• Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning - Wang et al. 2025.
• Superposition Yields Robust Neural Scaling - Liu et al. 2025.
• Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free - Qiu et al. 2025.
• ZeroSearch: Incentivize the Search Capability of LLMs without Searching - Sun et al. 2025.
• Absolute Zero: Reinforced Self-play Reasoning with Zero Data - Zhao et al.
• MF-LLM: Simulating Population Decision Dynamics via a Mean-Field Large Language Model Framework - Mi et al. 2025.
• Reinforcement Learning for Reasoning in Large Language Models with One Training Example - Wang et al. 2025.
• TTRL: Test-Time Reinforcement Learning - Zuo et al. 2025.
• Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? - Yue et al. 2025.
• One Model to Rig Them All: Diverse Skeleton Rigging with UniRig - Zhang et al. 2025.
• Kimina-Prover Preview: Towards Large Formal Reasoning Models with Reinforcement Learning - Wang et al. 2025.
• AdaptiveStep: Automatically Dividing Reasoning Step through Model Confidence - Liu et al. 2025.
• Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention - Yuan et al. 2025.
• Large Language Diffusion Models - Nie et al. 2025.
• Train for the Worst, Plan for the Best: Understanding Token Ordering in Masked Diffusions - Kim et al. 2025.
• SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training - Chu et al. 2025.
• Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps - Ma et al. 2025.

2024

• Training Large Language Models to Reason in a Continuous Latent Space - Hao et al. 2024.
• AdaSociety: An Adaptive Environment with Social Structures for Multi-Agent Decision-Making - Huang et al. 2024.
• AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models - Fang et al. 2024.
• Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters - Snell et al. 2024.
• Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process - Ye et al. 2024.
• From Theft to Bomb-Making: The Ripple Effect of Unlearning in Defending Against Jailbreak Attacks - Zhang et al. 2024.
• Refusal in Language Models Is Mediated by a Single Direction - Arditi et al. 2024.
• AI Sandbagging: Language Models can Strategically Underperform on Evaluations - Weij et al. 2024.
• Language Models Resist Alignment: Evidence From Data Compression - Ji et al. 2024.
• Safety Alignment Should Be Made More Than Just a Few Tokens Deep - Qi et al. 2024.
• Probabilistic Inference in Language Models via Twisted Sequential Monte Carlo - Zhao et al. 2024.
• Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study - Xu et al. 2024.
• Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewards - Hwang et al. 2024.
• PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Models - Meng et al. 2024.
• MineLand: Simulating Large-Scale Multi-Agent Interactions with Limited Multimodal Senses and Physical Needs - Yu et al. 2024.
• The Unreasonable Ineffectiveness of the Deeper Layers - Gromov et al. 2024.
• RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval - Wen et al. 2024.
• Massive Activations in Large Language Models - Sun et al. 2024.
• No Free Lunch in LLM Watermarking: Trade-offs in Watermarking Design Choices - Pang et al. 2024.
• Debating with More Persuasive LLMs Leads to More Truthful Answers - Khan et al. 2024.
• TOFU: A Task of Fictitious Unlearning for LLMs - Maini et al. 2024.

2023

• Point Transformer V3: Simpler, Faster, Stronger - Wu et al. 2023.
• The Earth is Flat because...: Investigating LLMs' Belief towards Misinformation via Persuasive Conversation - Xu et al. 2023.
• Language Agents with Reinforcement Learning for Strategic Play in the Werewolf Game - Xu et al. 2023.
• Controlled Decoding from Language Models - Mudgal et al. 2023.
• Detecting Pretraining Data from Large Language Models - Shi et al. 2023.
• EconAgent: Large Language Model-Empowered Agents for Simulating Macroeconomic Activities - Li et al. 2023.
• Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! - Qi et al. 2023.
• YaRN: Efficient Context Window Extension of Large Language Models - Peng et al. 2023.
• ProAgent: Building Proactive Cooperative Agents with Large Language Models - Zhang et al. 2023.
• Direct Preference Optimization: Your Language Model is Secretly a Reward Model - Rafailov et al. 2023.
• Towards Revealing the Mystery behind Chain of Thought: A Theoretical Perspective - Feng et al. 2023.
• Can Language Models Solve Graph Problems in Natural Language? - Wang et al. 2023.
• Visual Instruction Tuning - Liu et al. 2023.
• Generative Agents: Interactive Simulacra of Human Behavior - Park et al. 2023.
• MarioGPT: Open-Ended Text2Level Generation through Large Language Models - Sudhakaran et al. 2023.

2022

• Demonstrate-Search-Predict: Composing Retrieval and Language Models for Knowledge-Intensive NLP - Khattab et al. 2022.
• On the Blind Spots of Model-Based Evaluation Metrics for Text Generation - He et al. 2022.
• Fast Inference from Transformers via Speculative Decoding - Leviathan et al. 2022.
• Mass-Editing Memory in a Transformer - Meng et al. 2022.

2021

• Towards a Unified View of Parameter-Efficient Transfer Learning - He et al. 2021.
• RoFormer: Enhanced Transformer with Rotary Position Embedding - Su et al. 2021.

2020

• Transformer Feed-Forward Layers Are Key-Value Memories - Geva et al. 2020.
• Rethinking Attention with Performers - Choromanski et al. 2020.
• A Systematic Characterization of Sampling Algorithms for Open-ended Language Generation - Nadeem et al. 2020.
• On Layer Normalization in the Transformer Architecture - Xiong et al. 2020.
• Scaling Laws for Neural Language Models - Kaplan et al. 2020.

Blogs

• Your Efficient RL Framework Secretly Brings You Off-Policy RL Training - Feng Yao.
• Learning Beyond Gradients - Jiayi Weng.
• Language model harnesses are compositional generalizers - Alex Zhang.

Books

• Understanding the Digital World: What You Need to Know about Computers, the Internet, Privacy, and Security - Brian W. Kernighan. Second Edition, 2021.
• 《Thinking in C++》 - Bruce Eckel.
  点评：els最喜欢的一本讲C++的书。
• 《Effective Python》 - Brett Slatkin.
  点评：els最喜欢的一本讲python的书。
• 《The Deep Learning Book》 - Ian Goodfellow, Yoshua Bengio, Aaron Courville.
• 《Computer Science Theory for the Information Age》 - John Hopcroft, Ravi Kannan.
• 《Unity和C#游戏编程入门》 - Harrison Ferrone（美），Unity 2022。
  点评：同时入门Unity和C#，作为第一本书挺合适。
• 《Unity实战》 - Joseph Hocking（美）。
  点评：比较好的教学例子合集，可以作为第二本书看。
• 《游戏设计梦工厂》 - Tracy Fullerton。
  点评：全书教普通人从零完整设计电子游戏，不需要编程、美术基础，纯设计方法论。也可以作为第一本书看。
• 《3D数学基础for图形和游戏开发》 - Fletcher Dunn。
  点评：很建议想钻研 3D 图形和性能方面的同学看。虽说是数学书但写的非常新人友好。
• 《基层中国的运行逻辑》 - 聂辉华。
• 《围城》 - 钱钟书。