Yekun Chai

london.jpg

I study language models through the lens of compression. My work centers on large-scale pre-training, driven by one question: at what point does prediction become planning? I have contributed to ERNIE and StarCoder 2.

News

Aug 21, 2025 Four papers accepted to EMNLP 2025.
May 16, 2025 Curiosity-driven RLHF accepted to ACL 2025. code
Jan 23, 2025 MA-RLHF accepted to ICLR 2025. paper code

Latest Posts

Jun 24, 2024 Multimodal Tokenization with Vector Quantization: A Review
May 10, 2024 Memory-Efficient Attention: MHA vs. MQA vs. GQA vs. MLA
Jan 26, 2023 Positional Encoding in Transformers: From Sinusoidal to RoPE

Selected Publications

  1. ICLR
    MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions
    Yekun Chai*† , Haoran Sun*^, Huang Fang, Shuohuan Wang, and 2 more authors
    In The Thirteenth International Conference on Learning Representations, 2025
    TL;DR HTML PDF Code
  2. ICLR Spotlight
    Tool-Augmented Reward Modeling
    Lei Li*^, Yekun Chai*†, Shuohuan Wang, Yu Sun, and 3 more authors
    In The Twelfth International Conference on Learning Representations (top 5%) , 2024
    TL;DR HTML PDF Code Poster Slides
  3. ACL-Findings
    ERNIE-Code: Beyond English-Centric Cross-lingual Pretraining for Programming Languages
    Yekun Chai, Shuohuan Wang, Chao Pang, Yu Sun, and 2 more authors
    In Findings of the Association for Computational Linguistics: ACL 2023, Jul 2023
    TL;DR HTML PDF Code