Attention Sink Forges Native MoE in Attention Layers: Sink-Aware Training to Address Head Collapse

Zizhuo Fu

PhD Student (co-advised w/ Prof. Runsheng Wang, 2026)

Wenxuan Zeng

Tencent Inc (Talent Program)

5+ first-author papers; recipient of 北京大学优秀毕业论文

Meng Li

Assistant Professor

I develop efficient and privacy-preserving AI systems through algorithm–hardware co-design, with applications in large language models, embodied AI, and multimodal intelligence.