Algorithm/Software

ArkVale: Efficient Generative LLM Inference with Recallable Key-Value Eviction

Renze Chen, Zhuofeng Wang, Beiquan Cao, Tong Wu, Size Zheng, Xiuhong Li, Xuechao Wei, Shengen Yan, Meng Li, Yun Liang

PrivCirNet: Efficient Private Inference via Block Circulant Transformation

Tianshi Xu, Lemeng Wu, Runsheng Wang, Meng Li

AdapMoE: Adaptive Sensitivity-based Expert Gating and Management for Efficient MoE Inference

Shuzhang Zhong, Ling Liang, Yuan Wang, Runsheng Wang, Ru Huang, Meng Li

FlexHE: A flexible Kernel Generation Framework for Homomorphic Encryption-Based Private Inference

Jiangrui Yu, Wenxuan Zeng, Tianshi Xu, Renze Chen, Yun (Eric) Liang, Runsheng Wang, Ru Huang, Meng Li

MCUBERT: Memory-Efficient BERT Inference on Commodity Microcontrollers

Zebin Yang, Renze Chen, Taiqiang Wu, Ngai Wong, Yun (Eric) Liang, Runsheng Wang, Ru Huang, Meng Li

PrivQuant: Communication-Efficient Private Inference with Quantized Network/Protocol Co-Optimization

Tianshi Xu, Shuzhang Zhong, Wenxuan Zeng, Runsheng Wang, Meng Li

ProPD: Dynamic Token Tree Pruning and Generation for LLM Parallel Decoding

Shuzhang Zhong, Zebin Yang, Ruihao Gong, Runsheng Wang, Ru Huang, Meng Li

FastQuery: Communication-efficient Embedding Table Query for Private LLMs inference

Chenqi Lin, Tianshi Xu, Zebin Yang, Meng Li, Runsheng Wang, Ru Huang

CoPriv: Network/Protocol Co-Optimization for Communication-Efficient Private Inference

A network–protocol co-optimization framework for reducing the communication overhead of secure private inference.

Wenxuan Zeng, Meng Li, Haichuan Yang, Wen-Jie Lu, Runsheng Wang, Ru Huang

Falcon: Accelerating Homomorphically Encrypted Convolutions for Efficient Private Mobile Network Inference

Tianshi Xu, Meng Li, Runsheng Wang, Ru Huang