关键词: 混合专家模型（MoE）, 计算优化, 负载均衡, 专家结构, 内存管理

Abstract: Large language models have achieved remarkable results in natural language processing and other fields in recent years. Mixture of experts (MoE) reduces computational demands by employing sparse activation strategies. As the inference tasks faced by MoE grow increasingly complex, expert models deployed on edge devices often encounter resource requirements exceeding node computing power. Consequently, computational optimization of MoE under computational constraints has become a persistent research focus in the field. This paper introduces the concept and architecture of MoE, and categorizes and reviews relevant optimization methods across three dimensions： gating networks, expert structures and models, and memory management. At the gating network level, encompasses routing design, loss function optimization, and load balancing mechanisms are studied to achieve precise routing. At the expert structure level, structural innovations are summarized, including various expert designs, preprocessing methods, and expert merging strategies. At the memory management level, existing parameter compression and memory offloading techniques are reviewed to address resource constraints during model deployment. This paper analyzes the principles, strategies, and key technical challenges of computational optimization across different dimensions, and then identifies critical issues and potential research opportunities that need to be focused on.

Key words: mixture of experts (MoE), computational optimization, load balancing, expert structure, memory management