Abstract: In order to reduce the complexity of H.264 encoding algorithm，this paper proposes a fast mode decision algorithm for intra prediction.The prediction type of a macroblock is decided according to boundary information of inner block.Pan’s fast algorithm is improved to filter out some modes and then the best prediction mode is chosen by RDO technology.Simulation results for all-I frame-sequence show that compared with JM10.2，the proposed algorithm reduces the encoding time by about 70% on average，without significant degradation of visual quality and increase in bit rate.Compared with pan，its maximum timesaving is more than 40%.So it’s helpful for real time application of H.264.

Key words: H.264, intra prediction, prediction mode decision, edge direction histogram

摘要： 为了减小H.264的编码算法复杂度，提出了一种快速的帧内模式选择算法。根据中心8×8块边界信息判定宏块预测类型，利用改进的pan算法对选定了预测类型的宏块进行模式筛选，采取RDO准则选出最佳的预测模式。实验结果表明：对全I帧序列，与JM10.2中全模式遍历算法相比较，该算法编码时间平均减少了大约70%，而码率和峰值信噪比基本不变。与pan算法相比，编码时间减少最大超过40%。因此，该算法更便于实时应用。

关键词: H.264, 帧内预测, 预测模式选择算法, 边缘方向直方图