Abstract: The interference caused by the relay transmission to reception in full-duplex relaying has a negative effect on system performance. It is increasingly important to research on interference suppression. Different from the conventional suppression schemes which are based on the weighting matrices, the paper presents a means of suppressing the interference, in which relay antenna subset selection at both transmitted and received sides are considered jointly. The criterion for selecting the antenna subset is to minimize interference channel capacity over fading channels for MIMO relay system. It derives closed-form expression for the end-to-end capacity with this proposed method. As a result, simulations demonstrate that the discussed selection scheme mitigates the interference significantly. Therefore, the link quality is efficiently improved in terms of capacity and Bit Error Rate（BER） while with low system complexity.

Key words: interference suppression, full-duplex relay, antenna selection

摘要： 在全双工中继协作通信系统中，中继发送端和接收端之间的干扰严重影响了系统的传输性能，因此干扰抑制是全双工模式下的关键问题。与引入权重矩阵的抑制方法不同，介绍了一种在中继发送端和接收端同时进行天线选择的干扰抑制方法。在多天线中继衰落信道系统中，基于最小化干扰信道的信道容量准则，中继发送端和接收端均选择合适的天线子集。在该抑制方法的基础上，理论分析了系统信道容量的闭式表达。仿真结果表明，天线选择方案改善了系统的信道容量和误码率性能，干扰抑制效果明显，同时降低了系统的复杂性。

关键词: 干扰抑制, 全双工中继, 天线选择