Abstract
In this work, physical layer secure transmission of a cooperative wireless communication system over a three-phase amplify-and-forward (AF) two-way relaying channels is studied. A cooperative jamming and multi-antenna relay beamforming scheme that improves physical layer security of the system is proposed. Specifically, it is suggested that in the first and second timeslots when the legitimate users transmit alternatively, a jamming signal can be transmitted by the legitimate party that is not transmitting information-bearing signals This scheme, compared with the approach using a dedicated jammer, achieves a similar jamming effect with reduced complexity. In the third time slot of the protocol, the relay uses two beamforming matrices to process the bidirectional information signals. The jamming signal components at the relay are projected onto the null space of the legitimate relay-destination channels such that they do not affect legitimate communications. Through joint optimization, the sum throughput of the legitimate communication is maximized. In the numerical examples, the proposed scheme is shown to improve the system's secrecy transmission performance.