Abstract
With the increasing demand for the data-intensive wireless multimedia services over the time-varying wireless channels, the big-data based wireless network problem demands the 5G candidate framework to process such massive amount of multimedia data without causing extra burden to the backhaul links in supporting the heterogeneous statistical delay-bounded quality-of-service (QoS) provisionings. Due to the benefits of energy harvesting (EH) technologies, wireless devices are able to support the data-intensive wireless multimedia services by harvesting energy from the environment. Energy harvesting has emerged as the promising technology to solve the energy supply problem while bringing new challenges due to the stochastic nature of the harvested energy in supporting the heterogeneous statistical quality-of-service (QoS) provisionings. However, due to the unknown dynamics of the harvested energy as well as the channel state information (CSI), it is challenging to design the efficient routing protocol for selecting the optimal routing and power allocation policies under the statistical delay-bounded QoS constraints. To overcome the aforementioned problems, in this paper we propose the Q-learning based optimal routing and power allocation policies through learning from the history of the energy harvesting process while satisfying the heterogeneous statistical delay-bounded QoS constraints over multihop big-data relay networks. In particular, under the heterogeneous statistical delay-bounded QoS requirements, we formulate the end-to-end effective-capacity optimization problem for the battery-free energy harvesting based big-data multihop relay networks. Then, we apply the Markov decision process as well as Q-learning methods for deriving the optimal multihop routing algorithms over big-data multihop relay networks. Also conducted is a set of simulations which evaluate the system performances and show that our proposed Q-learning based multihop routing scheme outperforms the other existing schemes under the heterogeneous statistical delay-bounded QoS constraints over multihop big-data relay networks.