A Survey on Neural Network Hardware Accelerators

Tamador Mohaidat; Kasem Khalil

doi:10.1109/TAI.2024.3377147

IEEE Transactions on Artificial Intelligence

A Survey on Neural Network Hardware Accelerators

Aug. 2024, pp. 3801-3822, vol. 5

DOI Bookmark: 10.1109/TAI.2024.3377147

Authors

Tamador Mohaidat, Electrical and Computer Engineering Department, University of Mississippi, Oxford, MS, USA
Kasem Khalil, Electrical and Computer Engineering Department, University of Mississippi, Oxford, MS, USA

Keywords

Machine Learning, Hardware Acceleration, Throughput, Costs, Artificial Intelligence, Power Demand, Performance Evaluation, Artificial Intelligence AI, Artificial Neural Network ANN, Convolutional Neural Network CNN, Hardware Accelerator, Machine Learning, Machine Learning Design, Machine Learning On Chip, Neural Network, Recurrent Neural Network RNN, Neural Network, Hardware Accelerators, High Performance, Machine Learning, Convolutional Neural Network, High Speed, Artificial Neural Network, Recurrent Neural Network, Learning Performance, Resource Consumption, Hardware Implementation, Deep Neural Network, Energy Efficiency, Convolutional Layers, Power Consumption, Parallelization, Long Short Term Memory, Graphics Processing Unit, Speech Recognition, Low Latency, Hardware Components, Fewer Units, Graph Convolutional Network, MNIST Dataset, Digital Signal Processing, Off Chip Memory

Abstract

Artificial intelligence (AI) hardware accelerator is an emerging research for several applications and domains. The hardware accelerator's direction is to provide high computational speed with retaining low-cost and high learning performance. The main challenge is to design complex machine learning models on hardware with high performance. This article presents a thorough investigation into machine learning accelerators and associated challenges. It describes a hardware implementation of different structures such as convolutional neural network (CNN), recurrent neural network (RNN), and artificial neural network (ANN). The challenges such as speed, area, resource consumption, and throughput are discussed. It also presents a comparison between the existing hardware design. Last, the article describes the evaluation parameters for a machine learning accelerator in terms of learning and testing performance and hardware design.

Like what you’re reading?

Already a member?

Get this article FREE with a new membership!

References

[1]A. B. Nassif, I. Shahin, I. Attili, M. Azzeh, and K. Shaalan, “Speech recognition using deep neural networks: A systematic review,” IEEE Access, vol. 7, pp. 19143–19165, 2019.

[2]S. Dua , “Developing a speech recognition system for recognizing tonal speech signals using a convolutional neural network,” Appl. Sci., vol. 12, no. 12, 2022, Art. no. 6223.

[3]M. Chun, H. Jeong, H. Lee, T. Yoo, and H. Jung, “Development of Korean food image classification model using public food image dataset and deep learning methods,” IEEE Access, vol. 10, pp. 128732–128741, 2022.

[4]C. T. Sari and C. Gunduz-Demir, “Unsupervised feature extraction via deep learning for histopathological classification of colon tissue images,” IEEE Trans. Med. Imag., vol. 38, no. 5, pp. 1139–1149, May2019.

[5]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “Machine learning-based approach for hardware faults prediction,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 67, no. 11, pp. 3880–3892, Nov.2020.

[6]R. Malhotra, “A systematic review of machine learning techniques for software fault prediction,” Appl. Soft Comput., vol. 27, pp. 504–518, 2015.

[7]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “Intelligent fault-prediction assisted self-healing for embryonic hardware,” IEEE Trans. Biomed. Circuits Syst., vol. 14, no. 4, pp. 852–866, Aug.2020.

[8]L.-Q. Zuo, H.-M. Sun, Q.-C. Mao, R. Qi, and R.-S. Jia, “Natural scene text recognition based on encoder-decoder framework,” IEEE Access, vol. 7, pp. 62616–62623, 2019.

[9]Y. Xu, Y. Wang, W. Zhou, Y. Wang, Z. Yang, and X. Bai, “TextField: Learning a deep direction field for irregular scene text detection,” IEEE Trans. Image Process., vol. 28, no. 11, pp. 5566–5579, Nov.2019.

[10]U. P. Singh, S. S. Chouhan, S. Jain, and S. Jain, “Multilayer convolution neural network for the classification of mango leaves infected by anthracnose disease,” IEEE Access, vol. 7, pp. 43721–43729, 2019.

[11]K. Li, J. Daniels, C. Liu, P. Herrero, and P. Georgiou, “Convolutional recurrent neural networks for glucose prediction,” IEEE J. Biomed. Health Inform., vol. 24, no. 2, pp. 603–613, Feb.2020.

[12]C. N. Freitas, F. R. Cordeiro, and V. Macario, “MyFood: A food segmentation and classification system to aid nutritional monitoring,” in Proc. 33rd SIBGRAPI Conf. Graph. Patterns Images (SIBGRAPI), Piscataway, NJ, USA: IEEE Press, 2020, pp. 234–239.

[13]H. Jelodar, Y. Wang, R. Orji, and S. Huang, “Deep sentiment classification and topic discovery on novel coronavirus or COVID-19 online discussions: NLP using LSTM recurrent neural network approach,” IEEE J. Biomed. Health Inform., vol. 24, no. 10, pp. 2733–2742, Oct.2020.

[14]M. Li, W. Hsu, X. Xie, J. Cong, and W. Gao, “SACNN: Self-attention convolutional neural network for low-dose CT denoising with self-supervised perceptual loss network,” IEEE Trans. Med. Imag., vol. 39, no. 7, pp. 2289–2301, Jul.2020.

[15]B. Dey , “SEM image denoising with unsupervised machine learning for better defect inspection and metrology,” in Proc. Metrol. Inspection Process Control Semicond. Manuf. XXXV, vol. 11611, Bellingham, WA, USA: SPIE, 2021, pp. 245–254.

[16]B. Dey , “Unsupervised machine learning based SEM image denoising for robust contour detection,” in Proc. Int. Conf. Extreme Ultraviolet Lithography, vol. 11854, Bellingham, WA, USA: SPIE, 2021, pp. 88–102.

[17]Y. Liu , “Graph self-supervised learning: A survey,” IEEE Trans. Knowl. Data Eng., vol. 35, no. 6, pp. 5879–5900, Jun.2023.

[18]X. Wang, D. Kihara, J. Luo, and G.-J. Qi, “EnAET: A self-trained framework for semi-supervised and supervised learning with ensemble transformations,” IEEE Trans. Image Process., vol. 30, pp. 1639–1647, 2020.

[19]S. Ahmed, Y. Lee, S.-H. Hyun, and I. Koo, “Unsupervised machine learning-based detection of covert data integrity assault in smart grid networks utilizing isolation forest,” IEEE Trans. Inf. Forensics Secur., vol. 14, no. 10, pp. 2765–2777, Oct.2019.

[20]A. Uprety and D. B. Rawat, “Reinforcement learning for IoT security: A comprehensive survey,” IEEE Internet Things J., vol. 8, no. 11, pp. 8693–8706, Jun.2020.

[21]H. Xu, A. D. Domínguez-García, and P. W. Sauer, “Optimal tap setting of voltage regulation transformers using batch reinforcement learning,” IEEE Trans. Power Syst., vol. 35, no. 3, pp. 1990–2001, May2020.

[22]M. Saharkhizan, A. Azmoodeh, A. Dehghantanha, K.-K. R. Choo, and R. M. Parizi, “An ensemble of deep recurrent neural networks for detecting IoT cyber attacks using network traffic,” IEEE Internet Things J., vol. 7, no. 9, pp. 8852–8859, Sep.2020.

[23]P. Goswami, A. Mukherjee, M. Maiti, S. K. S. Tyagi, and L. Yang, “A neural-network-based optimal resource allocation method for secure IIoT network,” IEEE Internet Things J., vol. 9, no. 4, pp. 2538–2544, Feb.2022.

[24]M. Woźniak, J. Siłka, M. Wieczorek, and M. Alrashoud, “Recurrent neural network model for IoT and networking malware threat detection,” IEEE Trans. Ind. Informat., vol. 17, no. 8, pp. 5583–5594, Aug.2021.

[25]Z. Li, F. Liu, W. Yang, S. Peng, and J. Zhou, “A survey of convolutional neural networks: Analysis, applications, and prospects,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 12, pp. 6999–7019, Dec.2022.

[26]S. Kiranyaz, O. Avci, O. Abdeljaber, T. Ince, M. Gabbouj, and D. J. Inman, “1D convolutional neural networks and applications: A survey,” Mech. Syst. Signal Process., vol. 151, 2021, Art. no. 107398.

[27]V. Veerasamy , “LSTM recurrent neural network classifier for high impedance fault detection in solar PV integrated power system,” IEEE Access, vol. 9, pp. 32672–32687, 2021.

[28]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “Economic LSTM approach for recurrent neural networks,” IEEE Trans. Circuits Syst., II, Exp. Briefs, vol. 66, no. 11, pp. 1885–1889, Nov.2019.

[29]O. I. Abiodun , “Comprehensive review of artificial neural network applications to pattern recognition,” IEEE Access, vol. 7, pp. 158820–158846, 2019.

[30]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “An efficient approach for neural network architecture,” in Proc. 25th IEEE Int. Conf. Electron. Circuits Syst. (ICECS), Piscataway, NJ, USA: IEEE Press, 2018, pp. 745–748.

[31]K. Khalil, O. Eldash, B. Dey, A. Kumar, and M. Bayoumi, “Architecture of a novel low-cost hardware neural network,” in Proc. IEEE 63rd Int. Midwest Symp. Circuits Syst. (MWSCAS), Piscataway, NJ, USA: IEEE Press, 2020, pp. 1060–1063.

[32]E. Wang , “Deep neural network approximation for custom hardware: Where we’ve been, where we’re going,” ACM Comput. Surveys (CSUR), vol. 52, no. 2, pp. 1–39, 2019.

[33]K. Khalil, B. Dey, M. Abdelrehim, A. Kumar, and M. Bayoumi, “An efficient reconfigurable neural network on chip,” in Proc. 28th IEEE Int. Conf. Electron. Circuits Syst. (ICECS), Piscataway, NJ, USA: IEEE Press, 2021, pp. 1–4.

[34]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “N2 OC: Neural-network-on-chip architecture,” in Proc. 32nd IEEE Int. System–Chip Conf. (SOCC), Piscataway, NJ, USA: IEEE Press, 2019, pp. 272–277.

[35]K. Khalil, O. Eldash, B. Dey, A. Kumar, and M. Bayoumi, “A novel reconfigurable hardware architecture of neural network,” in Proc. IEEE 62nd Int. Midwest Symp. Circuits Syst. (MWSCAS), Piscataway, NJ, USA: IEEE Press, 2019, pp. 618–621.

[36]M. A. Rajput, S. Alyami, Q. A. Ahmed, H. Alshahrani, Y. Asiri, and A. Shaikh, “Improved learning-based design space exploration for approximate instance generation,” IEEE Access, vol. 11, pp. 18291–18299, 2023.

[37]G. Armeniakos, G. Zervakis, D. Soudris, and J. Henkel, “Hardware approximate techniques for deep neural network accelerators: A survey,” ACM Comput. Surveys, vol. 55, no. 4, pp. 1–36, 2022.

[38]K. Khalil, A. Kumar, and M. Bayoumi, “Low-power convolutional neural network accelerator on FPGA,” in Proc. IEEE 5th Int. Conf. Artif. Intell. Circuits Syst. (AICAS), Piscataway, NJ, USA: IEEE Press, 2023, pp. 1–5.

[39]C. Åleskog, H. Grahn, and A. Borg, “Recent developments in low-power AI accelerators: A survey,” Algorithms, vol. 15, no. 11, 2022, Art. no. 419.

[40]M. Giordano, L. Piccinelli, and M. Magno, “Survey and comparison of milliwatts micro controllers for tiny machine learning at the edge,” in Proc. IEEE 4th Int. Conf. Artif. Intell. Circuits Syst. (AICAS), Piscataway, NJ, USA: IEEE Press, 2022, pp. 94–97.

[41]S. S. Saha, S. S. Sandha, and M. Srivastava, “Machine learning for microcontroller-class hardware-a review,” IEEE Sens. J., vol. 22, no. 22, pp. 21362–21390, Nov.2022.

[42]K. Khalil, T. Mohaidat, and M. Bayoumi, “Low-cost hardware design approach for long short-term memory (LSTM),” in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), Piscataway, NJ, USA: IEEE Press, 2023, pp. 1–5.

[43]V. Sze, Y.-H. Chen, T.-J. Yang, and J. S. Emer, “How to evaluate deep neural network processors: TOPS/W (alone) considered harmful,” IEEE Solid-State Circuits Mag., vol. 12, no. 3, pp. 28–41, Summer 2020.

[44]N. Gupta, “Introduction to hardware accelerator systems for artificial intelligence and machine learning,” in Hardware Accelerator Systems for Artificial Intelligence and Machine Learning, S. Kim and G. C. Deka, Eds., Elsevier, 2021, ch. 1, vol. 122, pp. 1–21. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0065245820300541

[45]A. Reuther, P. Michaleas, M. Jones, V. Gadepally, S. Samsi, and J. Kepner, “Survey of machine learning accelerators,” in Proc. IEEE High Perform. Extreme Comput. Conf. (HPEC), Piscataway, NJ, USA: IEEE Press, 2020, pp. 1–12.

[46]M. F. Hashmi, R. Pal, R. Saxena, and A. G. Keskar, “A new approach for real time object detection and tracking on high resolution and multi-camera surveillance videos using GPU,” J. Central South Univ., vol. 23, pp. 130–144, 2016.

[47]M. Capra, B. Bussolino, A. Marchisio, G. Masera, M. Martina, and M. Shafique, “Hardware and software optimizations for accelerating deep neural networks: Survey of current trends, challenges, and the road ahead,” IEEE Access, vol. 8, pp. 225134–225180, 2020.

[48]Z. Qi, W. Chen, R. A. Naqvi, and K. Siddique, “Designing deep learning hardware accelerator and efficiency evaluation,” Comput. Intell. Neurosci., vol. 2022, 2022, Art. no. 1291103.

[49]S. Bavikadi , “A survey on machine learning accelerators and evolutionary hardware platforms,” IEEE Des. Test, vol. 39, no. 3, pp. 91–116, Jun.2022.

[50]Z. Zhang, K. Zhang, and A. Khelifi, Multivariate Time Series Analysis in Climate and Environmental Research. Springer, 2018.

[51]B. Dey, K. Khalil, A. Kumar, and M. Bayoumi, “A reversible-logic based architecture for artificial neural network,” in Proc. IEEE 63rd Int. Midwest Symp. Circuits Syst. (MWSCAS), Piscataway, NJ, USA: IEEE Press, 2020, pp. 505–508.

[52]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “Self-healing approach for hardware neural network architecture,” in Proc. IEEE 62nd Int. Midwest Symp. Circuits Syst. (MWSCAS), Piscataway, NJ, USA: IEEE Press, 2019, pp. 622–625.

[53]K. Khalil, A. Kumar, and M. Bayoumi, “Reconfigurable hardware design approach for economic neural network,” IEEE Trans. Circuits Syst., II, Exp. Briefs, vol. 69, no. 12, pp. 5094–5098, Dec.2022.

[54]T. V. Huynh, “Deep neural network accelerator based on FPGA,” in Proc. 4th NAFOSTED Conf. Inf. Comput. Sci., Piscataway, NJ, USA: IEEE Press, 2017, pp. 254–257.

[55]L. D. Medus, T. Iakymchuk, J. V. Frances-Villora, M. Bataller-Mompeán, and A. Rosado-Mu∼noz, “A novel systolic parallel hardware architecture for the FPGA acceleration of feedforward neural networks,” IEEE Access, vol. 7, pp. 76084–76103, 2019.

[56]K. Khalil, B. Dey, A. Kumar, and M. Bayoumi, “Adaptive hardware architecture for neural-network-on-chip,” in Proc. IEEE 65th Int. Midwest Symp. Circuits Syst. (MWSCAS), Piscataway, NJ, USA: IEEE Press, 2022, pp. 1–4.

[57]S. Xiao , “Neuronlink: An efficient chip-to-chip interconnect for large-scale neural network accelerators,” IEEE Trans. Very Large Scale Integr. VLSI Syst., vol. 28, no. 9, pp. 1966–1978, Sep.2020.

[58]B. Zhang , “PIMCA: A programmable in-memory computing accelerator for energy-efficient DNN inference,” IEEE J. Solid-State Circuits, vol. 58, no. 5, pp. 1436–1449, May2023.

[59]L. Song, J. Mao, Y. Zhuo, X. Qian, H. Li, and Y. Chen, “HyPar: Towards hybrid parallelism for deep learning accelerator array,” in Proc. IEEE Int. Symp. High Perform. Comput. Archit. (HPCA), 2019, pp. 56–68.

[60]X. Wei, Y. Liang, P. Zhang, C. H. Yu, and J. Cong, “Overcoming data transfer bottlenecks in DNN accelerators via layer-conscious memory managment,” in Proc. ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays (FPGA), New York, NY, USA: ACM, 2019, p. 120, doi: 10.1145/3289602.3293947.

[61]M. Capra, B. Bussolino, A. Marchisio, M. Shafique, G. Masera, and M. Martina, “An updated survey of efficient hardware architectures for accelerating deep convolutional neural networks,” Future Internet, vol. 12, no. 7, 2020, Art. no. 113.

[62]K. Khalil, B. Dey, A. Kumar, and M. Bayoumi, “A reversible-logic based architecture for convolutional neural network (CNN),” in Proc. IEEE Int. Midwest Symp. Circuits Syst. (MWSCAS),. Piscataway, NJ, USA: IEEE Press, 2021, pp. 1070–1073.

[63]H. Li, X. Yue, Z. Wang, W. Wang, H. Tomiyama, and L. Meng, “A survey of convolutional neural networks—From software to hardware and the applications in measurement,” Meas. Sens., vol. 18, 2021, Art. no. 100080.

[64]B. Dey, K. Khalil, A. Kumar, and M. Bayoumi, “A reversible-logic based architecture for VGGNet,” in Proc. 28th IEEE Int. Conf. Electron. Circuits Syst. (ICECS), Piscataway, NJ, USA: IEEE Press, 2021, pp. 1–4.

[65]Y. Tang, L. Tian, Y. Liu, Y. Wen, K. Kang, and X. Zhao, “Design and implementation of improved CNN activation function,” in Proc. 3rd Int. Conf. Comput. Vis. Image Deep Learn. Int. Conf. Comput. Eng. Appl. (CVIDL & ICCEA), Piscataway, NJ, USA: IEEE Press, 2022, pp. 1166–1170.

[66]K. Khalil, O. Eldash, A. Kumar, and M. Bayoumi, “Designing novel AAD pooling in hardware for a convolutional neural network accelerator,” IEEE Trans. Very Large Scale Integr. VLSI Syst., vol. 30, no. 3, pp. 303–314, Mar.2022.

[67]Q. Song, J. Zhang, L. Sun, and G. Jin, “Design and implementation of convolutional neural networks accelerator based on multidie,” IEEE Access, vol. 10, pp. 91497–91508, 2022.

[68]Y.-S. Ting, Y.-F. Teng, and T.-D. Chiueh, “Batch normalization processor design for convolution neural network training and inference,” in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), Piscataway, NJ, USA: IEEE Press, 2021, pp. 1–4.

[69]B. Khabbazan and S. Mirzakuchaki, “Design and implementation of a low-power, embedded CNN accelerator on a low-end FPGA,” in Proc. 22nd Euromicro Conf. Digit. Syst. Des. (DSD), Piscataway, NJ, USA: IEEE Press, 2019, pp. 647–650.

[70]H. Xiao, K. Li, and M. Zhu, “FPGA-based scalable and highly concurrent convolutional neural network acceleration,” in Proc. IEEE Int. Conf. Power Electron. Comput. Appl. (ICPECA), Piscataway, NJ, USA: IEEE Press, 2021, pp. 367–370.

[71]J. Lee, J. Rhim, D. Kang, and S. Ha, “SNAS: Fast hardware-aware neural architecture search methodology,” IEEE Trans. Comput.-Aided Design Integr. Circuits Syst., vol. 41, no. 11, pp. 4826–4836, Nov.2022.

[72]S. Liu, H. Fan, M. Ferianc, X. Niu, H. Shi, and W. Luk, “Toward full-stack acceleration of deep convolutional neural networks on FPGAs,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 8, pp. 3974–3987, Aug.2022.

[73]H. Wang, Y. Zhao, and F. Gao, “A convolutional neural network accelerator based on FPGA for buffer optimization,” in Proc. IEEE 5th Adv. Inf. Technol., Electron. Automat. Control Conf. (IAEAC), vol. 5, Piscataway, NJ, USA: IEEE Press, 2021, pp. 2362–2367.

[74]P. Achararit, M. A. Hanif, R. V. W. Putra, M. Shafique, and Y. Hara-Azumi, “APNAS: Accuracy-and-performance-aware neural architecture search for neural hardware accelerators,” IEEE Access, vol. 8, pp. 165319–165334, 2020.

[75]T. Yuan, W. Liu, J. Han, and F. Lombardi, “High performance CNN accelerators based on hardware and algorithm co-optimization,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 68, no. 1, pp. 250–263, Jan.2021.

[76]W. Huang , “FPGA-based high-throughput CNN hardware accelerator with high computing resource utilization ratio,” IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 8, pp. 4069–4083, Aug.2022.

[77]M. Kim and J.-S. Seo, “Deep convolutional neural network accelerator featuring conditional computing and low external memory access,” in Proc. IEEE Custom Integr. Circuits Conf. (CICC), Piscataway, NJ, USA: IEEE Press, 2020, pp. 1–4.

[78]Q. Cheng , “A low-power sparse convolutional neural network accelerator with pre-encoding Radix-4 booth multiplier,” IEEE Trans. Circuits Syst., II, Exp. Briefs, vol. 70, no. 6, pp. 2246–2250, Jun.2023.

[79]X. Yu , “A data-center FGPA acceleration platform for convolutional neural networks,” in Proc. 29th Int. Conf. Field Programmable Log. Appl. (FPL), 2019, pp. 151–158.

[80]R. Hwang, M. Kang, J. Lee, D. Kam, Y. Lee, and M. Rhu, “GROW: A row-stationary sparse-dense GEMM accelerator for memory-efficient graph convolutional neural networks,” in Proc. IEEE Int. Symp. High-Perform. Comput. Archit. (HPCA), 2023, pp. 42–55.

[81]A. Graves and J. Schmidhuber, “Framewise phoneme classification with bidirectional LSTM networks,” in Proc. IEEE Int. Joint Conf. Neural Netw., vol. 4, Piscataway, NJ, USA: IEEE Press, 2005, pp. 2047–2052.

[82]K. Smagulova and A. P. James, “A survey on LSTM memristive neural network architectures and applications,” Eur. Phys. J. Special Top., vol. 228, no. 10, pp. 2313–2324, 2019.

[83]K. Khalil, B. Dey, A. Kumar, and M. Bayoumi, “A reversible-logic based architecture for long short-term memory (LSTM) network,” in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), Piscataway, NJ, USA: IEEE Press, 2021, pp. 1–5.

[84]Y. Wei , “A review of algorithm & hardware design for AI-based biomedical applications,” IEEE Trans. Biomed. Circuits Syst., vol. 14, no. 2, pp. 145–163, Apr.2020.

[85]J. Wu, F. Li, Z. Chen, and X. Xiang, “A 3.89-GOPS/mW scalable recurrent neural network processor with improved efficiency on memory and computation,” IEEE Trans. Very Large Scale Integr. VLSI Syst., vol. 27, no. 12, pp. 2939–2943, Dec.2019.

[86]D. Kadetotad, S. Yin, V. Berisha, C. Chakrabarti, and J.-S. Seo, “An 8.93 TOPS/W LSTM recurrent neural network accelerator featuring hierarchical coarse-grain sparsity for on-device speech recognition,” IEEE J. Solid-State Circuits, vol. 55, no. 7, pp. 1877–1887, Jul.2020.

[87]G. Nan , “An energy efficient accelerator for bidirectional recurrent neural networks (BiRNNs) using hybrid-iterative compression with error sensitivity,” IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 68, no. 9, pp. 3707–3718, Sep.2021.

[88]C. Gao, A. Rios-Navarro, X. Chen, S.-C. Liu, and T. Delbruck, “EdgeDRNN: Recurrent neural network accelerator for edge inference,” IEEE J. Emerg. Sel. Topics. Circuits Syst., vol. 10, no. 4, pp. 419–432, Dec.2020.

[89]D. Shan, Y. Luo, X. Zhang, and C. Zhang, “DRRNets: Dynamic recurrent routing via low-rank regularization in recurrent neural networks,” IEEE Trans. Neural Netw. Learn. Syst., vol. 34, no. 4, pp. 2057–2067, Apr.2023.

[90]J. Chen, S. Hong, W. He, J. Moon, and S.-W. Jun, “Eciton: Very low-power LSTM neural network accelerator for predictive maintenance at the edge,” in Proc. 31st Int. Conf. Field-Programmable Log. Appl. (FPL), 2021, pp. 1–8.

[91]A. Vaswani , “Attention is all you need,” Proc. Adv. Neural Inf. Process. Syst., vol. 30, 2017.

[92]W. Li, S. Wang, and G. Liu, “Transformer-based model for fMRI data: ABIDE results,” in Proc. 7th Int. Conf. Comput. Commun. Syst. (ICCCS), 2022, pp. 162–167.

[93]S. Ansari and K. A. Alnajjar, “Multi-hop genetic-algorithm-optimized routing technique in diffusion-based molecular communication,” IEEE Access, vol. 11, pp. 22689–22704, 2023.

[94]M. S. Rao, K. Venkata Rao, and M. H. M. Krishna Prasad, “Hybrid security approach for database security using diffusion based cryptography and diffie-hellman key exchange algorithm,” in Proc. 5th Int. Conf. I-SMAC (IoT Soc. Mob. Analytics Cloud) (I-SMAC), 2021, pp. 1608–1612.

[95]Z. Zhao, R. Cao, K.-F. Un, W.-H. Yu, P.-I. Mak, and R. P. Martins, “An FPGA-based transformer accelerator using output block stationary dataflow for object recognition applications,” IEEE Trans. Circuits Syst., II, Exp. Briefs, vol. 70, no. 1, pp. 281–285, Jan.2023.

[96]Z. Cheng, Z. Zhang, J. Jiang, and J. Sun, “Signal detection of mobile multi-user molecular communication system using transformer-based model,” in Proc. 8th Int. Conf. Comput. Commun. Syst. (ICCCS), 2023, pp. 85–90.

[97]Y. Yan, W. Du, D. Yang, and D. Yin, “CIPTA: Contrastive-based iterative prompt-tuning using text annotation from large language models,” in Proc. 4th Int. Conf. Electron. Commun. Artif. Intell. (ICECAI), 2023, pp. 174–178.

[98]Y. Ye, H. You, and J. Du, “Improved trust in human-robot collaboration with ChatGPT,” IEEE Access, vol. 11, pp. 55748–55754, 2023.

[99]P. Maddigan and T. Susnjak, “Chat2VIS: Generating data visualizations via natural language using ChatGPT, Codex and GPT-3 large language models,” IEEE Access, vol. 11, pp. 45181–45193, 2023.

[100]W. Zhu , “Sensitivity, specificity, accuracy, associated confidence interval and ROC analysis with practical SAS implementations,” in Proc. Health Care Life Sci. (NESUG), Baltimore, MD, USA, vol. 19, 2010, p. 67.

BenchNN: On the broad potential application scope of hardware neural network accelerators
2012 IEEE International Symposium on Workload Characterization (IISWC)
Memory Requirements for Convolutional Neural Network Hardware Accelerators
2018 IEEE International Symposium on Workload Characterization (IISWC)
Software and Hardware Integrated Accelerators for Hadoop Appliance
2021 3rd International Conference on Applied Machine Learning (ICAML)
Special Session: Effective In-field Testing of Deep Neural Network Hardware Accelerators
2022 IEEE 40th VLSI Test Symposium (VTS)
Hardware Accelerator and Neural Network Co-Optimization for Ultra-Low-Power Audio Processing Devices
2022 25th Euromicro Conference on Digital System Design (DSD)
Hardware/Software Co-design for Machine Learning Accelerators
2023 IEEE 31st Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM)
Remote Identification of Neural Network FPGA Accelerators by Power Fingerprints
2023 33rd International Conference on Field-Programmable Logic and Applications (FPL)
LAMBDA: An Open Framework for Deep Neural Network Accelerators Simulation
2021 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)
SECDA: Efficient Hardware/Software Co-Design of FPGA-based DNN Accelerators for Edge Inference
2021 IEEE 33rd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)
A Reconfigurable Floating-Point Compliant Hardware Architecture for Neural Network Implementation
2024 IEEE International Symposium on Smart Electronic Systems (iSES)

A Survey on Neural Network Hardware Accelerators

Authors

Keywords

Abstract

References

Related Articles