Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems

Nikoli Dryden; Naoya Maruyama; Tim Moon; Tom Benson; Andy Yoo; Marc Snir; Brian Van Essen

doi:10.1109/MLHPC.2018.8638639

2018 IEEE/ACM Machine Learning in HPC Environments (MLHPC)

Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems

Year: 2018, Pages: 1-13

DOI Bookmark: 10.1109/MLHPC.2018.8638639

Authors

Nikoli Dryden, Lawrence Livermore National Laboratory
Naoya Maruyama, Lawrence Livermore National Laboratory
Tim Moon, Lawrence Livermore National Laboratory
Tom Benson, Lawrence Livermore National Laboratory
Andy Yoo, Lawrence Livermore National Laboratory
Marc Snir, University of Illinois at Urbana-Champaign, Department of Computer Science
Brian Van Essen, Lawrence Livermore National Laboratory

Abstract

The following topics are dealt with: learning (artificial intelligence); parallel processing; neural nets; convolutional neural nets; data analysis; stochastic processes; gradient methods; image classification; message passing; evolutionary computation.

Like what you’re reading?

Already a member?

Get this article FREE with a new membership!

Evaluation of Deep Learning Frameworks Over Different HPC Architectures
2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS)
Path Reducing Watershed for the GPU
2018 IEEE Winter Conference on Applications of Computer Vision (WACV)
A Code Generation Framework for Targeting Optimized Library Calls for Multiple Platforms
IEEE Transactions on Parallel & Distributed Systems
Considering Race a Problem of Transfer Learning
2019 IEEE Winter Applications of Computer Vision Workshops (WACVW)
CIS 2018 Preface
2018 14th International Conference on Computational Intelligence and Security (CIS)
Towards Java-based HPC using the MVAPICH2 Library: Early Experiences
2022 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW)
A Quantitative Study of Deep Learning Training on Heterogeneous Supercomputers
2019 IEEE International Conference on Cluster Computing (CLUSTER)
AEH-MTD: Adaptive Moving Target Defense Scheme for SDN
2019 IEEE International Conference on Smart Internet of Things (SmartIoT)
CuPhylo: A CUDA Based Application Program Interface and Library for Phylogenetic Analysis
2019 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Big Data & Cloud Computing, Sustainable Computing & Communications, Social Computing & Networking (ISPA/BDCloud/SocialCom/SustainCom)
Unified Collective Communication (UCC): An Unified Library for CPU, GPU, and DPU Collectives
2024 IEEE Symposium on High-Performance Interconnects (HOTI)

Aluminum: An Asynchronous, GPU-Aware Communication Library Optimized for Large-Scale Training of Deep Neural Networks on HPC Systems

Authors

Abstract

Related Articles