Bilkent University
Department of Computer Engineering
M.S.THESİS PRESENTATİON
Hybrid Parallelization of Stochastic Gradient Descent
Kemal Büyükkaya
MS Student
(Supervisor: Prof.Dr.Cevdet Aykanat)
Computer Engineering Department
Bilkent University
ABSTRACT: The purpose of this study is to investigate the efficient parallelization of the Stochastic Gradient Descent (SGD) algorithm for solving the matrix completion problem on a high-performance computing (HPC) platform in distributed memory setting. We propose a hybrid parallel decentralized SGD framework with asynchronous communication between processors to show the scalability of parallel SGD up to hundreds of processors. We utilize Message Passing Interface (MPI) for inter-node communication and POSIX threads for intra-node parallelism. We tested our method by using four different real-world benchmark datasets. Experimental results show that the proposed framework yields up to 5× better throughput on relatively sparse datasets, and displays comparable performance to available state-of-the-art algorithms on relatively dense datasets while providing a flexible partitioning scheme and a highly scalable hybrid parallel architecture.
DATE: 01 February 2022, Tuesday @ 19:00 Zoom