Bilkent University
Department of Computer Engineering

---

CS 590/690 SEMINAR

 

Partitioning for Communication Efficient Graph Neural Network Training

 

Kutay Taşçı

Master Student
(Supervisor: Assoc.Prof.Can Alkan)
Computer Engineering Department
Bilkent University

Abstract: Graph Neural Networks (GNNs) are among the most powerful and flexible tools used to solve problems on unstructured graph data. GNNs represent graph connectivity as a sparse matrix with low arithmetic density. However, connectivity associated with graphs introduces a dependency between data samples, and GNNs require full-batch training to achieve the best performance. This makes GNNs harder to scale to high concurrencies when compared to other deep learning models. We propose a multi-constraint hypergraph partitioning approach for optimizing communication between sending and receiving processes on both the forward and backward pass. We can represent the computation of each node in a GNN layer as a hypergraph and then partition the node features to different processors. Our multi-constraint hypergraph partitioning aims to balance communication across the full GNN training pipeline. Our method proposes a novel approach for optimizing sparse-dense algebraic operations in GNN training. Also, we can combine our approach by using it as a foundation for more complex state-of-the-art methods to achieve the best performance.

 

DATE: April 24, Monday @ 15:50 Place: Zoom