Bingyi Zhang

Graph Neural Networks (GNNs) have revolutionized many real-world applications where data can be represented as graphs. These applications include recommendation systems, social networks, traffic prediction, computer vision tasks (e.g., the tasks in autonomous driving), etc. Many applications require high-performance execution (e.g., low latency or high throughput) of GNNs. To this end, several state-of-the-art libraries (e.g., PyTorch Geometric and Deep Graph Library) have been developed for executing GNNs on general-purpose processors, including CPUs and GPGPUs. However, current GNN libraries on general-purpose processors achieve sub-optimal performance due to several challenges: 1. Irregular data structures: graphs in real-world applications are highly unstructured, with uneven degree distribution. Such irregularity leads to complex data access patterns. 2. Heterogeneous computation kernels: GNNs involve both sparse computation kernels (e.g., sparse-dense matrix multiplication) and dense computation kernels (dense-dense matrix multiplication). While general-purpose processors are efficient for dense computations, their data path and memory hierarchy are inefficient for sparse computations. 3. Dynamic data sparsity: In many applications, the graph connectivity and the data sparsity of vertex features are unknown before executing the GNN model. Therefore, a GNN system needs to deal with the data sparsity of the graphs dynamically. Such dynamic data sparsity makes it difficult for the compiler and runtime system to generate an optimal execution scheme for GNNs. 4. Mixture of models: Some GNN-based applications use the combined strength of different machine learning models. For example, GNN-based computer vision tasks utilize a mixture of convolutional neural networks (CNNs) and GNN models. Such a combination leads to complex data flow.In this dissertation, we address the above challenges through novel hardware-software codesigns on Field Programmable Gate Array (FPGA). First, to address the challenges of irregular data structures and heterogeneous computation kernels, we develop a hardware-software codesign on FPGA for GNN inference, named GraphAGILE. GraphAGILE incorporates a compiler and an accelerator design on FPGA. For a given input GNN model and an input graph, the compiler translates them into the intermediate representation (IR). Then, the compiler performs several compiler optimizations and generates a sequence of instructions for hardware execution. The hardware accelerator on FPGA executes various computation kernels through flexible customized data paths and memory organization. Second, we propose Dynasparse, an efficient codesign of runtime system and hardware to exploit the dynamic sparsity in GNN inference. The hardware design of Dynasparse has a flexible data path to execute the computation kernels of various data sparsity. The runtime system utilizes a theoretical performance model to dynamically map a GNN computation kernel to the computation primitive based on data sparsity. Third, we propose GCV-Turbo, a hardware-software codesign accelerating GNN-based computer vision (CV) tasks. GNN-based CV tasks involve a mixture of GNN layers and CNN layers. To this end, GCV-Turbo incorporates novel compiler optimizations to orchestrate the dataflow of two types of models. GCV-Turbo further incorporates novel hardware mechanisms to facilitate efficient data layout transformation between different layers. Our implementations based on our codesign methodology achieve superior performance on various GNN-based applications.