Seminar // CS Department / Multi-Accelerator Execution of Neural Network Inference on Diversely Heterogeneous SoCs
Speaker: Mehmet Esat Belviranli, Ph.D.
Title: ” Multi-Accelerator Execution of Neural Network Inference on Diversely Heterogeneous SoCs”
Date: Wednesday, November 2, 2022
Time: 10:30-11:20
Place: EA 409
Abstract:
Computing systems are becoming more complex by integrating specialized processing units, i.e., accelerators, that are optimized to perform a specific type of operation. This demand is fueled by the need to run distinct workloads in mobile and autonomous platforms. Such systems often embed diversely heterogeneous System-on-Chips(SoC) where an operation can be executed by more than a single type of accelerator with varying performance, energy, and latency characteristics. A hybrid (i.e., multi-accelerator) execution of popular workloads, such as neural network (NN) inference, collaboratively and concurrently on different types of accelerators in a diversely heterogeneous SoC is a relatively new and unexplored scheme. Multi-accelerator execution has the potential to provide unique benefits for computing systems with limited resources.
In this talk, we investigate a framework that enables resource-constraint aware multi-accelerator execution for diversely heterogeneous SoCs. We achieve this by distributing the layers of a NN inference across different accelerators so that the trade-off between performance and energy satisfies system constraints. We further explore improving total throughput by concurrently using different types of accelerators for executing NNs in parallel. Our proposed methodology uniquely considers inter-accelerator transition costs, shared-memory contention and accelerator architectures that embed internal hardware pipelines. We employ empirical performance models and constraint-based optimization problems to determine optimal multi-accelerator execution schedules.
Bio:
Mehmet Esat Belviranli received his Ph.D. degree in Computer Science from University of California, Riverside in 2016. Prior to joining Colorado School of Mines, he continued his research and mentoring activities at Oak Ridge National Laboratory as Staff Computer Scientist. Mehmet’s research targets increasing the resource utilization of heterogeneous systems and running emerging workloads such as neural networks more efficiently. He developed runtime systems, scheduling algorithms, analytical models, extended memory spaces, programming abstractions and OS & architecture level support to improve heterogeneous computing. His work has been published in major systems conferences including MICRO, DAC, PPoPP, SC, ICS, PACT and DATE.