Conditional Computation in Neural Networks: Low Complexity Design Algorithms and Fundamental Limits
Assoc. Prof. Erdem Koyuncu
University of Illinois Chicago
Date/Time: Friday, February 14, 2025 at 16:00-17:00 TSI
Place: EE 517 (in-person)
Abstract: Traditional neural networks utilize an unconditional, tunnel-like design in the sense that the same sequence of operations are applied to every input to the network. An emerging idea is conditional computation, where one activates only parts of the network in an input-adaptive fashion. If designed properly, a conditional network exhibits little to no performance loss as compared to an unconditional network with the same number of parameters. In addition, the sparsely-activated nature of the conditional network brings in computational savings. In this talk, we introduce low complexity design algorithms for conditional networks based on the class means of inputs or feature vectors. We also discuss recent results on fundamental limits of conditional networks, focusing on the achievable performance on a given limit on computation.
Biography: Erdem Koyuncu received the B.S. degree from the Department of Electrical and Electronics Engineering of Bilkent University in 2005. He received the M.S. and Ph.D. degrees from the Department of Electrical Engineering and Computer Science at the University of California, Irvine in 2006 and 2010, respectively. He is currently an Associate Professor in the Department of Electrical and Computer Engineering at the University of Illinois Chicago.