You are cordially invited to the Department of Mathematics Colloquium
Speaker: Özgür Martin (Mimar Sinan Fine Arts University)
“How to Train Your Large AI Models at a Lower Cost?”
Abstract: Training large neural networks is computationally expensive, and its efficiency
depends strongly on how step sizes are chosen throughout the optimization process. Small learning rates can lead to slow progress, while large ones may cause instability. Carefully designed learning rate schedules can improve training, but selecting and tuning them often requires costly experimentation. Can this process be automated?
This talk focuses on automatic learning rate scheduling in stochastic optimization for neural network training. Starting with stochastic gradient descent (SGD), we will discuss how learning rates influence convergence and examine methods that adapt step sizes using information gathered during training. We will distinguish coordinate-wise adaptation, as used in algorithms such as Adam, from the automatic adjustment of the overall learning
rate.
Date: Wednesday, November 4, 2026
Time: 15:40-16:40
Place: Mathematics Seminar Room, SA-141