You are cordially invited to the Analysis Seminar organized by the Department of Mathematics.<\/p>\n
Speaker: Ka\u011fan Akman (Bilkent University)<\/p>\n
\u201cAn Optimal Control Approach to Transformers\u201d<\/p>\n
Abstract: Since their introduction in 2017, Transformers have proven themselves to be an effective structure for large language models.<\/p>\n
Traditionally, a Transformer is trained over a data set by means of gradient descent or some modification of it. In this talk, we commence with presenting a discrete-time dynamical system formulation of a Transformer following Geshkovski et al. (2025). This process exhibits a deterministic McKean-Vlasov type of dynamics. We show that this dynamics can be lifted to a measure-valued discrete-time Markov decision process (MDP). Using the powerful theory of MDPs, we pose the corresponding dynamic programming equations and show the existence of deterministic closed-loop optimal controls, using which we construct open-loop policies compatible with Transformer architecture. Finally, we conclude by relating our formulation with the classical theory of neural networks.<\/p>\n
Date: October 22, Wednesday
\nTime: 14:00 \u2013 15:00
\nPlace: Mathematics Seminar Room, SA \u2013 141<\/p>\n","protected":false},"excerpt":{"rendered":"
You are cordially invited to the Analysis Seminar organized by the Department of Mathematics. Speaker: Ka\u011fan Akman (Bilkent University) \u201cAn Optimal Control Approach to Transformers\u201d Abstract: Since their introduction in 2017, Transformers have proven themselves to be an effective structure for large language models. Traditionally, a Transformer is trained over [\u2026] <\/p>\n