Sébastien Bubeck

Vice President, Microsoft GenAI

Talk Title: Small Language Models

Abstract:

Large language models (LLMs) have taken the field of AI by storm. But how large do they really need to be? I'll discuss the phi series of models from Microsoft, which exhibit many of the striking emergent properties of LLMs despite having merely a few billion parameters.

Speaker Bio:

Sébastien is Microsoft’s Vice President of Generative AI where he leads the Machine Learning Foundations group at Microsoft Research (MSR) in Redmond, Washington. He joined MSR in 2014 after serving as an Assistant Professor at Princeton University. His early research focused on convex optimization, online algorithms, and adversarial robustness in machine learning. His work received several best paper awards from STOC 2023, NeurIPS 2018, 2021, COLT 2009, 2016, and ALT 2018, 2023.

Today, his work centers around understanding how intelligence emerges in large language models (LLMs), and how to use this understanding to improve LLMs’ intelligence, building towards AGI through an approach termed the “Physics of AGI”. His most recent work proves an edge of stability phenomena related to non-convex training dynamics of learning threshold neurons. He received his PhD in Applied Mathematics from INRIA Nord Europe in Lille, France.