-
No. 05 · Research note · October 18, 2026
Sampling from Manifolds of Minimizers
How can loss and gradient evaluations let us sample an entire minimizer set? Gibbs concentration and Langevin sampling separate the geometric and computational questions. The Superquantile-Gibbs analysis extends these guarantees uniformly across a parameter family.
Saeed Masiha · 12 min read
-
No. 04 · Research note · October 16, 2026
Same Training Loss, Different Behavior: Which Solution Should We Choose?
A training objective can leave many solutions with different validation performance. Select-then-differentiate makes that choice part of bilevel optimization, with a practical application to learning data mixtures for LLM fine-tuning.
Saeed Masiha · 7 min read
-
No. 03 · Research note · September 22, 2026
Scaling Almost Anything?
Scaling laws let us trade compute for capability, but parameter count is only a coarse measure of capacity. Through a least-squares toy problem and three examples from LLMs — LoRA rank, Transformer depth, and MoE experts — larger can even hurt. What matters is how much of that capacity training actually exploits.
Bingcong Li · 12 min read
-
No. 02 · AI for Math and Optimization Seminar · September 15, 2026
Open Problems as Active Learning in the Age of AI
Lessons from a semester of the AI for Mathematics and Optimization seminar at ETH Zürich, where BSc and MSc students worked on nine genuine open problems with AI as a fallible partner. Three were solved, two produced partial progress — but the more important lesson was educational.
Niao He, Zebang Shen, Fan Wang · 11 min read
-
No. 01 · AI for Math and Optimization Seminar · July 7, 2026
Cycling and Non-Acceleration of the Heavy-Ball Method in One Dimension
Recent work rules out acceleration of the heavy-ball method in dimension two and above by constructing functions on which it cycles. We close the remaining case: by rotating and projecting the two-dimensional cycle onto a line, heavy ball provably cannot accelerate on univariate functions either. The core construction is formalized in Lean 4.
Konstantinos Fotopoulos, Michael Helcig, Karl Deck, Jannis Alsbach · 9 min read
