Day 1
Sergei Kalinin
Weston Fulton Chair Professor
University of Tennessee, Knoxville
From Human-Operated to Agentic Microscopes: Realizing Human Intent at the Instrument Limits
Scientific instruments make physical sciences possible by extending what we can observe, measure, and manipulate beyond human senses. A simple Levenhoek optical microscope does not change physics, but it turns what the eye can barely resolve into something measurable. Electron and scanning probe microscopes take the same idea to the nano and atomic scale. The 1990s were the turning point when computers became inseparable from instruments, enabling fast digital acquisition, programmable control, and new measurement paradigms. Yet the operating model largely stayed the same: humans make the decisions, instruments execute commands. At the same time, for many of these instruments the physical limits and even engineering limits of existing hardware are many orders of magnitude beyond human decision-making ability. Closing even part of that gap is a direct path to higher scientific throughput.
In this talk, we describe our effort to move from human-operated microscopes to ML-enabled instruments. We start with bottom-up, reward-driven agents and show how they work for practical tasks such as instrument optimization in electron and scanning probe microscopy, rapid discovery of structure–property relationships, and exploration of combinatorial novelty using novelty search with human-in-the-loop operation. We also show the limitation of this approach: in real experiments, rewards are rarely known precisely. The agent’s behavior is therefore only as good as the reward model, which motivates interactive workflows where humans shape objectives, constrain actions, and iteratively tune reward proxies.
We then show how this quickly becomes an open decision-making problem. Once the objective is not simple tasks as “maximize contrast” but “choose the next measurement that best advances a scientific hypothesis,” the right framing shifts to experimental planning. We illustrate the transition from single-loop optimization to multi-step planning using dynamic programming, and present a general framework that couples an experimental planning agent with digital twins of the instrument and the material system. Finally, we discuss the limits of scaling instrument agency and how it changes when autonomy is no longer a single microscope problem but a fleet problem. We outline the path toward federated instruments through BlueSky-style integration, where agents operate across workflows and sites, and where the unit of progress becomes not data volume but rate of decision making and execution.
Biography
Sergei Kalinin is a Weston Fulton chair professor at the University of Tennessee, Knoxville. In 2022 – 2023, he has been a principal scientist at Amazon special projects (moon shot factory). Before then, he spent 20 years at Oak Ridge National Laboratory where he was corporate fellow and group leader at the Center for Nanophase Materials Sciences. He received his MS degree from Moscow State University in 1998 and Ph.D. from the University of Pennsylvania (with Dawn Bonnell) in 2002. His research focuses on the applications of machine learning and artificial intelligence methods in materials synthesis, discovery, and optimization, automated experiment and autonomous imaging and characterization workflows in scanning transmission electron microscopy and scanning probes for applications including physics discovery, atomic fabrication, as well as mesoscopic studies of electrochemical, ferroelectric, and transport phenomena via scanning probe microscopy. When at ORNL, he led several major programs integrating ML/AI and physical sciences and instrumentation, including the Institute for Functional Imaging of Materials (IFIM 2014-2019), the first program in DOE integrating ML and physical sciences, and the microscopy effort in INTERSECT program that realized first ML-controlled scanning probe and electron microscopes. At UTK MSE, he participated in building one of the first efforts in the country on ML-driven materials exploration. At UTK, his team has now realized fully AI-controlled SPM and STEM systems and co-orchestration workflows between multiple characterization tools for scientific discovery. He has also taught multiple courses on the ML for materials science and microscopy including Bayesian optimization methods. Sergei has co-authored >650 publications, with a total citation of ~63,000 and an h-index of ~125. He is a fellow of NAI, Academia Europaea, AAAS, RSC, AAIA, MRS, APS, IoP, IEEE, Foresight Institute, and AVS; a recipient of the Adler Lectureship (APS 2025), Duncumb Award (MSA 2024), Medard Welch Award (AVS 2023), Orton Lectureship (ACerS 2023), Feynmann Prize of Foresight Institute (2022), Blavatnik Award for Physical Sciences (2018), RMS medal for Scanning Probe Microscopy (2015), Presidential Early Career Award for Scientists and Engineers (PECASE) (2009); Burton medal of Microscopy Society of America (2010); 5 R&D100 Awards (2008, 2010, 2016, 2018, and 2023); and a number of other distinctions. As part of his professional services, he organized many professional conferences and workshops at MRS, APS and AVS; for 15 years organized workshop series on PFM, and served/s on multiple Editorial Boards including NPJ Comp. Mat., J. Appl. Phys, and Appl. Phys Lett.
Day 2
Aleksandra Faust
Director of Research
Google, DeepMind
From Simulators to Synthetic Worlds: Training AI for High-Consequence Realities
The ability to deploy AI into high-consequence realities is fundamentally tied to our mastery of simulated data. This talk explores the evolution of world-building in training intelligent agents, beginning with classical simulators that taught us how to automatically generate curriculum tasks and optimize behaviors for complex navigation. To bridge the gap between procedural generation and reality, the field then transitioned to data-driven simulators grounded in large-scale, real-world datasets, where combining imitation with reinforcement learning allowed agents to safely navigate unpredictable, data-scarce edge cases. Today, these historical lessons power the current frontier: generative foundation models that thrive on complex synthetic environments. Whether predicting physical molecular structures to overcome experimental data scarcity, training conversational agents through multi-turn, adversarial simulations to master high-stakes clinical decision-making, shaping synthetic personality traits through psychometric testing, or leveraging language models as partial world models to efficiently focus planning, robust simulation remains the core ingredient. As technical performance rapidly accelerates, we must pause to consider the profound implications of what we are building. Ultimately, by anchoring generative AI in proven safety paradigms, we can deploy systems that positively transform safety-critical domains, solving challenges too complex for humans to tackle alone.
Biography
Aleksandra Faust is a Director of Research at Google DeepMind, where she leads Frontier AI Health efforts. Her research focuses on foundation models and world models for complex adaptive systems, treating the AI design pipeline as a learnable, sequential, and self-improving decision-making process. This methodology has driven state-of-the-art improvements across drug discovery, robotics, autonomous driving, and web agents, and led to her founding the field of Automated Reinforcement Learning (AutoRL). Notably, she co-authored the seminal “Levels of AGI” framework and led the Gemini Self-improvement research team, developing the reinforcement learning methods behind the Gemini model family. Previously, Aleksandra served as Chief AI Officer at Genesis Molecular AI and held foundational leadership roles at Google Brain, Google Robotics, and Waymo/X. Earlier in her career, she was a Senior R&D Engineer at Sandia National Laboratories. Faust holds a Ph.D. in Computer Science with distinction from the University of New Mexico and an M.S. from the University of Illinois at Urbana-Champaign. She is an IEEE Fellow and a recipient of the IEEE RAS Early Career Award for Industry and the Tom L. Popejoy Dissertation Award, and was named a Distinguished Alumna of the UNM School of Engineering. Her work has been featured in The New York Times, The Economist, and Forbes, and has received multiple Best Paper Awards at premier robotics, machine learning, and systems architecture venues.