Dr. Valerio Mariani received his PhD in Biophysics at the University of Basel in Switzerland and is currently Data Analytics Department Head at Linac Coherent Light Source (LCLS), part of the SLAC National Accelerator Laboratory. He has worked in the past at the Center for Free Electron Laser Science (CFEL) in Hamburg, Germany, and collaborated with world-scale user facilities and light sources like PETRA III and the European XFEL. His previous experiences also include work in CryoEM, electron crystallography and homology modeling. He is especially interested in real time data analysis and data streaming.
Presentation Title:
LCLStream: high throughput data streaming for AI workflows and next generation X-ray science
Presentation Abstract:
Next-generation X-ray light sources are poised to generate unprecedented volumes of high-throughput data at extreme rates. Extracting meaningful scientific insights from these massive datasets will quickly outpace the localized computational and storage capacities available at individual experimental facilities. To resolve this critical bottleneck, X-ray sources will have to seamlessly transfer data to external HPC facilities, such as the DOE Leadership Computing Facilities, in formats natively optimized for modern analysis workflows and artificial intelligence (AI) applications. We introduce LCLStream, a novel data-streaming framework designed from the ground up for next-generation workloads, including AI training, high-rate crystal structure determination, and custom data science pipelines. The framework successfully bridges cloud microservices with traditional high-performance computing (HPC) batch execution models, ensuring secure, low-latency data movement while granting users complete control over their data. Currently being deployed in production at LCLS, LCLStream provides a critical, high-speed streaming resource for the facility’s users.