New AI Project: Automation of Scanning Electron Microscope Operations

"Machine learning (ML) has become critical for post-acquisition data analysis in (scanning) transmission electron microscopy, (S)TEM, imaging and spectroscopy. An emerging trend is the transition to real-time analysis and closed-loop microscope operation. The effective use of ML in electron microscopy now requires the development of strategies for microscopy-centric experiment workflow design and optimization. Here, we discuss the associated challenges with the transition to active ML, including sequential data analysis and out-of-distribution drift effects, the requirements for edge operation, local and cloud data storage, and theory in the loop operations. Specifically, we discuss the relative contributions of human scientists and ML agents in the ideation, orchestration, and execution of experimental workflows, as well as the need to develop universal hyper languages that can apply across multiple platforms. These considerations will collectively inform the operationalization of ML in next-generation experimentation." [Kalinin, S.V., Mukherjee, D., Roccapriore, K. et al. Machine learning for automated experimentation in scanning transmission electron microscopy. npj Comput Mater 9, 227 (2023). https://doi.org/10.1038/s41524-023-01142-0]

Starting here, we'll be automating the operation of the ASDRP SEM, exploring the interconnections of electron microscopy, Operating Systems & a 'rules-based' AI to standardize the operation & retrieval of data from a 'remote' machine. Operating Systems covered will be Windows & Linux, from device drivers through remote procedure calls [RPCs] for runtime-loadable modules [RLMs]. All this will be backed by advanced networking intended to provide for end-to-end security while making remote access available for incremental functions.