MAM-STM: A software for autonomous control of single moieties towards specific surface positions

In this publication we introduce MAM-STM, a software to autonomously manipulate arbitrary moieties towards specific positions on a metal surface utilizing the tip of a scanning tunneling microscope (STM). Finding the optimal manipulation parameters for a specific moiety is challenging and time consuming, even for human experts. MAM-STM combines autonomous data acquisition with a sophisticated Q-learning implementation to determine the optimal bias voltage, the z-approach distance, and the tip position relative to the moiety. This then allows to arrange single molecules and atoms at will. In this work, we provide a tutorial based on a simulated response to offer a comprehensive explanation on how to use and customize MAM-STM. Additionally, we assess the performance of the machine learning algorithm by benchmarking it within a simulated stochastic environment. PROGRAM SUMMARY: Program title: MAM-STM CPC Library link to program files: (to be added by Technical Editor) Developer's repository link: https://gitlab.tugraz.at/software_public/mam_stm.git Code Ocean capsule: (to be added by Technical Editor) Licensing provisions: GNU General Public License 3 (GPL) Programming language: Python 3 Nature of problem: Achieving precise control over the arrangement of individual molecules on surfaces is essential for advancing nanofabrication and understanding molecular interaction processes. While self-assembly offers a method for forming nanostructures, achieving arbitrary arrangements of moieties remains difficult. Current approaches, such as scanning probe microscopy (SPM), require extensive manual intervention and precise control is difficult to achieve consistently due to the stochastic nature of quantum mechanical systems at the nanoscale. Thus, learning to manipulate several moieties in order to build even relatively small structures is challenging and time consuming and the automation through conventional expert systems is hindered by the lack of prior knowledge about the surface-moiety interaction processes. Solution method: This scenario is ideal for machine learning algorithms, like reinforcement learning (RL), which do not require an underlying model but are able to autonomously learn the optimal manipulation parameters by performing manipulations directly at the machine. Introducing MAM-STM, which stands for Molecular and Atomic Manipulation via Scanning Tunneling Microscopy. MAM-STM allows to control molecules and atoms by learning the manipulation parameters for either vertical or lateral manipulations. However, the vast number of manipulation parameter combinations and the inefficient learning procedure of RL agents exhibit several challenges. MAM-STM overcomes these challenges with an autonomous masking routine that eliminates manipulation parameters that induce structural changes to the moiety or lift it off the surface. Additionally, a sophisticated Q-learning approach is developed that speeds up the learning procedure, enabling molecular manipulations within one day of training.