Speaker
Description
The work explores the application of Deep Reinforcement Learning (DRL) to optimize the locking procedure of high-finesse Fabry-Perot (FP) cavities, critical components in Gravitational Wave (GW) detectors. Improving and speeding up the locking procedure for a correct resonance acquisition of these cavities aim to improve the detector’s duty cycle, enhancing the Science Mode time (Accadia et al. 2011).
The locking process is highly challenging due to several non-linear effects, such as cavity ringing and resonance drifts depending on the cavity configuration. These effects spoil the signals used for resonance recognition. Traditional methods like the Modified Guided Lock technique (Bersanetti et al. 2020) mitigate effects as the cavity ring-down by dynamically reducing the cavity speed before engaging the Pound-Drever-Hall technique (Black 2001). However, they remain limited providing non-optimized control impulses for each cavity state. Previous studies have applied DRL and machine learning to control tasks in the GW context, such as alignment optimization (Sorokin et al. 2021, Mukund et al. 2023) and handling non-linear dynamics in longitudinal lock acquisition (Ma, Vajente 2023). Building on these results, we take a different approach by leveraging the power of DRL agents in adapting to the dynamic and non-linear nature of cavity behavior, determining the optimal action for each state. We developed a simulator in order to model the optical response of a FP cavity, considering only the longitudinal degree of freedom and taking into account some of the non-linear effects. Subsequently, the simulator was used to develop a custom Gymnasium environment (Towers et al. 2024) to let the DRL agent, Deep Deterministic Policy Gradient (DDPG) (Lillicrap et al.2016), interacting with the system and learning. Finally, we address the critical challenge of SimToReal (Zhao et al. 2020) transfer and the reality gap, laying the groundwork for real optical applications and potentially offering a predictive, adaptive approach to enhance FP cavity lock acquisition efficiency and reliability.
