Finding the Ground State of Spin Hamiltonians with Reinforcement Learning
By Kyle Mills, Pooya Ronagh, & Isaac Tamblyn
Reinforcement learning (RL) has become a proven method for optimizing a procedure for which success has been defined, but the specific actions needed to achieve it have not. Using a method we call “controlled online optimization learning” (COOL), we apply the so-called “black box” method of RL to simulated annealing (SA), demonstrating that an RL agent based on proximal policy optimization can, through experience alone, arrive at a temperature schedule that surpasses the performance of standard heuristic temperature schedules for two classes of Hamiltonians. When the system is initialized at a cool temperature, the RL agent learns to heat the system to “melt” it and then slowly cool it in an effort to anneal to the ground state; if the system is initialized at a high temperature, the algorithm immediately cools the system. We investigate the performance of our RL-driven SA agent in generalizing to all Hamiltonians of a specific class. When trained on random Hamiltonians of nearest-neighbour spin glasses, the RL agent is able to control the SA process for other Hamiltonians, reaching the ground state with a higher probability than a simple linear annealing schedule. Furthermore, the scaling performance (with respect to system size) of the RL approach is far more favourable, achieving a performance improvement of almost two orders of magnitude on L = 14² systems. We demonstrate the robustness of the RL approach when the system operates in a “destructive observation” mode, an allusion to a quantum system where measurements destroy the state of the system. The success of the RL agent could have far-reaching impacts, from classical optimization, to quantum annealing and to the simulation of physical systems.
PDF ARXIV PREPRINT
Journal reference: Mills, K., Ronagh, P., & Tamblyn, I. Finding the ground state of spin Hamiltonians with reinforcement learning. Nat Mach Intell 2, 509–517 (2020).
Most Recent Papers
Neural Error Mitigation of Near-Term Quantum Simulations
By Elizabeth R. Bennewitz, Florian Hopfmueller, Bohdan Kulchytskyy, Juan Carrasquilla, & Pooya Ronagh
One of the promising applications of early quantum computers is the simulation of quantum systems. Variational methods for near-term quantum computers, such as the variational quantum eigensolver (VQE), are a promising approach to finding ground states of quantum systems relevant in physics, chemistry, and materials science…
Benchmark Study of Quantum Algorithms for Combinatorial Optimization: Unitary versus Dissipative
By Krishanu Sankar, Artur Scherer, Satoshi Kako, Sam Reifenstein, Navid Ghadermarzy, Willem B. Krayenhoff, Yoshitaka Inui, Edwin Ng, Tatsuhiro Onodera, Pooya Ronagh, & Yoshihisa Yamamoto
We study the performance scaling of three quantum algorithms for combinatorial optimization: measurement-feedback coherent Ising machines (MFB-CIM), discrete adiabatic quantum computation (DAQC), and the Dürr-Hoyer algorithm for quantum minimum finding (DH-QMF) that is based on Grover’s search. We use MaxCut problems as our reference for comparison, and time-to-solution (TTS) as a practical measure of performance for these optimization algorithms…
Scaling Up Electronic Structure Calculations on Quantum Computers: The Frozen Natural Orbital Based Method of Increments
By Prakash Verma, Lee Huntington, Marc Coons, Yukio Kawashima, Takeshi Yamazaki, & Arman Zaribafiyan
The method of increments and frozen natural orbital (MI-FNO) framework is introduced to help expedite the application of noisy, intermediate-scale quantum (NISQ) devices for quantum chemistry simulations. The MI-FNO framework provides a systematic reduction of the occupied and virtual orbital spaces for quantum chemistry simulations. The correlation energies of the resulting increments from the MI-FNO reduction can then be solved by various algorithms, including quantum algorithms such as the phase estimation algorithm and the variational quantum eigensolver (VQE)…
Variationally Scheduled Quantum Simulation
By Shunji Matsuura, Samantha Buck, Valentin Senicourt, & Arman Zaribafiyan
Eigenstate preparation is ubiquitous in quantum computing, and a standard approach for generating the lowest-energy states of a given system is by employing adiabatic state preparation (ASP). In the present work, we investigate a variational method for determining the optimal scheduling procedure within the context of ASP. In the absence of quantum error correction, running a quantum device for any meaningful amount of time causes a system to become susceptible to the loss of relevant information…
Efficient and Accurate Electronic Structure Simulation Demonstrated on a Trapped-Ion Quantum Computer
By Yukio Kawashima, Marc P. Coons, Yunseong Nam, Erika Lloyd, Shunji Matsuura, Alejandro J. Garza, Sonika Johri, Lee Huntington, Valentin Senicourt, Andrii O. Maksymov, Jason H. V. Nguyen, Jungsang Kim, Nima Alidoust, Arman Zaribafiyan, & Takeshi Yamazaki
Quantum computers have the potential to perform accurate and efficient electronic structure calculations, enabling the simulation of properties of materials. However, today’s noisy, intermediate-scale quantum (NISQ) devices have a limited number of qubits and gate operations due to the presence of errors. Here, we propose a systematically improvable end-to-end pipeline to alleviate these limitations…