The Power of Forgetting: Improving the Last-Good-Reply Policy in Monte Carlo Go
Drake, Peter D.
|Board games; Computer Science; Computer Science, Artificial Intelligence; Computer Science, Software Engineering; Go; machine learning; Monte Carlo methods
|IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
|IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES
The dominant paradigm for programs playing the game of Go is Monte Carlo tree search. This algorithm builds a search tree by playing many simulated games (playouts). Each playout consists of a sequence of moves within the tree followed by many moves beyond the tree. Moves beyond the tree are generated by a biased random sampling policy. The recently published last-good-reply policy makes moves that, in previous playouts, have been successful replies to immediately preceding moves. This paper presents a modification of this policy that not only remembers moves that recently succeeded but also immediately forgets moves that recently failed. This modification provides a large improvement in playing strength. We also show that responding to the previous two moves is superior to responding to the previous one move. Surprisingly, remembering the win rate of every reply performs much worse than simply remembering the last good reply (and indeed worse than not storing good replies at all).
Show full item record
checked on Feb 23, 2024