The Power of Forgetting: Improving the Last-Good-Reply Policy in Monte Carlo Go

Autor(en): Baier, Hendrik
Drake, Peter D.
Stichwörter: Board games; Computer Science; Computer Science, Artificial Intelligence; Computer Science, Software Engineering; Go; machine learning; Monte Carlo methods
Erscheinungsdatum: 2010
Herausgeber: IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
Journal: IEEE TRANSACTIONS ON COMPUTATIONAL INTELLIGENCE AND AI IN GAMES
Volumen: 2
Ausgabe: 4, SI
Startseite: 303
Seitenende: 309
Zusammenfassung: 
The dominant paradigm for programs playing the game of Go is Monte Carlo tree search. This algorithm builds a search tree by playing many simulated games (playouts). Each playout consists of a sequence of moves within the tree followed by many moves beyond the tree. Moves beyond the tree are generated by a biased random sampling policy. The recently published last-good-reply policy makes moves that, in previous playouts, have been successful replies to immediately preceding moves. This paper presents a modification of this policy that not only remembers moves that recently succeeded but also immediately forgets moves that recently failed. This modification provides a large improvement in playing strength. We also show that responding to the previous two moves is superior to responding to the previous one move. Surprisingly, remembering the win rate of every reply performs much worse than simply remembering the last good reply (and indeed worse than not storing good replies at all).
ISSN: 1943068X
DOI: 10.1109/TCIAIG.2010.2100396

Show full item record

Page view(s)

1
Last Week
0
Last month
0
checked on Feb 23, 2024

Google ScholarTM

Check

Altmetric