FAQ: A Flexible Accelerator for Q-Learning with Configurable Environment
Autor(en): | Rothmann, M. Porrmann, M. |
Herausgeber: | Pericas, M. Pnevmatikatos, D.N. Trancoso, P.P.M. Sourdis, I. |
Stichwörter: | Domain specific architectures; Domain-specific architectures; Field programmable gate arrays (FPGA); High-throughput; Learning algorithms; Machine Learning; Machine-learning; NASA; Pipeline stages; Q-learning; Q-learning algorithms; Q-values; Reconfigurable architectures; Reconfigurable hardware; Reinforcement Learning; Reinforcement learning algorithms; Reinforcement learnings, Reinforcement learning; Software testing, Bit-Width | Erscheinungsdatum: | 2022 | Herausgeber: | Institute of Electrical and Electronics Engineers Inc. | Journal: | Proceedings of the International Conference on Application-Specific Systems, Architectures and Processors | Volumen: | 2022-July | Startseite: | 106 | Seitenende: | 114 | Zusammenfassung: | Reinforcement Learning is an area of machine learning that is concerned with optimizing the behavior of an agent in an environment by maximizing cumulative rewards. This can be done with classical reinforcement learning algorithms such as Q-Learning and SARSA. This paper presents FAQ, a flexible FPGA-based accelerator for the Q-Learning algorithm. The architecture of the accelerator can be configured in multiple ways, like adjusting the bit width of Q-values or changing the number of pipeline stages. The evaluation shows that FAQ achieves 249% higher throughput than state-of-the-art FPGA implementations while decreasing DSP and BRAM utilization. Additionally, a software-configurable environment was implemented, and the whole system was tested on an Ultra96-V2 development board utilizing the PYNQ framework. Compared to a CPU implementation, FAQ is more than 13 times faster, including communication overhead caused by transferring the environment onto the FPGA and reading the resulting Q-table. © 2022 IEEE. |
Beschreibung: | Conference of 33rd IEEE International Conference on Application-Specific Systems, Architectures and Processors, ASAP 2022 ; Conference Date: 12 July 2022 Through 14 July 2022; Conference Code:183622 |
ISBN: | 9781665483087 | ISSN: | 1063-6862 | DOI: | 10.1109/ASAP54787.2022.00026 | Externe URL: | https://www.scopus.com/inward/record.uri?eid=2-s2.0-85140964922&doi=10.1109%2fASAP54787.2022.00026&partnerID=40&md5=d0d93433951d00e19634419bf07a4e81 |
Show full item record