Bridging Sub-Tasks to Long-Horizon Task in Hierarchical Goal-Based Reinforcement Learning
- Title
- Bridging Sub-Tasks to Long-Horizon Task in Hierarchical Goal-Based Reinforcement Learning
- Authors
- 이강복
- Date Issued
- 2024
- Publisher
- 포항공과대학교
- Abstract
- Hierarchical goal-based reinforcement learning (HGRL) is a promising approach to learning a long-horizon task by decomposing it into a series of subtasks of achieving subgoals in a shorter horizon. However, the performance of HGRL crucially depends on the design of intrinsic rewards for these subtasks: as frequently observed in prac- tice, short-sighted reward designs often lead the agent into undesirable states where the final goal is no longer achievable. One potential remedy to the issue is to provide the agent with a means to evaluate the achievability of the final goal upon the com- pletion of the subtask; yet, evaluating this achievability over a long planning horizon is a challenging task by itself. In this work, we propose a subtask reward scheme to bridge the gap between the long-horizon primary goal and short-horizon subtasks by incorporating look-ahead information towards the next subgoals. We provide an extensive empirical analysis in MuJoCo environments, demonstrating the importance of looking ahead to the subsequent sub-goals and improving the proposed framework applied to the existing HGRL baselines.
- URI
- http://postech.dcollection.net/common/orgView/200000734373
https://oasis.postech.ac.kr/handle/2014.oak/123372
- Article Type
- Thesis
- Files in This Item:
- There are no files associated with this item.
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.