Reinforcement Learning Rocket League Agent
PPO-based reinforcement learning agent using reward shaping and curriculum learning.
Nov 2024 - Dec 2024 (2 months)
Overview
A deep reinforcement learning project that trains an agent to play Rocket League, a physics-based vehicular soccer game. The system implements a PPO (Proximal Policy Optimization) architecture to teach the AI agent complex 3D movement, ball control, positioning, and basic-to-intermediate strategy, effectively translating high-dimensional games states into meaningful actions.
The project uses curriculum learning to teach the agent basic controls and strategies first, and building upon that knowledge to proceed to more advanced game mechanics and tactics. Custom functions were used for reward shaping to direct the agent in certain learning directions. Using RLGym and Stable Baseline 3, the agent was trained over tens of millions of timesteps with checkpoint models, final models, and best models being saved for each new training session.
The database logging serves as a final visual of the agent's training by defining major changes in the curriculum learning goal to show how the agent maximized its rewards over time, along with various useful statistics of the game such as goals scored, ball touches, average distance from the ball, and others. Once the project is finalized, I plan on accreting my work and displaying the agent's various tracked metrics over time.
Challenges
- •Implementing curriculum learning transitions without disrupting training stability
- •Handling high-dimensional state spaces while maintaining training efficiency
- •Balancing exploration vs exploitation during training