Advanced Energy Management Strategies for Plug-In Hybrid Electric Vehicles via Deep Reinforcement Learning

Amr Mousa*, Gerhard Benedikt Weiß

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

Abstract

Plug-in Hybrid Electric Vehicles (PHEVs) achieve significant fuel economy by utilizing advanced energy management strategies in controlling the power distribution decision in real-time. Traditional heuristic approaches bring no additional benefits, including efficiency and development cost, considering the increasing complexity in control objectives. This paper extends a previous study of the same problem (RL) and vehicle topology to develop a Reinforcement Learning agent by investigating the performance of state-of-the-art algorithms, such as Rainbow-DQN with its variants, PPO and A3C, against the baseline rule-based and Dynamic Programming (DP) strategies. The developed RL agent is optimizing challenging control objectives such as fuel economy, vehicle drivability and driver comfort. The Rainbow-DQN is studied separately to optimize the agent compared to all the algorithm variants and after wards, the best performing variant is compared to tuned PPO and A3C agents. Proper evaluation criteria is defined and the concerned agents are tested with nine different scenarios to examine the generalization capabilities and performance robustness. The results revealed that the A3C agent surplussed both the PPO and the Rainbow-DQN achieving a maximum performance of 98.43% of the DP with a robustness of 97.32% ± 0.78 for the other cycles and an average of 177.7 sec for each engine start compared to 96.3 sec for the rule-based approach. Furthermore, as a future work, the paper investigated and proposed a cloud-based training concept for automated scaled-up training, evaluation and deployment of RL policies for the (P)HEVs of the future.
Original languageEnglish
Article number7109
Number of pages15
JournalSAE Technical Papers
Volume2022
Issue number01
Early online date22 Dec 2022
DOIs
Publication statusPublished - 22 Mar 2023
EventSAE 2022 Intelligent and Connected Vehicles Symposium - Shanghai, Virtuell, China
Duration: 22 Sept 202223 Sept 2022

Keywords

  • Plug-in Hybrid Electric Vehicles
  • Energy Management Strategy
  • Reinforcement Learning
  • Proximal Policy Optimization (PPO)
  • Asynchronous Advantage Actor Critic (A3C)
  • Deep Q-Network (DQN)
  • Cloud-based Distributed Training

Cite this