Reinforcement learning based process optimization and strategy development in conventional tunneling

Georg H. Erharter*, Tom F. Hansen, Zhongqiang Liu, Thomas Marcher

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Reinforcement learning (RL) - a branch of machine learning - refers to the process of an agent learning to achieve a certain goal by interaction with its environment. The process of conventional tunneling shows many similarities, where a geotechnician (agent) tries to achieve a breakthrough (goal) by excavating the rockmass (environment) in an optimum way. In this paper we present a novel RL based framework for strategy development for conventional tunneling. We developed a virtual environment with the goal of a tunnel breakthrough and with a deep Q-network as the agent's architecture. It can choose from different excavation sequences to reach that goal and learns to do so in an economical and safe way by getting feedback from a specially designed reward system. Result analyses show that the optimal policies have great similarities to current practices of sequential tunneling and the framework has the potential to discover new tunneling strategies.

Original languageEnglish
Article number103701
JournalAutomation in Construction
Publication statusPublished - Jul 2021


  • Conventional tunneling
  • Excavation sequences
  • Machine learning
  • Reinforcement learning
  • Tunnel excavation strategy

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Civil and Structural Engineering
  • Building and Construction

Cite this