Author(s)

MingFu Zheng¹, Yao Mo², Ying Zhang², Yin Bo^1*, Rongwen Chen¹

Affiliation(s)

¹Changjiang Survey, Planning, Design and Research Co., Ltd., Wuhan 430000, Hubei, China.

²Shiyan City Water Source Co., Ltd., Shiyan 442000, Hubei, China.

Corresponding Author

Yin Bo

ABSTRACT

In complex geological conditions, such as variable rock hardness, Tunnel Boring Machines (TBMs) frequently suffer from trajectory deviations. Traditional control strategies, based on operator experience or simplified mechanical models, often lack the necessary adaptability to handle the non-linearity and randomness of surrounding rock, making precise and efficient trajectory correction difficult. This study introduces Deep Reinforcement Learning (DRL) to address the challenges of robustness and self-adaptation in TBM posture control. We first establish a high-fidelity TBM-geology interaction simulation environment, defining a multi-dimensional state space and action space that includes critical information such as posture deviation, thrust distribution, and geological parameters. To balance excavation accuracy and efficiency, we design a multi-objective composite reward function that incorporates penalties for posture deviation, rewards for advance rate, and constraints for control input smoothness. For policy learning, we improve DRL algorithms suitable for continuous action spaces and introduce a Prioritized Experience Replay mechanism to enhance the policy's stability under abrupt environmental changes. Simulation results demonstrate that, compared to conventional PID control, the DRL-based autonomous correction strategy achieves an improvement of over 30% in posture control accuracy and a reduction of over 20% in response time to sudden disturbances. This research validates the significant advantages of DRL in handling the high-dimensional, highly delayed, and non-linear control challenges inherent in TBM excavation, providing an innovative theoretical framework and technical support for the autonomous and intelligent development of TBM operations.

KEYWORDS

TBM; Deep Reinforcement Learning (DRL); Posture control; Autonomous correction; Complex geology; Multi-objective reward

CITE THIS PAPER

MingFu Zheng, Yao Mo, Ying Zhang, Yin Bo, Rongwen Chen. Autonomous trajectory correction control strategy for TBM in complex geology: a deep reinforcement learning approach. World Journal of Engineering Research. 2025, 3(5): 60-73. DOI: https://doi.org/10.61784/wjer3062.

REFERENCES

[1] Kim J I, Fischer M, Suh M J. Formal representation of cost and duration estimates for hard rock tunnel excavation. Automation in Construction, 2018, 96: 337–349.

[2] Yue Z, Wang Y, Duan J, et al. TS2Vec: Towards universal representation of time series. Proceedings of the AAAI Conference on Artificial Intelligence, 2022, 36(8): 8980–8987.

[3] Hasselt H V, Guez A, Silver D. Deep reinforcement learning with double Q-learning. In: Proceedings of the 30th AAAI Conference on Artificial Intelligence. Phoenix, Arizona, USA: AAAI Press, 2016: 2094–2100.

[4] Kober J, Bagnell J A, Peters J. Reinforcement learning in robotics: A survey. International Journal of Robotics Research, 2013, 32(11): 1238–1274.

[5] He L, Xu Z G, Jia Y, et al. TOC reward function for reconstructing multi-target trajectories using deep reinforcement learning. Computer Applications Research, 2020, 37(6): 1626–1632.

[6] Liu J W, Gao F, Luo X L. A review of deep reinforcement learning based on value function and policy gradient. Chinese Journal of Computers, 2019, 42(6): 1406–1438.

[7] Zhang A J. Research on fuzzy control of shield tunneling posture construction parameters in soft upper and hard lower strata. Journal of Railway Science and Engineering, 2018, 15(11): 2920–2927.

[8] Guo D, Li J, Jiang S H, et al. Intelligent assistant driving method for tunnel boring machine based on big data. Acta Geotechnica, 2022, 17(4): 1019–1030.

[9] Liu B, Wang Y, Zhao G, et al. Intelligent decision method for main control parameters of tunnel boring machine based on multi-objective optimization of excavation efficiency and cost. Tunnelling and Underground Space Technology, 2021, 116: 104054.

[10] Li N B. Intelligent decision-making method for TBM tunneling parameters and attitude based on rock mass geological information perception. Jinan: Shandong University, 2024.

[11] Fan L, Yuan J, Niu X, et al. RockSeg: A novel semantic segmentation network based on a hybrid framework combining a convolutional neural network and transformer for deep space rock images. Remote Sensing, 2023, 15(16): 3935.

[12] Xie W Q, Zhang X P, Liu X L, et al. Real-time perception of rock-machine interaction information in TBM tunnelling using muck image analysis. Tunnelling and Underground Space Technology, 2023, 136: 105096.

[13] Gong Q M, She Q R, Wang J M, et al. Influence of layered rock masses of different thicknesses on TBM excavation. Chinese Journal of Rock Mechanics and Engineering, 2010, 29(7): 1442–1449.

[14] Farrokh E, Rostami J. Correlation of tunnel convergence with TBM operational parameters and chip size in the Ghomroud tunnel, Iran. Tunnelling and Underground Space Technology, 2008, 23(6): 700–710.

[15] Sun J S, Lu W B, Su L J, et al. Identification of rock mass quality indicators based on TBM tunneling parameters and slag characteristics. Chinese Journal of Geotechnical Engineering, 2008, 30(12): 1847–1854.

[16] Yang Z. Research and system implementation of TBM slag morphology recognition based on deep learning. Wuhan: Huazhong University of Science and Technology, 2022.

[17] Jing L J, Zhang N, Yang C. Development and trends of TBM and its construction technology in China. Tunnel Construction, 2016, 36(3): 331–337.

[18] Xu Z H, Wang C Y, Zhang J Y, et al. Geological perception and rock-machine digital twin in TBM tunnel excavation: methods, current status and digital intelligent development direction. Journal of Applied Basic and Engineering Sciences, 2023, 31(6): 1361–1381.

[19] Deng M J, Tan Z S. Analysis and research on TBM cluster excavation technology for extra-long tunnels. Tunnel Construction , 2021, 41(11): 1809–1826.