A unified framework for train driver behavior recognition and riskassessment from a human–machine interaction perspective

Train driver behavior is pivotal for rail safety. Existing onboard log analysis fails to capture non-contact behaviors like ‘Pointing and Calling,’ while visual methods based on pre-defined categories lack generalizability for complex interaction patterns and quantitative assessment of full-process behavioral sequence risks. To address these issues, this study proposes a unified framework for driver behavior recognition and risk assessment from a Human–Machine Interaction (HMI) perspective. First, we constructed a fine-grained train driver HOI dataset containing ‘action–device’ relationships through driving simulation experiments. Second, targeting the spatially dense layout of display and control devices on the driving console, we designed the DriverHOI recognition model based on Graph Parsing Neural Networks (GPNN). By fusing 3D hand poses and device geometric priors, the model achieves joint inference of driving actions and interaction objects. Furthermore, a Bayesian Network-based risk assessment model was built to quantify behavioral sequence risks by modeling error propagation mechanisms. Experimental results demonstrate that DriverHOI performs excellently in complex interaction scenarios with densely distributed devices, achieving an overall accuracy of 94.0%, an action recognition accuracy of 98.3%, and a Top-1 device selection accuracy of 94.6%. Meanwhile, the risk assessment model effectively validates error propagation patterns in typical driving scenarios. This framework achieves systematic analysis from visual perception to risk quantification, providing a novel technical pathway for train driver behavior risk assessment.

Kun Wang , Haifeng Bao , Weining Fang. A unified framework for train driver behavior recognition and risk assessment from a human–machine interaction perspective. Reliability Engineering and System Safety.https://github.com/wang-10086/DriverHOI