Obsidian/Zotero/001_artiklid/Road Safety Indicators/liucollisionriskpredictiontakeover2025.md

---

category: literaturenote citekey: liucollisionriskpredictiontakeover2025 title: "Collision risk prediction and takeover requirements assessment based on radar-video integrated sensors data: A system framework based on LLM" authors: "Liu, Qingchao; Yu, Ruohan; Cai, Yingfeng; Yuan, Quan; Wei, Henglai; Lv, Chen" year: 2025 date: 2025-00-00 2025 doi: 10.1016/j.aap.2025.108041 publication: "Accident Analysis & Prevention" url: "https://doi.org/10.1016/j.aap.2025.108041" zotero_key: TTGKUWQG zotero_storage: 3TTXFNFW collections: transpordi_planeerimine folder: 001_artiklid/Road Safety Indicators firstAuthor: "Liu, Qingchao"

status: converted

Contents lists available at ScienceDirect

Accident Analysis and Prevention

journal homepage: www.elsevier.com/locate/aap

Collision risk prediction and takeover requirements assessment based on radar-video integrated sensors data: A system framework based on LLM

Qingchao Liu a ,∗ , Ruohan Yu a , Yingfeng Cai a , Quan Yuan b , Henglai Wei c , Chen Lv d

• ^a Automotive Engineering Research Institute, Jiangsu University, Zhenjiang, China
• ^b School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China
• ^c School of Transportation Science and Engineering, Beihang University, Beijing, China
• ^d School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore

A R T I C L E I N F O

Keywords: Large language model Driving takeover Collision risk prediction Interpretability of artificial intelligence Radar-video integrated sensors data

A B S T R A C T

There are safety risks when drivers take over the control of autonomous driving vehicles, and reducing unnecessary takeovers is essential to improve driving safety. This study seeks to develop an interpretable system framework for collision risk prediction and takeover requirements analysis (CPTR-LLM) utilizing a large language model (LLM). The model's inference performance is enhanced through the collection of extensive perception data and the design of a two-stage training strategy, reasoning chain framework, and an error detection and correction mechanism. In terms of collision risk prediction, the experimental results show that the accuracy of CPTR-LLM can reach 0.88. The Cross-sectional-autoregressive-distributed lag (ARDL) model and Augmented Mean Groups (AMG) confirm the reliability of the model's predictive performance by revealing the association between different variables and collision risk. Regarding takeover requirement analysis, CPTR-LLM accurately comprehends the characteristics of the pre-takeover scene and comprehensively assesses the takeover requirement level in conjunction with collision risk, thereby effectively reducing unnecessary takeovers in simple driving scenarios and unsafe takeovers in scenarios with multiple moving targets. Overall, the findings of this paper offer significant insights into the application and takeover requirements of LLM in the domain of road safety.

1. Introduction

Conditional autonomous driving vehicles require the driver to assume control of the vehicle promptly to prevent collisions in complex scenarios or emergencies (Deng et al., 2024). However, frequent or unnecessary takeover requests may lead to three key issues: first, unnecessary takeovers increase the likelihood of driver errors or accidents (Zhu et al., 2023); secondly, excessive takeover requirements significantly increase the driver's cognitive load, leading to a decline in takeover performance (Melnicuk et al., 2021); finally, redundant takeover prompts from the system may result in a crisis of trust (Jin et al., 2021). Therefore, avoiding unnecessary takeovers is crucial for improving the safety of conditional autonomous driving vehicles.

The primary objective of a takeover is collision avoidance. Existing research primarily focuses on collision risk as a critical condition for the emergence of the requirements for takeover (Zhu et al., 2023; Ou et al., 2021). Therefore, predicting collision risk is crucial for assessing takeover requirements. The research field related to collision risk prediction is currently well-established. Common methods for collision risk prediction include statistical approaches and artificial intelligence models. The former offers convincing interpretability but suffers from poor prediction accuracy, whereas the latter provides the opposite. Consequently, researchers have attempted to combine the two to achieve accurate and credible predictions (Jin et al., 2023). These studies provide accurate collision risk prediction methods for judging takeover requirements. However, their methods still require further analysis and judgment in conjunction with specific real-world traffic scenarios. Additionally, these methods are unable to autonomously generate detailed analyses and specific recommendations in natural language. At present, there is no such collision risk prediction and take-over requirement judgment system that can autonomously analyze the scene characteristics, predict the collision risk and comprehensively judge the take-over requirements, and finally generate easy-to-understand instructions and driving assistance suggestions.

Significant progress has been made in the field of large language models (LLMs) over the past few months (Zhang et al., 2024b). In comparison with conventional traffic-related models, which are frequently constrained by limited interpretability, poor adaptability to

^∗ Corresponding author. E-mail address: lqc@ujs.edu.cn (Q. Liu).

varying scenes, inadequate environmental perception, and data biases, large language models (LLMs) capitalize on their extensive knowledge base as well as their advanced contextual understanding and reasoning capabilities to facilitate real-time analysis of dynamic traffic scenarios. These models are also capable of human-like reasoning and decision-making in rare and unpredictable long-tail situations, thereby offering robust solutions for collision risk prediction and the development of human-like autonomous driving systems in real-world traffic environments (Xu et al., 2025). One of the most promising LLMs is GPT-4, which not only supports multi-modal input but also demonstrates more reliable performance in handling complex traffic tasks, such as predicting risk levels in traffic images (Driessen et al., 2024) and understanding drivers' intentions (Liao et al., 2024). However, a significant limitation of GPT-4 is its requirement for paid access via the API. In addition, reasoning judgments using GPT-4's API may also be affected by network conditions and hackers' attacks. In the event of a failure, this could result in an unavoidable collision. To address this issue, LLaMA3, one of the most advanced open-source multimodal LLMs, not only possesses excellent inference capabilities but also supports local deployment and professional fine-tuning by users. Inspired by this, we propose a collision risk prediction and takeover requirement assessment model (CPTR-LLM) based on LLaMA3. To enhance the model's reasoning performance and reliability, we have carefully designed a reasoning chain framework and incorporated error detection and correction mechanisms. For the training and development of CPTR-LLM, we introduce a two-stage training strategy and have collected a substantial amount of radar-video integrated data.

Previous studies have predominantly utilized officially reported datasets or simulation data to train autonomous driving vehicle collision risk prediction models (Liu et al., 2024; Kar et al., 2023). However, these data are either insufficient or not realistically representative. Although an increasing number of studies have begun utilizing sensors to acquire micro-motion data of vehicles in recent years (Singh et al., 2024), different sensors have inherent limitations that affect the quality of the collected data. For instance, radar cannot provide information such as the vehicle's ID, and cameras are influenced by the angle of acquisition. The multi-sensor system of autonomous vehicles can comprehensively capture micro-traffic data from the surrounding environment through data fusion technology, but it lacks the capability to obtain macro-level traffic information. As an emerging traffic data road measurement device, the radar-video integrated sensors integrate millimeter-wave radar and cameras, offering not only excellent environmental adaptability but also the capability to simultaneously capture micro-data on vehicle movement and macro-data on traffic flow operations. This provides more comprehensive and accurate data support for CPTR-LLM analysis and understanding of road traffic safety risks. This study primarily aims to develop a framework for an integrated system that predicts collision risks and assesses takeover requirements using radar-video integrated sensors data. To the best of our knowledge, this is the first large language model (LLM) applied to the analysis of takeover requirement in autonomous driving. The primary contributions of this paper are threefold:

• 1. An autonomous vehicle takeover requirement analysis system has been developed based on LLM, offering new insights into reducing unnecessary takeovers and enhancing the safety of conditionally automated driving. This expands the application of LLMs in the field of road safety.
• 2. A multi-stage training strategy is introduced in CPTR-LLM, ensuring that the model comprehends the essential characteristics of collision risks and is proficient in predicting varying degrees of collision risk. This provides a reference for enhancing LLMs' understanding of complex driving scene features in the traffic domain.
• 3. A reasoning chain framework is designed for CPTR-LLM, effectively linking and integrating modules for scene feature understanding, collision risk prediction, and takeover requirement judgment. This enables logical reasoning for complex driving tasks, improving the model's accuracy in risk prediction and the reliability of its decision-making.

The remainder of the paper is organized as follows: Section 2 reviews the relevant literature. Section 3 details the modeling and evaluation methodology. Section 4 describes the data sources. Section 5 compares model performance, conducts ablation experiments, and presents empirical studies. Section 6 demonstrates the distribution of collision risks and takeover requirements under different scenarios, discussing the practical application potential of the model. Finally, the paper concludes with a summary

2. Literature review

2.1. Research related to takeover

In recent years, extensive research has been conducted on the takeover challenges associated with autonomous driving vehicles. These studies have primarily focused on exploring the effects of various factors on takeover quality, including environmental, system, and driver-related factors. Environmental factors include traffic flow information (Gold et al., 2016) and weather conditions (Li et al., 2018); system factors are primarily related to the design of the autonomous driving system, encompassing takeover reservation time (Li et al., 2022), takeover mode (Yoon et al., 2019), takeover request mode (Ou et al., 2021), and duration of autonomous driving (Portron et al., 2024), among others; driver-related factors pertain to the driver's physiological state (Deng et al., 2024), psychological state (Sanghavi et al., 2023), individual differences (Lin et al., 2020), and driving style of the driver (Zhang et al., 2024a). Furthermore, more detailed and in-depth investigations have been conducted on specific influencing factors. For instance, Wang et al. (2022) explored driver takeover performance under different auditory takeover request modes, focusing on the effects of various vocal prompts. Melnicuk et al. (2021) examined the influence of drivers' psychological states, with particular attention given to the relationship between cognitive load and takeover performance. Additionally, several studies have attempted to develop more advanced models to accurately predict takeover quality. These models are primarily based on advanced machine learning or deep learning techniques. Du et al. (2020) compared the prediction performance of six machine learning models using driver physiological data and driving environment data. The results showed that within a 3-s time window, the Random Forest (RF) classifier achieved the highest prediction accuracy. Building on this, Chen et al. (2024) further introduced driver situational awareness variables and used the Extreme Gradient Boosting (XGBoost) model to predict takeover time, employing Shapley Additive Explanations (SHAP) to explain the impact of different variables on takeover time. Rangesh et al. (2021) developed and trained a takeover time prediction model based on Long Short-Term Memory (LSTM) neural networks to assess driver takeover performance in different tasks.

The aforementioned studies provide important references for enhancing takeover safety. However, most experiments have been conducted based on predefined takeover conditions, without fully considering the takeover requirements in real-world scenarios. Currently, the exploration of pre-takeover scene features and takeover requirements remains insufficient, and further in-depth research in this area is urgently needed. Moreover, although existing takeover analysis models can accurately predict driver takeover performance and support proactive intervention measures, these models still have limitations in alleviating driver situational awareness load and takeover pressure. In contrast, the model proposed in this study enables effective determination of takeover requirements through an in-depth understanding of environmental states and precise collision risk assessment. It can avoid unnecessary takeovers and provide intuitive, easily understandable textual assistance to the driver, fundamentally improving the overall safety of conditional autonomous driving vehicles.

2.2. Collision risk prediction model

Models for collision risk prediction can generally be categorized into two types: statistical models and artificial intelligence models. Common statistical models include count data models (e.g., negative binomial models Bhowmik et al., 2022), discrete outcome models (e.g., binary logit models Islam et al., 2023), statistical regression models (e.g., logistic regression models Yuan et al., 2021) and probabilistic graph models (e.g., Bayesian networks Song et al., 2021). These models are characterized by strong interpretability and the ability to account for unobserved heterogeneity. However, they are inherently limited in both accuracy and efficiency. Therefore, researchers have increasingly focused on artificial intelligence models, which primarily consist of machine learning and deep learning approaches. Yang et al. (2021) demonstrated that the XGBoost machine learning model more accurately captures the impact of relevant factors on collision severity compared to statistical models. Liu et al. (2022) further demonstrated the portability of machine learning models by modeling collision risk using TransferBoost. Deep learning-based collision risk prediction models mainly include LSTM (Li et al., 2020), deep clustering networks (Li et al., 2021), and Graph Convolutional Networks (GCN) (Hu et al., 2025) etc., which can achieve more accurate risk predictions by capturing high-dimensional spatiotemporal features of driving scenarios. Recently, deep reinforcement learning (DRL) frameworks, such as Deep Q-learning (DQN) (Tseng et al., 2020), have provided key technological support for autonomous driving risk reasoning and decision-making. However, these models still face challenges such as high data requirements and slow convergence speeds. Additionally, these artificial intelligence models generally suffer from poor interpretability, which prevents them from providing valuable insights related to collision mechanisms.

Recent research has begun combining statistical and artificial intelligence models to construct interpretable frameworks for collision risk prediction. For example, Jin et al. proposed a hybrid method for predicting collision risks in highway tunnels, which first analyzes the impact mechanisms of different variables on collision risks using a random parameter Logit model, and then employs a deep cross-network model to achieve real-time, accurate risk prediction (Jin et al., 2023). Yang et al. based on traffic flow data collected from cross-area freeways, used the Random Forest algorithm to identify key features and introduced a Bayesian logistic regression model to establish statistical relationships between input variables and accident risks (Yang et al., 2022). However, these hybrid frameworks are often quite complex, and the statistical models involved may lead to a decrease in inference prediction efficiency. Furthermore, these models are typically designed for specific driving scenarios, and their generalization performance in other scenarios is limited. Due to these two major limitations complexity and scenario dependence — the hybrid methods combining statistical models and machine learning do not yet possess the potential for application in real-time driving scenarios.

To address the shortcomings of existing collision risk prediction models, this study adopts a multi-stage training approach to enhance the model's learning and understanding of collision risk features. Additionally, the Low-Rank Adaptation (LoRA) lightweight technique is introduced to improve the model's prediction efficiency. A reasoning chain framework is also designed to guide the model in conducting step-by-step reasoning and analysis based on real-time driving scene features. By leveraging the human-like learning and reasoning abilities of large language models (LLMs), CPTR-LLM's scene understanding capabilities are enhanced, improving the model's adaptability to complex dynamic scenarios. Furthermore, the reasoning results of each step in the model can be converted into easily understandable text outputs, providing excellent interpretability.

2.3. Data sources for road safety analysis

Collision risk analyses are typically conducted using a reactive approach, in which vehicles must first be involved in a collision before their mechanisms are analyzed. Officially reported datasets have traditionally been regarded as a reliable source (Liu et al., 2024; Tobin et al., 2021). However, these datasets inevitably suffer from well-known shortcomings, such as poor data quality, incomplete data, and underreporting of collisions. With the advancement of intelligent transportation technologies, increasingly sophisticated road test devices are being applied to real-time collision risk analysis. Accident data were collected by Peng et al. (2020) through multiple sets of loop detectors, providing a basis for accurate real-time collision risk prediction. Zhao et al. (2024) utilized large floating vehicle trajectory data to explore the association between travel patterns and collisions. Wang et al. (2024) and Xing et al. (2020) used surveillance videos and unmanned aerial vehicles to extract vehicle trajectories, respectively, and thus analyze actual hazardous traffic conditions. However, loop detection devices are unable to capture data from adjacent lanes and microvehicle data; floating vehicles fail to provide temporally and spatially evolving dynamics; video detection technology is limited by building height; and drone monitoring effectiveness is susceptible to object occlusion. Simulation techniques or driving simulators appear to overcome these shortcomings, but they lack the realism of actual magnetic field dynamics and are not representative of conflict situations in realworld scenarios (Kar et al., 2023). In such cases, real-world vehicle data collected by onboard multi-sensors are widely regarded as more representative and realistic, as they effectively integrate the complementary advantages of multiple sensing modalities. The autonomous driving vehicle dataset from Ford's subsidiary, Argose, is derived from a fleet of autonomous driving vehicles equipped with multiple LIDARs and cameras (Singh et al., 2024). Furthermore, radar-video integrated sensors, as a new type of road test equipment, have also been equipped with multi-sensor technology and are capable of collecting data over a broader spatial range. This device not only enables the acquisition of microscopic vehicle operation data but also facilitates the capture of macroscopic traffic.

3. Research methodology

3.1. Two-stage training method

CPTR-LLM is fine-tuned using a large volume of traffic scenario description texts and delineated conflict types and levels (Fig. 1). In the first step, CPTR-LLM is trained using the first-step training dataset, which consists of paired traffic scene descriptions and corresponding conflict types. This training enables the model to discern the characteristics of conflict and non-conflict events and to grasp the fundamental differences between these two categories. In the second step, the model is fine-tuned using the second-step dataset. This dataset is all conflict data, consisting of paired traffic scene descriptions and conflict levels, which aims to enhance the model's proficiency in recognizing conflict events, understanding the nuances of varying conflict levels, and facilitating accurate predictions of collision risk. Through this two-step fine-tuning process, CPTR-LLM improves its understanding of conflict types in diverse traffic scenarios, leading to precise predictions of specific conflict classes (collision risk).

3.2. RC framework

To further enhance the performance of collision risk prediction, a reasoning chain (RC) framework was carefully designed to consistently and comprehensively capture the essential details of input information, thereby enabling structured reasoning. Given that large language models are inherently capable of performing reasoning over textual inputs, the data collected by the radar-camera fusion device were

Fig. 1. Two-stage training process of CPTR-LLM.

transformed into descriptive language representations, which provided the model with specific information about surrounding traffic scenarios and enriched the RC prompts. Based on these inputs, the model can be guided to systematically analyze the collision risks and takeover requirements of autonomous vehicles through the five steps illustrated in Fig. 2:

Step1. Vehicle operation state assessment: The model evaluates the speed, acceleration, and steering angle of the autonomous vehicle and surrounding objects, reasoning about their respective movements, such as whether they are driving at high speed or low speed, whether they are accelerating slightly or decelerating greatly, and whether they are driving in a straight line or making a turn.

Step2. Relative motion state analysis: Based on the previous step, the model further assesses the relative motion between the autonomous vehicle and surrounding objects, such as their relative directions, distances, and movement trends.

Step3. Surrounding traffic flow state analysis: The model examines the surrounding traffic flow's volume, density, and speed, interpreting these indices to enhance understanding of the surrounding macro traffic environment.

Step4. Conflict type and collision risk analysis: The model synthesizes information from the macro traffic environment and micro vehicle operation to determine conflict types and collision risk levels for the autonomous vehicle.

Step5. Takeover requirements analysis: Based on the collision risk level and surrounding traffic scene, the model determines the vehicle's takeover requirements, providing reasons and specific decision-making suggestions to assist the driver in safely taking control.

By sequentially executing the steps outlined above, CPTR-LLM is progressively guided to develop a deeper understanding of the surrounding driving scene features, thereby establishing logical relationships among scene comprehension, collision risk prediction, and takeover requirement analysis. This process enhances the coherence and consistency of the model's reasoning, thereby improving its reasoning capacity and enabling more sophisticated reasoning and accurate predictions in complex traffic scenarios. The reasoning processes and outcomes at each step are capable of being translated into textual outputs, thereby significantly enhancing the model's interpretability and assisting drivers or researchers in comprehending the model's inference logic. Furthermore, by examining the outputs at each stage, potential reasoning errors can be identified, facilitating timely adjustments and the enhancement of overall model performance.

3.3. Error detection and correction mechanism

Given the sensitivity of large language models (LLMs) to input data and the possibility of erroneous results, an Error Detection and Correction (EDC) mechanism is incorporated into CPTR-LLM. This mechanism, structured around the model's reasoning chain, is designed to evaluate the accuracy of each reasoning step, thereby preventing the generation of unreasonable decision suggestions and continuously enhancing the model's analytical performance. Additionally, due to the inherent uncertainty and instability of LLM responses, the standardization of output formats is verified at each step to ensure the interpretability and analyzability of the results. The operational workflow of this mechanism is illustrated in Fig. 3 and can be divided into the following four stages:

Step 1. Based on the input vehicle motion data, surrounding target motion data, and traffic flow data, the model's interpretation of sensor data is evaluated for accuracy. The evaluation accuracy of relative motion states is further verified by calculating the relative distance, orientation, and velocity between the vehicle and surrounding objects.

Step 2. The accuracy of the model's risk assessment is evaluated by calculating four conflict indicators: Modified Time to Collision (MTTC), Post Encroachment Time (PET), Stopping Distance Proportion (PSD), and Collision Avoidance Deceleration Rate (DRAC).

Step 3. The model's output decisions regarding speed, acceleration, and steering angle are extracted. In conjunction with the vehicle's motion data and surrounding target data, the rationality and safety of the model's output decisions are checked.

Step 4. Detection results from each step are compiled and input into the error analysis module to support further refinement of the model's reasoning capabilities.

The corresponding detection criteria for each stage are presented in Table 1. As the experimental validation was conducted on the campus of Jiangsu University, the speed detection refers to the vehicle speed limits specified in the ''Motor Vehicle Management Regulations'' of Jiangsu University, while the detection of acceleration, steering angle, and relative distance follows the vehicle dynamics standards defined by the Society of Automotive Engineers (SAE) (Green, 2013) and the safety performance standards for automated systems (International, 2021). Traffic flow detection refers to the Highway Capacity Manual (HCM) (Manual, 2000), and collision risk detection is based on a comprehensive evaluation using multiple conflict indicators (specific details in Section 4.3). The detection of takeover decisions follows the evaluation procedure for takeover decision strategies proposed by Naujoks et al. (2018). It should be noted that we do not detect the model's output for takeover requirement, as there is currently limited literature on this topic and a lack of evaluation standards in this area. The key goal of this paper is to leverage the LLM's rich knowledge base and excellent reasoning capabilities to guide the model in analyzing and judging takeover requirement, and, through its interpretability, gain insightful knowledge regarding takeover requirement assessment.

Fig. 2. RC framework of CPTR-LLM.

3.4. Parameter efficient fine-tuning with lora

To improve the fine-tuning efficiency of CPTR-LLM, the Low-Rank Adaptation (LoRA) technique was introduced (Hu et al., 2021), aiming to approximate the incremental parameters obtained from fullparameter fine-tuning of LLM using fewer training parameters (Fig. 1). Specifically, by freezing the original LLM parameters, LoRA employs a reduced number of parameters to conduct low-rank approximation training on the matrix. Assuming that the parameter weight matrix obtained by pre-training is ^{0 ∈ R}d×^d , the weight is updated according to the following formula in the fine-tuning stage:

$$W = W_0 + \Delta W \tag{1}$$

where, and represent the parameter weight matrix and the incremental weight matrix obtained after fine-tuning, respectively. LoRA decompresses into two matrices A and B, with a reduced number of parameters when updating the weights:

$$\Delta W = BA$$ (2)

Table 1 Detection criteria for model generation responses.

Variable		Threshold	Description
		<4.17	Low
	Speed (m/s)	≥4.17 and <8.33	Medium
		≥8.33	Fast
		>0 and <2	Small acceleration
		≥2	Large acceleration
Motion variables of the	Acceleration (m/s2 )	0	Keep zero
self-vehicle and surrounding		>−2 and <0	Small deceleration
moving objects		≤ − 2	Large deceleration
		≥5 and <30	Small right turn
		≥30	Large right turn
	◦ Steering angle ( )	>−5 and <5	Straight line movement
		≤ − 5 and >−30	Small left turn
		≤ − 30	Large left turn
		x_diff=0 and y_diff>0	Front
		x_diff=0 and y_diff<0	Back
		x_diff<0 and y_diff=0	Left
		x_diff>0 and y_diff=0	Right
	Relative direction (m)	x_diff<0 and y_diff>0	Front left
		x_diff>0 and y_diff>0	Front right
		x_diff<0 and y_diff<0	Back left
Relative motion variable		x_diff>0 and y_diff<0	Back right
		<5	Small
	Relative Distance (m)	≥5 and <20	Medium
		≥20	Large
		Relative speed > 0	Away
	Movement trend Speed (m/s)	Relative speed = 0	Unchanged
		Relative speed < 0	Close
		<200	Small
	Volume (veh/h)	≥200 and <500	Medium
		≥500	Large
		<10	Small
Traffic flow	Density (veh/km)	≥10 and <30	Medium
		≥30	Large
		<20	Small
	Speed (km/h)	≥20 and <30	Medium
		≥30	Large
Crash risk		See Section 4.3
		Accelerates under a large acceleration	Unreasonable acceleration
	Decision rationality	Large turns at high speeds	Unreasonable heading change
Takeover decision suggestions	Decision security	Accelerates when closing to a surrounding move object	Collision risk

where, ∈ R× , both ∈ Rd×^r and ∈ R× are LoRA lowrank adaptive weight matrices and ≪ min(, ). Therefore, when fine-tuning the LLM, only the low-rank matrices A and B require optimization. Consequently, the total number of parameters fine-tuned changes from the original × to ( + ) × and has:

$$(d+k) \times r \ll d \times k \tag{3}$$

Therefore, for an LLM with a given input x ∈ R^d , the output after fine-tuning using LoRA is expressed as:

$$h = Wx = W_0x + BAx \tag{4}$$

3.5. Conflict evaluation indicators

Given the safety concerns and technical limitations associated with obtaining actual collision data, traffic conflicts are utilized as a safety measure to substitute observable collisions. Four conflict indicators — Modified Time to Collision (MTTC), Post Encroachment Time (PET), Percentage of Stopping Distance (PSD), and Deceleration Rate of Collision Avoidance (DRAC) — are introduced to assess the severity of traffic conflicts in terms of collision risk from multiple dimensions (Arun et al., 2021). These indicators are defined and calculated as follows.

Based on the common conflict index Time-to-Collision (TTC), MTTC further accounts for changes in vehicle acceleration and speed. It represents the remaining time until a collision occurs when the front and rear vehicles in the same lane continue driving with a certain acceleration. MTTC can be calculated using the following formula:

$$MTTC = \left(\Delta v \pm \sqrt{\Delta v^2 + 2\Delta a \left(x_l - x_f - L_l\right)}\right) / (\Delta a)$$ (5)

where, and represent the positions of the leading vehicle and the following vehicle at the current moment, respectively. denotes the body length of the leading vehicle. and represent the speed difference and acceleration difference between the two vehicles at the current moment, respectively. PET represents the time difference between two vehicles passing the conflict point on an intersecting trajectory, which is suitable for assessing collision risk at unsignalized intersections. The calculation formula is as follows:

$$PET = t{f} - t{l} \tag{6}$$

where, represents the moment at which the following vehicle reaches the conflict line, and represents the moment at which the leading vehicle leaves the conflict line. MTTC and PET both indicate temporal proximity, while PSD is a spatial proximity indicator and is calculable using the following formula:

$$PSD = (x_l - x_f - L_l) / ((v_f^2) / (2\mu g))$$ (7)

where, is the friction coefficient of the road surface, which is set at 0.4, and is the acceleration of gravity, which is at 9*.*8 m∕s. and denote the positions of the leading vehicle and the following vehicle at the current moment, respectively. represents the body length of the leading vehicle, and denotes the speed of the following vehicle. DRAC is defined as the required speed reduction for a following vehicle

Fig. 3. Error detection and correction mechanism workflow.

to avoid a collision with the leading vehicle. It is a kinematic indicator, and the specific calculation formula is as follows:

$$DRAC = (\Delta v^2) / (2(x_l - x_f - L_l))$$ (8)

where, and represent the positions of the leading vehicle and the following vehicle at the current moment, respectively. denotes the body length of the leading vehicle, and represents the speed difference between the two vehicles at the present moment.

3.6. Empirical analysis methods

An autoregressive distributed lag model (ARDL) was introduced to reveal the long- and short-term correlations between each variable and collision risk. As a method of time series analysis, this model is often used to analyze the extent to which one variable is influenced by other variables, as well as the long-term equilibrium relationship between variables and the short-term adjustment mechanism. However, traditional ARDL frequently yields unreliable estimates due to its default assumption of cross-sectional independence. By employing the standards of the PMG estimator, Mean Group estimator, and CCE method, the Cross-Sectionally Augmented Autoregressive Distributed Lag (CS-ARDL) model overcomes and surpasses the limitations of traditional ARDL (Chudik and Pesaran, 2015). For robustness testing, the Augmented Mean Group (AMG) was additionally employed to determine the long-term impact of each variable on collision risk and to complement the results of the CS-ARDL estimator, as AMG produces reliable estimates even in the presence of heterogeneity and cross-sectional dependence (Eberhardt and Bond, 2009).

Table 2 Hyperparameter of model fine-tuning.

Hyperparameter	Value
Batch size	2
Learning rate	5.00E−05
Epoch	3
Maximum gradient norm	1
Gradient accumulation	8
Cutoff length	1024
LR schedule	Cosine

4. Data preparation

4.1. Experiment sceneries

Due to their complex road characteristics and collision mechanism, unsignalized intersections, roundabouts, vehicle overtaking, and lane changes are regarded as typical traffic scenarios with a high incidence of traffic accidents. Therefore, these scenarios are often used to assess the predictive performance of collision risk models and analyze driver takeover performance. In this study, the above four scenarios (Fig. 4) were selected to evaluate the effectiveness of CPTR-LLM in predicting collision risk and analyzing takeover requirements. To more accurately replicate real traffic conditions, these experimental scenarios were not entirely closed off and included other moving objects in addition to the autonomous driving minibus participating in the experiment, all of which operated at normal speeds on the road.

4.2. Apparatus

In this study, a Donglan radar-video integrated sensors were used to collect experimental data. The device integrates real-time video structured analysis and radar data fusion algorithms, operates within a frequency range of 79 to 81 GHz, covers a lateral detection range of 10 lanes and a longitudinal detection distance of 350 to 500 m, and achieves a range accuracy of 0.1 m (close range) and 0.1 km/h, respectively. The autonomous driving vehicle utilized in the experiment is the Jinlong Skywell ''Blue Whale'' autonomous driving minibus, with a top speed of 30 km/h. In the process of autonomous driving, the driver can take over the control of the vehicle at any time through the remote control. A detailed schematic diagram of the equipment is provided in Fig. 5.

A single RTX4090 GPU was used for fine-tuning the training of the model, and the fine-tuning parameters are shown in Table 2. The fine-tuning process took about six hours to complete. After fine-tuning, the models are deployed on a GPU with 30 GB of memory, and 4 bit quantization was employed to accelerate the efficiency of model reasoning. As a result, the model was able to perform a series of inference tasks efficiently and effectively.

4.3. Data collection and processing

In each experiment scenario, 20 groups of experiments were conducted, with the autonomous vehicle operating entirely in automatic driving mode, while the radar-video integrated sensors collected realtime trajectory data of the autonomous vehicle and surrounding moving objects. Format conversion, data cleaning, and feature extraction were performed on the collected data. The processed autonomous driving vehicle data and surrounding moving objects data include variables such as position, speed, acceleration, and steering angle. The velocity, density, and volume of the surrounding traffic flow were subsequently calculated. The specific variables and their descriptions are shown in Table 3. The conflict indicators MTTC, PET, PSD, and DRAC were used to categorize the conflict type and level for each frame (approximately 0.1 s) of data. The thresholds for conflict types are set at 4s, 4s, 1, and 3*.*35 m∕s , respectively (Arun et al., 2021, 2022; Bhattarai et al.,

Fig. 4. Autonomous driving vehicle test environment.

2023). Considering that the MTTC, PET, and DRAC values increase with rising collision risk, while the PSD value decreases as collision risk increases, we further classify the conflict samples into different levels, representing varying degrees of collision risk (Arun et al., 2021). The specific classification thresholds are shown in Table 4. Finally, based on the categorization results of different indicators, the distribution of conflict types and conflict levels in different scenarios was obtained, as shown in Table 5. The higher the conflict level, the greater the collision risk.

4.4. Descriptive statistics

In order to enhance understanding of the data set and facilitate its representation, descriptive statistics for vehicle motion micro-variables and traffic flow macro-variables across different scenarios are provided. Details are provided in Table 6.

5. Experiments

5.1. Model performance comparison

5.1.1. Comparison of collision risk prediction performance

The performance of CPTR-LLM in collision risk prediction was systematically compared with four popular collision risk prediction models: XGBoost, LSTM, GCN, and DQN, as well as two advanced LLMs (GPT-4 and SafeDrive). SafeDrive is a specialized LLM designed specifically for risk quantification and safety decision-making for autonomous

Table 3 Variable description.

Variable	Description	Unit
𝑥	The horizontal coordinate of the self-vehicle (take	m
	the radar-video Integrated Sensors as the
	coordinate origin).
𝑦	The ordinate of the self-vehicle.	m
𝑣	Speed of the self-vehicle	m∕s
𝑎	Acceleration of the self-vehicle	2 m∕s
𝜃	Steering angle of the self-vehicle	◦
𝑥𝑠	The horizontal coordinate of the moving object	m
	around the self-vehicle.
𝑦𝑠	The ordinate of the moving object around the	m
	self-vehicle.
𝑣𝑠	Speed of the moving object around the self-vehicle.	m∕s
𝑎𝑠	Acceleration of the moving object around the	2 m∕s
	self-vehicle.
𝜃𝑠	Steering angle of the moving object around the	◦
	self-vehicle.
𝑉	Average speed of the traffic flow	km∕h
𝐾	Average density of the traffic flow	Veh∕km
𝑄	Average volume of the traffic flow	Veh∕h

vehicles (Zhou et al., 2024). The final performance comparison results of all models are shown in Table 7.

The experimental results indicate that CPTR-LLM significantly outperforms other models in collision risk prediction across all four driving scenarios. In addressing complex real-world driving scenarios with multiple vehicle interactions, traditional collision risk prediction models such as XGBoost, LSTM, GCN, and DQN failed to demonstrate

Fig. 5. Apparatus diagram.

Table 4 Conflict level classification thresholds.

Crash risk	Conflict level	MTTC(s)		PET(s)		PSD		DRAC (m/s2 )
		Min	Max	Min	Max	Min	Max	Min	Max
0.2	1	3	4	3	4	0.75	1	3.35	4.35
0.4	2	2	3	2	3	0.5	0.75	4.35	5.35
0.6	3	1	2	1	2	0.25	0.5	5.35	6.35
0.8	4	0	1	0	1	0	0.25	6.35	–

Table 5 Distribution of samples.

		Intersection	Roundabout	Overtake	Lane change	Total
	Conflict events	1432	1380	1283	1049	5144
Conflict type	Non-conflict events	2326	2544	2587	3806	11263
	Proportion	19/31	7/13	33/67	11/39	31/69
	Conflict level-1	296	344	330	262	1232
	Conflict level-2	337	407	431	301	1476
Conflict level	Conflict level-3	414	346	243	262	1265
	Conflict level-4	385	283	279	224	1171
	Proportion	21/24/29/27	25/30/25/21	26/34/19/22	25/29/25/21	24/29/25/23

satisfactory performance. GPT-4, as one of the most advanced generalpurpose LLMs, possesses exceptional reasoning abilities; however, due to a lack of domain-specific knowledge and training related to collision risk prediction, it made numerous judgment errors when performing related tasks. In contrast, SafeDrive, built upon GPT-4 with specialized training, significantly improved its performance in the field of collision risk prediction but still did not surpass CPTR-LLM in accuracy. By incorporating the Reasoning Chain (RC) framework and a two-stage training approach, CPTR-LLM effectively understands and extracts features of conflict events while accurately identifying different levels of collision risk. This design enables it to outperform existing models in prediction performance across complex driving scenarios, demonstrating stronger applicability and generalization capabilities.

5.1.2. Comparison of takeover requirement analysis performance

The performance of CPTR-LLM in takeover requirement analysis was compared with GPT-4 and SafeDrive. Tables 8–10 present the takeover requirement analysis process of three LLMS for a sample with conflict level 2. The experimental results show that, under the guidance of the reasoning chain, CPTR-LLM gradually analyzes the current driving scene features and accurately predicts the collision risk of the sample as 0.4. It further integrates the motion states of the

Table 6 Variable descriptive statistics.

Variable		Intersections	Roundabout	Overtake	Lane change
	Mean	0.24	−0.77	−1.13	−1.33
𝑣	Max	9.59	6.52	9.34	9.58
	Min	−9.63	−5.72	−9.65	−9.69
	S.D.	5.63	2.79	5.09	5.65
	Mean	0.01	−0.01	0.03	−0.02
𝑎	Max	2.13	7.57	5.45	6.55
	Min	−0.20	−5.83	−7.96	−6.60
	S.D.	0.29	0.89	0.51	0.42
	Mean	−37.23	−29.99	−33.73	−30.66
𝜃	Max	74.05	90.00	90.00	80.54
	Min	−90.00	−90.00	−90.00	−90.00
	S.D.	48.99	51.53	51.01	50.39
	Mean	2.04	1.92	3.13	1.35
𝑣𝑠	Max	9.63	8.53	9.49	9.66
	Min	−9.38	−8.50	−9.72	−9.56
	S.D.	5.42	2.92	4.63	4.88
	Mean	0.02	−0.01	−0.02	−0.01
𝑎𝑠	Max	7.80	7.95	4.33	5.65
	Min	−3.06	−7.67	−7.44	−7.99
	S.D.	0.42	0.72	0.46	0.42
	Mean	−36.01	−37.33	−33.05	−29.64
𝜃𝑠	Max	66.37	88.92	90.00	90.00
	Min	−90.00	−90.00	−90.00	−90.00
	S.D.	48.96	50.76	51.2	50.08
	Mean	11.33	4.97	8.03	9.24
𝑉	Max	31.02	20.00	20.00	28.86
	Min	0.17	0.00	4.00	0.00
	S.D.	8.23	4.90	6.53	6.98
	Mean	8.22	6.06	8.80	8.16
𝐾	Max	24.00	12.00	29.30	22.00
	Min	4.00	4.00	0.07	4.00
	S.D.	4.19	2.42	3.74	3.48
	Max	94.09	29.27	76.19	74.26
𝑄	Min	469.44	129.76	298.40	334.80
	Mean	0.80	0.00	0.54	0.00
	S.D.	78.27	22.54	60.2	57.41

autonomous vehicle, surrounding targets, and traffic flow conditions to comprehensively analyze the takeover requirement. Moreover, CPTR-LLM provides clear and executable takeover assistance suggestions, including reducing the vehicle's speed to 20 km/h and adjusting the steering angle to the left by 10 degrees to avoid potential conflict. In contrast, GPT-4 made an error in the first step by identifying the collision risk of the sample as 0.6 and emphasized that the driver should immediately prepare to take over. This misjudgment could lead to unnecessary takeover actions, increasing the risk of human takeover errors and thereby raising the possibility of a vehicle collision. Regarding takeover assistance suggestions, GPT-4 offered a range of guidance, but its output contained some overly general suggestions (e.g., ''Communicate with the Environment'' and ''Plan an Escape Route''), which lacked specificity and failed to provide concrete operational values, limiting their practicality. SafeDrive, as a specialized LLM for the traffic domain, demonstrated stronger reasoning abilities when handling autonomous driving issues. However, due to the lack of specialized training, it still made errors in assessing collision risk and analyzing takeover requirement. These experimental results further highlight the importance of domain adaptability and specialized training in enhancing the predictive analysis capabilities of CPTR-LLM.

5.2. Ablation experiment

To demonstrate that CPTR-LLM can enhance the performance of collision risk prediction by learning traffic characteristics from conflict event data, we conducted ablation experiments, with the results presented in Tables 11 and 12. Initially, the dataset from Step 1 was used to enable CPTR-LLM to differentiate between conflicting and nonconflicting data and to become acquainted with the characteristics of both types of conflicts. Subsequently, the dataset from Step 2 was used

Table 7 Comparison of model collision risk prediction performance.

		Intersection	Roundabout	Overtake	Lanechange
	Accuracy	0.84	0.88	0.84	0.86
CPTR-LLM	Precision	0.83	0.85	0.86	0.85
	Recall	0.82	0.85	0.86	0.87
	Accuracy	0.36	0.4	0.32	0.38
GPT-4	Precision	0.51	0.29	0.35	0.29
	Recall	0.32	0.32	0.3	0.38
	Accuracy	0.62	0.6	0.55	0.53
SafeDrive	Precision	0.64	0.45	0.6	0.6
	Recall	0.67	0.53	0.67	0.63
	Accuracy	0.31	0.28	0.25	0.29
XGBoost	Precision	0.46	0.24	0.47	0.41
	Recall	0.31	0.25	0.25	0.29
	Accuracy	0.33	0.4	0.35	0.33
LSTM	Precision	0.11	0.44	0.35	0.49
	Recall	0.33	0.29	0.46	0.33
	Accuracy	0.45	0.4	0.39	0.38
GCN	Precision	0.44	0.38	0.37	0.38
	Recall	0.45	0.22	0.36	0.38
	Accuracy	0.47	0.47	0.48	0.47
DQN	Precision	0.48	0.3	0.47	0.5
	Recall	0.46	0.32	0.47	0.46

to train CPTR-LLM to achieve a more precise determination of conflict levels. Regarding conflict type judgment, CPTR-LLM trained solely on the step 1 dataset demonstrated proficiency in identifying both conflict and non-conflict events with an accuracy exceeding 0.95. However, its highest accuracy for conflict level identification was only 0.42. This indicates that integrating the step2 training dataset is essential for more accurately determining conflict levels. In contrast, the CPTR-LLM trained solely on the step 2 dataset also demonstrated high accuracy in recognizing conflict types, suggesting that the step 2 training data significantly contributes to the model's understanding of conflict types. However, CPTR-LLM trained solely on the step2 dataset did not demonstrate a significant advantage in identifying conflict types, achieving a maximum accuracy of 0.58. In contrast, CPTR-LLM, following the two-step training, demonstrated improved performance in recognizing conflict types and levels, suggesting that this training process enhanced the model's comprehension of conflict events and their characteristics, enabling it to effectively discern and differentiate between various conflict levels. Nevertheless, the model's performance in predicting conflict levels remains suboptimal.

To further improve the model's predictive performance, we carefully designed an RC (Reasoning Chain) framework to deepen the model's understanding of the input traffic scene information. Additionally, considering the possibility of model misjudgment, we incorporated an Error Detection and Correction (EDC) mechanism to assess the reliability of the model's analysis and enhance its reasoning performance. The results show that the introduction of the reasoning chain significantly improved the model's prediction performance. Ultimately, CPTR-LLM achieved an accuracy of approximately 1.00 in predicting collision types and around 0.80 in predicting collision levels. This indicates that the RC framework enables the model to better capture the characteristics of conflict events, thereby improving prediction accuracy. The inclusion of EDC helps correct model misjudgments to some extent and further enhances the reasoning performance, resulting in an overall prediction accuracy of around 0.85 in the four driving scenarios.

5.3. Reliability of collision risk prediction

5.3.1. Short- and long-run relationship

In order to verify the predictive performance of CPTR-LLM and further understand the collision risk mechanism of autonomous vehicles, the CS-ARDL technology was employed to analyze both the long-term and short-term correlations between various variables and the collision risk of autonomous vehicles. The results of the CS-ARDL estimator are

Table 8 Analysis of takeover requirements generated by CPTR-LLM.

Takeover requirement analysis of CPTR-LLM Prompt An autonomous vehicle is driving on the road. At this moment, its speed, acceleration, steering angle and position coordinates are: 8.64 m/s, 0.0 m/s² −90.0◦ and [−5.74 m, 61.48 m], respectively. The autonomous vehicle width is 1.5 m. The width of the moving object is less than 1.5 m. At this time, the traffic flow speed is 10.96 km/h, the traffic flow density is 20.0 veh/km, the traffic flow volume is 219.2 veh/h. There is a moving object F moving. At this moment, its speed, acceleration, steering angle and position coordinates are: 5.56 m/s, 0.0 m/s² −0.17◦ and [53.24 m, 217.64 m], respectively. There is a moving object G moving. At this moment, its speed, acceleration, steering angle and position coordinates are: −3.87 m/s, 0.0 m/s² , −0.55◦ and [−24.15 m, −93.69 m], respectively. There is a moving object H moving. At this moment, its speed, acceleration, steering angle and position coordinates are: 6.62 m/s, 0.0 m/s² , 12.26◦ and [−0.87 m, 45.65 m], respectively. Which option best describes the speed of the autonomous vehicle (options: fast, medium, slow). Response Fast Prompt Analyze the acceleration of the autonomous vehicle (options: large acceleration, large deceleration, small acceleration, small deceleration, keep zero). Response Keep zero Prompt Analyze the steering angle of the autonomous vehicle (options: large left turn, large right turn, small left turn, small right turn, straight line movement). Response Large left turn Prompt Predict the traffic conflict type (options: conflict, non-conflict). Response Conflict Prompt Based on the above analysis, predict the crash risk of the autonomous vehicle (options: 0.2, 0.4, 0.6, 0.8). Response 0.4 Prompt Based on the motion state of the autonomous vehicle, the characteristics of the surrounding environment and the collision risk, which option best describes the need for drivers to take over control of autonomous vehicles (Options: very necessary, Necessary, optional, not necessary). Response Not necessary Prompt Please explain why you think the takeover requirement is not necessary at this time. Response The autonomous vehicle is not involved in a conflict with other vehicles or obstacles, and there are no close or fast-moving objects nearby that could pose a collision risk. The traffic flow volume, density, and speed are moderate, indicating a relatively stable environment. While the autonomous vehicle is turning left, it's a gentle turn, which does not require immediate human intervention. Therefore, takeover requirements of the autonomous vehicle at this time are not necessary. Prompt Please provide takeover assistance suggestions: Acceleration (Options: [large acceleration, small acceleration, large deceleration, small deceleration, keep unchanged]) Speed (Range: 0–40 km/h) Heading (Options: [Turn left, Turn right, Keep straight]) Steering angle (Range: 0–360) Response Format (JSON): ''Acceleration'': ''...'', ''Speed'': ''...'', ''Heading'': ''...'', ''Steering angle'': ''...'' Response ''Acceleration'': ''small deceleration'', ''Speed'': ''20 km/h'', ''Heading'': ''Turn left'', ''Steering angle'': ''10''

presented in Tables 13 and 14. The conflict indicators are utilized as a reference model, as their estimates reflect the actual conflict scenarios. The estimated results from CPTR-LLM and the conflict indicators exhibit similar trends. Both models show a negative and significant impact of the autonomous driving vehicle's speed on collision risk, indicating that collision risk decreases by 0.001% for every 1% increase in speed, as estimated by both CPTR-LLM and the conflict indicators, with other factors held constant. Additionally, the effect of the steering angle of surrounding moving objects on collision risk is positive and significant. Holding other factors constant, the collision risk estimated by CPTR-LLM and the conflict indicators increases by 0.000152% and 0.000134%, respectively, for each 1% increase in the steering angle of surrounding moving objects. Similar results are observed in the short term. These findings suggest that higher speeds of autonomous vehicles

do not necessarily correlate with increased collision risk but may instead reflect the safety and stability of the surrounding environment, thereby potentially reducing collision risk. Additionally, the results underscore the significant impact of the steering angle of surrounding moving objects on the safety of autonomous vehicles, highlighting the importance of closely monitoring surrounding vehicle movements to prevent collisions during actual driving. For CPTR-LLM that only employs the first and second steps of training, the CS-ARDL estimates yielded results that differed from those of the conflict indicators. No variables were found to be significantly associated with collision risk in these two models. Furthermore, the impact of various variables on collision risk contradicted the results obtained from the conflict indicators. In comparison, for CPTR-LLM employing two-step training, the CS-ARDL estimates showed a significant impact of the surrounding

Table 9 Analysis of takeover requirements generated by GPT-4.

Takeover requirement analysis of GPT-4

Prompt An autonomous vehicle is driving on the road. At this moment, its speed, acceleration, steering angle and position coordinates are: 8.64 m/s, 0.0 m/s² −90.0◦ and [−5.74 m, 61.48 m], respectively. The autonomous vehicle width is 1.5 m. The width of the moving object is less than 1.5 m. At this time, the traffic flow speed is 10.96 km/h, the traffic flow density is 20.0 veh/km, the traffic flow volume is 219.2 veh/h. There is a moving object F moving. At this moment, its speed, acceleration, steering angle and position coordinates are: 5.56 m/s, 0.0 m/s² −0.17◦ and [53.24 m, 217.64 m], respectively. There is a moving object G moving. At this moment, its speed, acceleration, steering angle and position coordinates are: −3.87 m/s, 0.0 m/s² , −0.55◦ and [−24.15 m, −93.69 m], respectively. There is a moving object H moving. At this moment, its speed, acceleration, steering angle and position coordinates are: 6.62 m/s, 0.0 m/s² , 12.26◦ and [−0.87 m, 45.65 m], respectively. Please estimate the crash risk (options: 0.2, 0.4, 0.6, 0.8) and analyze the takeover requirements (options: very necessary, Necessary, optional, not necessary) of autonomous vehicle at this time and provide specific decision-making and operational advice to the autonomous vehicle driver to help him or her drive safely.

Crash Risk:

The crash risk in this scenario is 0.6, this indicates a moderate to high risk of collision. In such a scenario, immediate attention and possible intervention by the driver are crucial to ensure safety.

Takeover Requirements:

The takeover requirement in this scenario is necessary. The driver should be alerted immediately about the increased risk and be prepared to take control of the vehicle. Driving Decision-Making and Operational Guidance:

Response

1. Monitor Object H:

Continuously monitor Object H, as it is the closest and most likely to intersect with the AV's path.

1. Speed Adjustment:

Gradually reduce the speed of the AV to create a greater buffer distance from Object H. This will provide more time for both the AV and Object H to react and adjust their paths.

1. Lane Positioning:

Increase the distance between the AV and Object H. This can be particularly important if the AV and Object H are converging on the same path.

4. Communicate with the Environment:

Signals: Use turn signals or hazard lights to indicate any lane changes or deceleration, making sure other road users are aware of the AV's intentions. Sound Alert: If Object H is a pedestrian or non-motorized vehicle, consider using a sound alert (horn) to draw attention and prompt a response.

5. Plan an Escape Route:

Emergency Maneuver: Identify a potential escape route (e.g., an empty lane or shoulder) in case Object H suddenly changes direction towards the AV.

objects' steering angles on collision risk. Moreover, the introduction of inference chain framework and error detection and correction mechanism makes the predicted results of the model more similar to the actual results, including the importance of variables and the influence mechanism. The above results effectively prove the importance of two-stage training strategy, inference chain framework and error detection and correction mechanism to improve the performance of model collision risk prediction.

5.3.2. Robustness

As a robustness check, the Augmented Mean Group (AMG) approach is employed in this study to assess the long-term effects of various variables on collision risk. The AMG estimation results, presented in Tables 15 and 16, reveal that, from a long-term perspective, the speed and acceleration of surrounding moving objects, along with the speed of surrounding traffic flow, significantly influence the collision risk of the ego vehicle. Specifically, an increase in the speed of surrounding moving objects reduces collision risk, whereas an increase in acceleration leads to a higher collision risk. Furthermore, an increase in traffic flow speed is also associated with a reduction in collision risk. These findings highlight that an increase in the speed of surrounding moving objects or traffic flow does not necessarily indicate a higher collision risk in the surrounding environment. Instead, the increase in the acceleration of surrounding moving objects is identified as a crucial factor contributing to the increase in collision risk.

5.4. Consistency of model analysis

Since LLMs generate diverse results based on probability, evaluating the consistency of their outputs is essential, particularly in vehicle collision risk prediction. To this end, four samples (one from each conflict level) were randomly selected from four traffic scenarios. For each sample, ten independent predictions were conducted, and the results are presented in Fig. 6. The results indicate that CPTR-LLM's predictions for the same sample are generally consistent. For the inconsistent samples, the manual analysis revealed that their traffic scenes were predominantly complex, making it challenging to classify the collision risk level straightforwardly. Overall, CPTR-LLM's performance in collision risk prediction is generally consistent and reliable.

5.5. Reliability of the takeover requirements analysis

To validate the reliability of the CPTR-LLM's takeover requirement analysis, the detection results from the Error Detection and Correction (EDC) mechanism were extracted. As shown in Table 17, the mechanism effectively detects the model's judgment errors, identifying misinterpretations of the autonomous vehicle's speed, misjudgments of the relative orientation of surrounding moving objects, and incorrect collision risk estimates. Based on the detection results, CPTR-LLM thoroughly reviewed and analyzed the causes of these errors, leading to continuous improvements in the model's analytical performance. Furthermore, the errors detected in the entire test set were summarized.

Table 10 Analysis of takeover requirements generated by SafeDrive.

Takeover requirement analysis of SafeDrive

Response

Prompt An autonomous vehicle is driving on the road. At this moment, its speed, acceleration, steering angle and position coordinates are: 8.64 m/s, 0.0 m/s² −90.0◦ and [−5.74 m, 61.48 m], respectively. The autonomous vehicle width is 1.5 m. The width of the moving object is less than 1.5 m. At this time, the traffic flow speed is 10.96 km/h, the traffic flow density is 20.0 veh/km, the traffic flow volume is 219.2 veh/h. There is a moving object F moving. At this moment, its speed, acceleration, steering angle and position coordinates are: 5.56 m/s, 0.0 m/s² −0.17◦ and [53.24 m, 217.64 m], respectively. There is a moving object G moving. At this moment, its speed, acceleration, steering angle and position coordinates are: −3.87 m/s, 0.0 m/s² , −0.55◦ and [−24.15 m, −93.69 m], respectively. There is a moving object H moving. At this moment, its speed, acceleration, steering angle and position coordinates are: 6.62 m/s, 0.0 m/s² , 12.26◦ and [−0.87 m, 45.65 m], respectively. Please estimate the crash risk (options: 0.2, 0.4, 0.6, 0.8) and analyze the takeover requirements (options: very necessary, Necessary, optional, not necessary) of autonomous vehicle at this time and provide specific decision-making and operational advice to the autonomous vehicle driver to help him or her drive safely.

• 1. First, let's analyze the current state of the autonomous vehicle. It's moving at a speed of 8.64 m/s, with a steering angle of −90 degrees, and its current position is at [−5.74 m, 61.48 m].
• 2. Next, we examine the moving objects around the autonomous vehicle. Object F is moving at a speed of 5.56 m/s, with a steering angle of −0.17 degrees, and its current position is at [513.24 m, 217.64 m]. Given its speed, direction, and significant distance from the autonomous vehicle, it poses no risk of collision. 3. Object G is moving at a speed of −3.87 m/s, with a steering angle of −0.55 degrees, and it's positioned at [−24.15 m, −93.69 m]. This object is moving in the opposite direction to the autonomous vehicle and is positioned far behind it, which
• means it presents no risk of collision. 4. Object H, however, is moving at a speed of 6.62 m/s, with a steering angle of 12.26 degrees, and it's positioned at [−0.87 m, 45.65 m]. Given its speed, direction, and proximity to the autonomous vehicle, it presents a potential risk of collision. 5. The traffic flow speed is 10.96 km/h, the traffic flow density is 20.0 veh/km, and the traffic flow volume is 219.2 veh/h. These traffic conditions suggest a moderate level of traffic, but given the proximity and direction of object H, the risk of collision is high.

Crash Risk:

Given the current scenario, the crash risk estimation would be 0.6, indicating a moderate to high risk.

Takeover Requirements:

The takeover requirement for the autonomous vehicle at this time is necessary. Driving Decision-Making and Operational Guidance:

The autonomous vehicle driver should decelerate to allow object H to pass safely. Therefore, the appropriate action is to decelerate.

Table 11 Ablation experiment of conflict types prediction.

		Intersection	Roundabout	Overtake	Lanechange
	Accuracy	0.96	0.98	0.98	1.00
CPTR-LLM(step1 only)	Precision	0.90	0.99	0.99	1.00
	Recall	0.98	0.83	0.93	1.00
	Accuracy	0.90	0.94	0.84	0.92
CPTR-LLM(step2 only)	Precision	0.82	0.74	0.60	0.50
	Recall	0.79	0.81	0.55	0.46
	Accuracy	0.98	0.98	0.94	1.00
CPTR-LLM(without RC)	Precision	0.94	0.99	0.89	1.00
	Recall	0.99	0.83	0.85	1.00
	Accuracy	0.98	1.00	1.00	1.00
CPTR-LLM(with RC)	Precision	0.99	1.00	1.00	1.00
	Recall	0.94	1.00	1.00	1.00
	Accuracy	1.00	1.00	1.00	1.00
CPTR-LLM(with RC and EDC)	Precision	1.00	1.00	1.00	1.00
	Recall	1.00	1.00	1.00	1.00

As seen in Fig. 7, CPTR-LLM can accurately interpret the pre-takeover scene data and predict the associated risks, achieving reliable takeover requirement analysis. Among the detected samples, approximately 80% were assessed as correct by the detection mechanism, while those evaluated as incorrect generally accounted for less than 20%. Additionally, a small number of samples were evaluated as having output format errors, which are influenced by the inherent uncertainty in LLM outputs.

6. Results and discussions

6.1. Collision risk prediction results

Leveraging CPTR-LLM's strong collision risk prediction capabilities, samples from various traffic scenarios were randomly selected for collision risk prediction, with the results illustrated in Fig. 8. The high collision risk areas identified by the experiment align closely

Table 12 Ablation experiment of conflict level prediction.

		Intersection	Roundabout	Overtake	Lanechange
	Accuracy	0.42	0.34	0.2	0.2
CPTR-LLM(step1 only)	Precision	0.25	0.27	0.19	0.19
	Recall	0.32	0.43	0.31	0.31
	Accuracy	0.54	0.56	0.54	0.58
CPTR-LLM(step2 only)	Precision	0.63	0.67	0.7	0.69
	Recall	0.51	0.61	0.55	0.61
	Accuracy	0.68	0.64	0.56	0.66
CPTR-LLM(without RC)	Precision	0.75	0.78	0.7	0.69
	Recall	0.67	0.7	0.55	0.61
	Accuracy	0.8	0.84	0.76	0.8
CPTR-LLM(with RC)	Precision	0.85	0.82	0.82	0.83
	Recall	0.82	0.87	0.74	0.83
	Accuracy	0.84	0.88	0.84	0.86
CPTR-LLM(with RC and EDC)	Precision	0.83	0.85	0.86	0.85
	Recall	0.82	0.85	0.86	0.87

Table 13 Short- and long-run relationship examining results of different training stages.

Variable	Short-run
	CPTR-LLM (step1 only)			CPTR-LLM (step2 only)			CPTR-LLM			Traffic conflict index
	Impact	z	P-value	Impact	z	P-value	Impact	z	P-value	Impact	z	P-value
𝑣	0.003	0.91	0.362	0.028	0.91	0.362	−0.003***	−6.41	0	−0.002***	−8.23	0
𝑎	−1.227	−0.96	0.337	−1.03	−0.88	0.378	−1.091	−0.95	0.343	−1.016	−0.94	0.346
𝜃	−3.34E−04	−1.56	0.12	−8.22E−04	−0.83	0.404	6.32E−05	−0.13	0.894	3.90E−06	0.01	0.994
𝑣𝑠	−0.008	−1.77	0.077	0.004	0.14	0.887	−0.003	−0.61	0.542	−0.002	−0.45	0.651
𝑎𝑠	0.176	1.52	0.129	0.33	1.34	0.181	0.009	0.31	0.758	0.006	0.2	0.838
𝜃𝑠	1.49E−04	0.76	0.444	−5.04E−05	−0.04	0.969	3.00e−04**	2.5	0.012	2.61e−04*	1.96	0.05
𝑄	1.20E−04	0.54	0.587	0.002	0.64	0.524	2.20E−04	0.21	0.832	2.21E−04	0.22	0.828
𝐾	−3.04E−04	−0.14	0.891	−0.032	−0.56	0.578	−0.002	−0.47	0.636	−0.003	−0.54	0.586
𝑉	−7.56E−04	−0.33	0.741	−0.012	−0.33	0.741	1.11E−04	0.01	0.993	2.18E−04	−0.02	0.986
	Long-run outcomes
𝑣	0.001	0.94	0.346	0.014	0.89	0.373	−0.001***	−7.69	0	−0.001***	−8.16	0
𝑎	−0.645	−0.96	0.336	−0.552	−0.89	0.373	−0.577	−0.95	0.343	−0.528	−0.94	0.347
𝜃	−1.91E−04	−1.42	0.157	−3.75E−04	−0.79	0.429	−3.83E−05	−0.16	0.873	2.24E−06	0.01	0.993
𝑣𝑠	−0.004	−1.83	0.067	0.002	0.12	0.907	−0.001	−0.64	0.521	−0.001	−0.5	0.617
𝑎𝑠	0.008	1.58	0.115	0.172	1.43	0.152	0.006	0.35	0.724	0.003	0.23	0.82
𝜃𝑠	7.66E−05	0.77	0.442	−5.43E−05	−0.08	0.932	1.52e−04***	2.66	0.008	1.34e−04*	2.03	0.042
𝑄	5.74E−05	0.48	0.628	0.001	0.6	0.55	7.12E−05	0.13	0.9	7.04E−05	0.13	0.899
𝐾	−9.40E−05	−0.08	0.932	−0.015	−0.51	0.613	−9.96E−04	−0.36	0.717	−0.001	−0.42	0.672
𝑉	−4.81E−04	−0.43	0.665	−0.005	−0.28	0.776	6.40E−04	0.09	0.927	0.001	0.07	0.946

Note: ***, **, and * denote P < 0.01, P < 0.05 and P < 0.10.

Table 14

Variable	Short-run
	CPTR-LLM (without RC)			CPTR-LLM (with RC)			CPTR-LLM (with RC and EDC)			Traffic conflict index
	Impact	z	P-value	Impact	z	P-value	Impact	z	P-value	Impact	z	P-value
𝑣	−0.001	−0.23	0.819	−0.002**	−2.69	0.007	−0.003***	−6.41	0	−0.002***	−8.23	0
𝑎	−0.749	−0.98	0.33	−0.944	−0.88	0.377	−1.091	−0.95	0.343	−1.016	−0.94	0.346
𝜃	−2.51E−04	−0.78	0.434	−8.13E−05	−0.2	0.838	6.32E−05	−0.13	0.894	3.90E−06	0.01	0.994
𝑣𝑠	0.001	0.66	0.512	−0.005	−1.72	0.085	−0.003	−0.61	0.542	−0.002	−0.45	0.651
𝑎𝑠	0.013	1.73	0.083	0.011	0.68	0.494	0.009	0.31	0.758	0.006	0.2	0.838
𝜃𝑠	−4.93e−04*	−2.09	0.037	2.99e−04*	2.12	0.034	3.00e−04**	2.5	0.012	2.61e−04*	1.96	0.05
𝑄	−3.12E−04	−0.65	0.517	3.32E−04	0.4	0.69	2.20E−04	0.21	0.832	2.21E−04	0.22	0.828
𝐾	0.003	0.76	0.445	−0.005	−0.95	0.342	−0.002	−0.47	0.636	−0.003	−0.54	0.586
𝑉	0.005	0.85	0.398	0.002	0.26	0.794	1.11E−04	0.01	0.993	2.18E−04	−0.02	0.986
	Long-run outcomes
𝑣	−1.67E−04	−0.15	0.88	−0.001**	−2.41	0.013	−0.001***	−7.69	0	−0.001***	−8.16	0
𝑎	−0.401	−0.98	0.329	−0.443	−0.87	0.383	−0.577	−0.95	0.343	−0.528	−0.94	0.347
𝜃	−1.31E−04	−0.79	0.428	−3.82E−05	−0.2	0.841	−3.83E−05	−0.16	0.873	2.24E−06	0.01	0.993
𝑣𝑠	0.001	0.76	0.449	−0.003	−1.74	0.082	−0.001	−0.64	0.521	−0.001	−0.5	0.617
𝑎𝑠	0.007	1.73	0.084	0.005	0.68	0.498	0.006	0.35	0.724	0.003	0.23	0.82
𝜃𝑠	−2.58e−04*	−2.07	0.038	1.53e−04**	2.08	0.038	1.52e−04***	2.66	0.008	1.34E−04	2.03	0.042
𝑄	−1.73E−04	−0.65	0.516	1.27E−04	0.3	0.765	7.12E−05	0.13	0.9	7.04E−05	0.13	0.899
𝐾	−0.002	−0.85	0.395	−0.002	−0.85	0.395	−9.96E−04	−0.36	0.717	−0.001	−0.42	0.672
𝑉	0.002	0.34	0.735	−0.005	−0.28	0.776	6.40E−04	0.09	0.927	0.001	0.07	0.946

Note: ***, **, and * denote P < 0.01, P < 0.05 and P < 0.10.

Table 15 Robustness examining results of different training stages.

| Variable | CPTR-LLM (step1 only) | | | | CPTR-LLM (step2 only) | | | CPTR-LLM | | | Traffic conflict index | | | |----------|-----------------------|-------|---------|--------|-----------------------|---------|-----------|----------|---------|-----------|------------------------|---------|--| | | Impact | z | P-value | Impact | z | P-value | Impact | z | P-value | Impact | z | P-value | | | 𝑣 | 0.004** | 2.69 | 0.007 | −0.008 | −1.67 | 0.096 | 0.001 | 0.31 | 0.756 | 0.001 | 0.3 | 0.767 | | | 𝑎 | −1.397 | −1 | 0.317 | −1.369 | −0.99 | 0.322 | −1.258 | −1.01 | 0.312 | −1.261 | −1.01 | 0.312 | | | 𝜃 | −1.68E−04 | −0.92 | 0.359 | −0.001 | −0.85 | 0.395 | −4.31E−05 | −0.2 | 0.842 | 4.95E−05 | 0.21 | 0.831 | | | 𝑣𝑠 | −0.009** | −2.75 | 0.006 | −0.004 | −0.92 | 0.358 | −0.006*** | −3.47 | 0.001 | −0.006*** | −3.44 | 0.001 | | | 𝑎𝑠 | 0.006 | 1.49 | 0.135 | 0.104 | 1.42 | 0.157 | 0.020** | 2.07 | 0.038 | 0.015* | 2.02 | 0.043 | | | 𝜃𝑠 | 2.10E−04 | 0.68 | 0.494 | −0.001 | −0.41 | 0.68 | 7.18E−05 | 0.54 | 0.59 | 3.66E−05 | 0.26 | 0.791 | | | 𝑄 | 9.30E−06 | 0.03 | 0.976 | 0.002 | 1.12 | 0.264 | 0.001 | 1.92 | 0.055 | 0.001 | 1.9 | 0.057 | | | 𝐾 | 0.003*** | 4.67 | 0 | −0.007 | −0.47 | 0.638 | −0.009 | −1.4 | 0.162 | −0.009 | −1.46 | 0.143 | |

0.001 0.33 0.742 −0.015 −1.16 0.246 −0.007*** −2.59 0.009 −0.006*** −2.78 0

Note: ***, **, and * denote P < 0.01, P < 0.05 and P < 0.10.

Table 16 Robustness examining results of different reasoning forms.

Variable	CPTR-LLM (without RC)			CPTR-LLM (with RC)			CPTR-LLM (with RC and EDC)			Traffic conflict index
	Impact	z	P-value	Impact	z	P-value	Impact	z	P-value	Impact	z	P-value
𝑣	0.002	0.86	0.39	0.002	0.76	0.449	0.001	0.31	0.756	0.001	0.3	0.767
𝑎	−0.598	−1	0.318	−1.269	−1.01	0.312	−1.258	−1.01	0.312	−1.261	−1.01	0.312
𝜃	−3.39E−04	−1.11	0.267	−1.39E−04	−0.49	0.623	−4.31E−05	−0.2	0.842	4.95E−05	0.21	0.831
𝑣𝑠	−0.002	−1.28	0.2	−0.005***	−4.95	0	−0.006***	−3.47	0.001	−0.006***	−3.44	0.001
𝑎𝑠	0.017	1.24	0.215	0.029	1.77	0.077	0.020**	2.07	0.038	0.015**	2.02	0.043
𝜃𝑠	−3.15E−04	−1.59	0.111	−2.12E−05	−0.13	0.9	7.18E−05	0.54	0.59	3.66E−05	0.26	0.791
𝑄	2.79E−04	0.43	0.671	0.001	1.44	0.15	0.001	1.92	0.055	0.001	1.9	0.057
𝐾	−0.002	−0.22	0.826	−0.005	−1.27	0.204	−0.009	−1.4	0.162	−0.009	−1.46	0.143
𝑉	0.001	0.23	0.82	0.001	0.44	0.658	−0.007***	−2.59	0.009	−0.006***	−2.78	0

Note: ***, **, and * denote P < 0.01, P < 0.05 and P < 0.10.

Fig. 6. Consistency test results for CPTR-LLM responses.

with the common conflict zones described in traffic flow theory. In intersection scenarios, high collision risk for autonomous vehicles is primarily concentrated in the areas of entry and exit. In roundabout scenarios, high collision risk is similarly concentrated at the four junctions of the roundabout. These areas typically experience convergence of traffic flows from various directions, leading to frequent conflicts between moving objects and, consequently, collisions. Additionally, in overtaking scenarios, three high-risk areas were identified: Region 1 represents the risk during the overtaking maneuver of the autonomous vehicle; Area 2 pertains to the risk associated with vehicles exiting the intersection; Zone 3 denotes the risk involved in lane changing. Lastly, in lane change scenarios, collision risk is heightened in Zone 1 and Zone 2 as a result of vehicle lane changes. In Region 3, the collision risk is elevated due to the high density of moving objects and proximity to the intersection.

6.2. Takeover requirements analysis results

Based on the excellent takeover analysis performance of CPTR-LLM, takeover requirements are evaluated and the output results are classified according to the necessity of the takeover, including ''not necessary'', ''optional'', ''necessary'', and ''very necessary''. The final distribution of takeover requirements in different traffic scenarios is presented. As shown in Fig. 9, CPTR-LLM comprehensively considers the characteristics of the surrounding environment, effectively avoiding unnecessary and unsafe takeovers. For example, in the case of an intersection without traffic signals, CPTR-LLM suggests a takeover just before entering and after leaving the intersection. For a roundabout, CPTR-LLM determines that only the traffic environments at takeover points 1 and 2 are sufficiently complex to require driver intervention. In overtaking and lane-change scenarios, a takeover is only necessary when the autonomous vehicle is about to execute a complex maneuver such as overtaking or lane-changing before entering an intersection.

Fig. 7. Reliability evaluation of takeover requirements analysis.

Table 17 Error detection and correction mechanism analysis results.

	Error detection and correction mechanism analysis results
	Your reasoning contains some errors. Let's review them to help you improve:
	Scene Understanding:
	1. Autonomous vehicles: the autonomous vehicle's speed (8.64 m/s) indicates a
Prompt	'fast' state, not 'slow'.
	2. Moving objects around the autonomous vehicle: the direction of the moving
	object 'F' is 'front right', but the provided result was 'back left'. Check the
	calculation of relative positioning.
	Risk Assessment: the estimated crash risk '0.4' does not match the actual risk
	'0.2'. Please revise.
	Takeover Assistance Suggestions: Your decision is sound and safe, well done!
	Take some time to review your reasoning process, and use these insights to
	enhance your future performance. Remember, identifying and addressing mistakes is
	a valuable step toward improving your capabilities.
	Thank you for the feedback! I'll take this as an opportunity to improve my
	autonomous vehicle reasoning skills.
	Upon reviewing my previous response:
	1. Scene Understanding: I can see that I made errors in understanding the
	autonomous vehicle's speed and the moving object F's direction. I will be more
Response	precise in my analysis going forward.
	2. Risk Assessment: It seems I misestimated the crash risk. I'll make sure to
	double-check my calculations and consider multiple factors when assessing risk.
	3. Takeover Assistance Suggestions: Thankfully, it appears I made a safe and
	sound decision!
	To improve my performance, I will:
	* Pay closer attention to scene details, ensuring accurate understanding of object
	movements and positions.
	* Double-check my calculations for speed and crash risk.
	* Continuously learn from mistakes and refine my decision-making processes. Thank you again for the feedback! I'll strive to be more precise and effective in my
	autonomous vehicle reasoning.

In scenarios with multiple surrounding moving objects, CPTR-LLM generally suggests that takeover is unnecessary, as the autonomous vehicle possesses an obstacle avoidance function, reducing the need for unsafe takeovers.

These results indicate that evaluating takeover requirement requires a comprehensive analysis of the vehicle's motion characteristics, the scene features, and collision risks. High collision risk does not necessarily imply a high need for takeover. In scenarios where there are multiple moving objects around, a takeover may be unwise, as it could further increase the collision risk. A takeover is only necessary when executing complex, hazardous driving maneuvers or when there are multiple obstacles requiring a change in the driving path.

6.3. Practical application of the model

To reveal the practical application value of CPTR-LLM, we conducted an analysis and discussion of its response time and misjudgment situations. Fig. 10 compares the single response time of CPTR-LLM with that of GPT-4 and SafeDrive. Compared to the other two models, CPTR-LLM, as a locally deployed model, achieves fast response times with an average response time of approximately 7 s. Even with the inclusion of the reasoning chain and error detection mechanisms, the highest response time of CPTR-LLM remains consistently lower than that of GPT-4 and SafeDrive. Additionally, as shown in Fig. 10(b), GPT-4 and SafeDrive, due to their reliance on API calls, experience significant fluctuations in response time, which can be heavily influenced by

Fig. 8. Collision risk distribution in different scenarios. Red block: collision risk is 0.8. Orange block: collision risk is 0.6. Yellow block: collision risk is 0.4. Green block: Collision risk is 0.2. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

network conditions. The maximum response time for both models even reached 100 s, which could potentially result in unavoidable collisions in actual driving scenarios. Therefore, CPTR-LLM demonstrates considerable practical application potential, though its response time may still require further optimization to better facilitate real-time risk prediction and takeover requirement analysis during autonomous vehicle operation.

Considering the safety risks associated with model misjudgments in real-world driving, the collision risk prediction performance and the reliability of the output decisions in CPTR-LLM were thoroughly evaluated. As shown in Fig. 11, the model accurately predicts samples at various levels of collision risk, especially those with the highest and lowest risks. However, the model also exhibits certain misjudgments. These misjudgments consistently reveal the model's tendency to overestimate collision risks, such as misclassifying a sample with a collision risk of 0.4 as 0.6 or a sample with a collision risk of 0.6 as 0.8. In practical applications, this could lead to overreporting of collision risks, triggering unnecessary takeovers. Additionally, CPTR-LLM may also underreport collision risks, for example, misjudging a sample with a collision risk of 0.8 as 0.6 or even 0.4. In contrast, underreporting collision risks is usually more dangerous, as it could lead to unavoidable collisions. Regarding the takeover decision assistance, as shown in Fig. 7, the decisions output by CPTR-LLM are mostly reasonable and safe, but there are also some misleading suggestions. Therefore, we aim to further enhance the reliability of CPTR-LLM in future research. Potential improvements include expanding the training sample size, particularly for high-risk samples, and upgrading the training strategy, such as guiding the model to learn from simpler scenarios before progressing to more complex ones, thereby enhancing the model's understanding of risk evolution patterns. Furthermore, developing a rule-based backup decision mechanism could be an effective way to ensure driving safety. Once the model is detected to have generated erroneous judgments, it should switch to the alternative decision-making mechanism and adjust the driving strategy based on the actual situation, thereby minimizing the risk of generating erroneous decision suggestions.

7. Conclusions

This study proposes a collision risk prediction and takeover requirement analysis system framework, CPTR-LLM, based on large language models (LLMs), aiming to address the challenges of takeover requirement assessment in real-world traffic scenarios. The designed twostage training strategy, reasoning chain framework, and error detection and correction mechanisms effectively enhance the model's reasoning performance. Empirical analysis results demonstrate the model's performance in collision risk prediction and takeover requirement analysis, while also revealing the positive impact of the speed of autonomous vehicles and surrounding moving objects on collision risk, and the negative impact of the acceleration and steering angles of surrounding objects on collision risk. Based on its outstanding takeover requirement analysis capabilities, CPTR-LLM effectively reduces unnecessary or unsafe takeovers, including those in simple driving scenarios or in dangerous scenarios with multiple surrounding moving targets, and

Fig. 9. Takeover requirements distribution in different scenarios. Red block: takeover requirement is 0.8. Orange block: takeover requirement is 0.6. Yellow block: takeover requirement is 0.4. Green block: takeover requirement is 0.2. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of this article.)

Fig. 10. Comparison of model response time (sample size = 20).

shows considerable practical application potential. Overall, this study highlights the potential of using LLMs to enhance takeover requirement assessment for autonomous vehicles, providing valuable insights for the future incorporation of LLMs in road safety research.

However, there are certain limitations in this study: first, due to technical constraints, the data collected does not include driver-related data or the behavior of surrounding moving objects. These data are crucial for further enhancing the model's predictive performance and

Fig. 11. Confusion matrix for collision risk prediction.

accurately assessing the need for takeover. Second, the performance of the proposed CPTR-LLM still relies on the quality of sensors, and it may be affected by adverse weather conditions and environmental noise. Future developments may require the integration of more advanced multimodal data fusion techniques and data preprocessing methods to improve the performance of radar-video integrated sensors in complex scenarios. Additionally, incorporating adaptive algorithms into the prediction model could help compensate for sensor data discrepancies, enhancing the model's robustness to data inconsistencies. Third, determining takeover requirements in complex traffic scenarios presents an additional challenge for CPTR-LLM. In real-world driving environments, complex traffic situations frequently occur, but there is currently a lack of such sample data. Addressing this issue would facilitate the application of CPTR-LLM in even more complex traffic scenarios. Finally, the ''hallucination'' problem of LLMs presents another potential challenge. In road safety, erroneous information can have fatal consequences. Given that current LLMs are trained on various sources of information, it is essential to train more specialized LLMs for collision risk prediction and takeover judgment for autonomous vehicles, using mechanisms like voting systems to generate iterative decisions.

It can be anticipated that with the release of more advanced LLMs and the availability of more comprehensive and reliable data, the accuracy and quality of LLM output will further improve. It is important to emphasize that no AI system is infallible, and errors cannot be entirely avoided. Therefore, the design of CPTR-LLM is not intended to replace drivers in making takeover decisions, but rather to serve as a continually evolving and improving AI system that assists drivers in making better takeover decisions, providing a more comfortable and secure driving experience. Future research will focus on further developing and refining this technology, continuing to explore the potential of LLMs in the field of road safety.

CRediT authorship contribution statement

Qingchao Liu: Writing – review & editing, Funding acquisition, Conceptualization. Ruohan Yu: Writing – original draft, Visualization, Investigation. Yingfeng Cai: Supervision. Quan Yuan: Supervision, Methodology. Henglai Wei: Formal analysis, Data curation. Chen Lv: Supervision, Software.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgments

Thanks to our editors and all the reviewers who reviewed the manuscript anonymously. This work was supported by the National Key R&D Program of China (2023YFB2504400); National Natural Science Foundation of China (52372413, 52225212); Beijing Natural Science Foundation (L247007); Overseas training plan for outstanding young and middle-aged teachers and principals in colleges and universities in Jiangsu Province and the Young Talent Cultivation Project of Jiangsu University.

Data availability

Data will be made available on request.

References

Arun, A., Haque, M.M., Washington, S., Sayed, T., Mannering, F., 2021. A systematic review of traffic conflict-based safety measures with a focus on application context. Anal. Methods Accid. Res. 32, 100185. http://dx.doi.org/10.1016/j.amar.2021. 100185.

Arun, A., Haque, M.M., Washington, S., Sayed, T., Mannering, F., 2022. How many are enough?: Investigating the effectiveness of multiple conflict indicators for crash frequency-by-severity estimation by automated traffic conflict analysis. Transp. Res. Part C: Emerg. Technol. 138, 103653. http://dx.doi.org/10.1016/j.trc.2022.103653.

Bhattarai, N., Zhang, Y., Liu, H., Xu, H., 2023. Crash frequency prediction based on extreme value theory using roadside lidar-based vehicle trajectory data. Accid. Anal. & Prev. 193, 107306. http://dx.doi.org/10.1016/j.aap.2023.107306.

Bhowmik, T., Yasmin, S., Eluru, N., 2022. Accommodating for systematic and unobserved heterogeneity in panel data: Application to macro-level crash modeling. Anal. Methods Accid. Res. 33, 100202. http://dx.doi.org/10.1016/j.amar.2021. 100202.

Chen, H., Zhao, X., Li, H., Gong, J., Fu, Q., 2024. Predicting driver's takeover time based on individual characteristics, external environment, and situation awareness. Accid. Anal. & Prev. 203, 107601. http://dx.doi.org/10.1016/j.aap.2024.107601.

Chudik, A., Pesaran, M.H., 2015. Common correlated effects estimation of heterogeneous dynamic panel data models with weakly exogenous regressors. J. Econometrics 188 (2), 393–420. http://dx.doi.org/10.1016/j.jeconom.2015.03.007.

Deng, M., Gluck, A., Zhao, Y., Li, D., Menassa, C.C., Kamat, V.R., Brinkley, J., 2024. An analysis of physiological responses as indicators of driver takeover readiness in conditionally automated driving. Accid. Anal. & Prev. 195, 107372. http://dx.doi.org/10.1016/j.aap.2023.107372.

Driessen, T., Dodou, D., Bazilinskyy, P., de Winter, J., 2024. Putting ChatGPT vision (GPT-4V) to the test: Risk perception in traffic images. R. Soc. Open Sci. 11 (5), 231676. http://dx.doi.org/10.1098/rsos.231676.

Du, N., Zhou, F., Pulver, E.M., Tilbury, D.M., Robert, L.P., Pradhan, A.K., Yang, X.J., 2020. Predicting driver takeover performance in conditionally automated driving. Accid. Anal. & Prev. 148, 105748. http://dx.doi.org/10.1016/j.aap.2020.105748.

Eberhardt, M., Bond, S., 2009. Cross-Section Dependence in Nonstationary Panel Models: a Novel Estimator. Mpra paper, University Library of Munich, Germany, https://EconPapers.repec.org/RePEc:pra:mprapa:17692.

Gold, C., Körber, M., Lechner, D., Bengler, K., 2016. Taking over control from highly automated vehicles in complex traffic situations: The role of traffic density. Hum. Factors 58 (4), 642–652. http://dx.doi.org/10.1177/0018720816634226.

Green, P., 2013. Standard definitions for driving measures and statistics: overview and status of recommended practice J2944. In: Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications. pp. 184–191. http://dx.doi.org/10.1145/2516540.251654.

Hu, J., Bai, J., Yang, J., Lee, J.J., 2025. Crash risk prediction using sparse collision data: Granger causal inference and graph convolutional network approaches. Expert Syst. Appl. 259, 125315. http://dx.doi.org/10.1016/j.eswa.2024.125315.

Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., 2021. LoRA: Low-rank adaptation of large language models. https://arxiv.org/abs/2106. 09685.

International, S., 2021. Taxonomy and definitions for terms related to driving automation systems for on-road motor vehicles. SAE Int. 4970 (724), 1–5, https: //www.sae.org/standards/content/j3016\_202104.

• Islam, S.M., Washington, S., Kim, J., Haque, M.M., 2023. A hierarchical multinomial logit model to examine the effects of signal strategies on right-turn crash risks by crash movement configuration. Accid. Anal. & Prev. 184, http://dx.doi.org/10. 1016/j.aap.2023.106993.
• Jin, J., Huang, H., Yuan, C., Li, Y., Zou, G., Xue, H., 2023. Real-time crash risk prediction in freeway tunnels considering features interaction and unobserved heterogeneity: A two-stage deep learning modeling framework. Anal. Methods Accid. Res. 40, 100306. http://dx.doi.org/10.1016/j.amar.2023.100306.
• Jin, M., Lu, G., Chen, F., Shi, X., Tan, H., Zhai, J., 2021. Modeling takeover behavior in level 3 automated driving via a structural equation model: Considering the mediating role of trust. Accid. Anal. & Prev. 157, 106156. http://dx.doi.org/10. 1016/j.aap.2021.106156.
• Kar, P., Venthuruthiyil, S.P., Chunchu, M., 2023. Non-stationary crash risk modelling of powered two-wheelers using extreme value analysis of surrogate crash events. Accid. Anal. & Prev. 183, 106973. http://dx.doi.org/10.1016/j.aap.2023.106973.
• Li, P., Abdel-Aty, M., Cai, Q., Yuan, C., 2020. The application of novel connected vehicles emulated data on real-time crash potential prediction for arterials. Accid. Anal. & Prev. 144, 105658. http://dx.doi.org/10.1016/j.aap.2020.105658.
• Li, H., Bai, Q., Zhao, Y., Qu, Z., Xin, W., 2021. TSDCN: Traffic safety state deep clustering network for real-time traffic crash-prediction. IET Intell. Transp. Syst. 15 (1), 132–146. http://dx.doi.org/10.1049/itr2.12011.
• Li, S., Blythe, P., Guo, W., Namdeo, A., 2018. Investigation of older driver's takeover performance in highly automated vehicles in adverse weather conditions. IET Intell. Transp. Syst. 12 (9), 1157–1165. http://dx.doi.org/10.1049/iet-its.2018.0104.
• Li, Q., Wang, Z., Wang, W., Zeng, C., Li, G., Yuan, Q., Cheng, B., 2022. An adaptive time budget adjustment strategy based on a take-over performance model for passive fatigue. IEEE Trans. Human- Mach. Syst. 52 (5), 1025–1035. http://dx.doi.org/10. 1109/THMS.2021.3121665.
• Liao, H., Shen, H., Li, Z., Wang, C., Li, G., Bie, Y., Xu, C., 2024. GPT-4 enhanced multimodal grounding for autonomous driving: Leveraging cross-modal attention with large language models. Commun. Transp. Res. 4, 100116. http://dx.doi.org/ 10.1016/j.commtr.2023.100116.
• Lin, Q., Li, S., Ma, X., Lu, G., 2020. Understanding take-over performance of high crash risk drivers during conditionally automated driving. Accid. Anal. & Prev. 143, 105543. http://dx.doi.org/10.1016/j.aap.2020.105543.
• Liu, Q., Li, C., Jiang, H., Nie, S., Chen, L., 2022. Transfer learning-based highway crash risk evaluation considering manifold characteristics of traffic flow. Accid. Anal. & Prev. 168, 106598. http://dx.doi.org/10.1016/j.aap.2022.106598.
• Liu, Q., Wang, X., Liu, S., Yu, C., Glaser, Y., 2024. Analysis of pre-crash scenarios and contributing factors for autonomous vehicle crashes at intersections. Accid. Anal. & Prev. 195, 107383. http://dx.doi.org/10.1016/j.aap.2023.107383.
• Manual, H.C., 2000. Highway capacity manual. Wash. DC 2 (1), 1. http://dx.doi.org/ 10.17226/24798.
• Melnicuk, V., Thompson, S., Jennings, P., Birrell, S., 2021. Effect of cognitive load on drivers' state and task performance during automated driving: Introducing a novel method for determining stabilisation time following take-over of control. Accid. Anal. & Prev. 151, 105967. http://dx.doi.org/10.1016/j.aap.2020.105967.
• Naujoks, F., Wiedemann, K., Schömig, N., Jarosch, O., Gold, C., 2018. Expert-based controllability assessment of control transitions from automated to manual driving. MethodsX 5, 579–592. http://dx.doi.org/10.1016/j.mex.2018.05.007.
• Ou, Y.-K., Huang, W.-X., Fang, C.-W., 2021. Effects of different takeover request interfaces on takeover behavior and performance during conditionally automated driving. Accid. Anal. & Prev. 162, 106425. http://dx.doi.org/10.1016/j.aap.2021.
• Peng, Y., Li, C., Wang, K., Gao, Z., Yu, R., 2020. Examining imbalanced classification algorithms in predicting real-time traffic crash risk. Accid. Anal. & Prev. 144, 105610. http://dx.doi.org/10.1016/j.aap.2020.105610.
• Portron, A., Perrotte, G., Ollier, G., Bougard, C., Bourdin, C., Vercher, J.-L., 2024. Getting back in the loop: Does autonomous driving duration affect driver's takeover performance? Heliyon 10 (3), e24112. http://dx.doi.org/10.1016/j.heliyon.2024.

• Rangesh, A., Deo, N., Greer, R., Gunaratne, P., Trivedi, M.M., 2021. Autonomous vehicles that alert humans to take-over controls: Modeling with real-world data. In: 2021 IEEE International Intelligent Transportation Systems Conference. ITSC, IEEE, pp. 231–236. http://dx.doi.org/10.1109/ITSC48978.2021.9564434.

• Sanghavi, H., Zhang, Y., Jeon, M., 2023. Exploring the influence of driver affective state and auditory display urgency on takeover performance in semi-automated vehicles: Experiment and modelling. Int. J. Hum.-Comput. Stud. 171, 102979. http://dx.doi.org/10.1016/j.ijhcs.2022.102979.

• Singh, S., Ali, Y., Haque, M.M., 2024. A Bayesian extreme value theory modelling framework to assess corridor-wide pedestrian safety using autonomous vehicle sensor data. Accid. Anal. & Prev. 195, 107416. http://dx.doi.org/10.1016/j.aap. 2023.107416.

• Song, Y., Kou, S., Wang, C., 2021. Modeling crash severity by considering risk indicators of driver and roadway: A Bayesian network approach. J. Saf. Res. 76, 64–72. http://dx.doi.org/10.1016/j.jsr.2020.11.006.

• Tobin, D.M., Kumjian, M.R., Black, A.W., 2021. Effects of precipitation type on crash relative risk estimates in kansas. Accid. Anal. & Prev. 151, 105946. http://dx.doi. org/10.1016/j.aap.2020.105946.

• Tseng, H.-T., Hsieh, C.-C., Lin, W.-T., Lin, J.-T., 2020. Deep reinforcement learning for collision avoidance of autonomous vehicle. In: 2020 IEEE International Conference on Consumer Electronics-Taiwan. ICCE-Taiwan, IEEE, pp. 1–2. http://dx.doi.org/ 10.1109/ICCE-Taiwan49838.2020.9258199.

• Wang, B., Chen, T., Zhang, C., Wong, Y.D., Zhang, H., Zhou, Y., 2024. Toward safer highway work zones: An empirical analysis of crash risks using improved safety potential field and machine learning techniques. Accid. Anal. & Prev. 194, 107361. http://dx.doi.org/10.1016/j.aap.2023.107361.

• Wang, Y., Zhang, W., Zhou, R., 2022. Speech-based takeover requests in conditionally automated driving: Effects of different voices on the driver takeover performance. Appl. Ergon. 101, 103695. http://dx.doi.org/10.1016/j.apergo.2022.103695.

• Xing, L., He, J., Li, Y., Wu, Y., Yuan, J., Gu, X., 2020. Comparison of different models for evaluating vehicle collision risks at upstream diverging area of toll plaza. Accid. Anal. & Prev. 135, 105343. http://dx.doi.org/10.1016/j.aap.2019.105343.

• Xu, C., Liu, J., Hang, P., Sun, J., 2025. Tell-drive: Enhancing autonomous driving with teacher LLM-guided deep reinforcement learning. arXiv preprint arXiv:2502.01387. URL https://arxiv.org/abs/2502.01387.

• Yang, C., Chen, M., Yuan, Q., 2021. The application of xgboost and SHAP to examining the factors in freight truck-related crashes: An exploratory analysis. Accid. Anal. & Prev. 158, 106153. http://dx.doi.org/10.1016/j.aap.2021.106153.

• Yang, Y., He, K., Wang, Y.-p., Yuan, Z.-z., Yin, Y.-h., Guo, M.-z., 2022. Identification of dynamic traffic crash risk for cross-area freeways based on statistical and machine learning methods. Phys. A 595, 127083. http://dx.doi.org/10.1016/j.physa.2022. 127083.

• Yoon, S.H., Kim, Y.W., Ji, Y.G., 2019. The effects of takeover request modalities on highly automated car control transitions. Accid. Anal. & Prev. 123, 150–158. http://dx.doi.org/10.1016/j.aap.2018.11.018.

• Yuan, J., Abdel-Aty, M.A., Yue, L., Cai, Q., 2021. Modeling real-time cycle-level crash risk at signalized intersections based on high-resolution event-based data. IEEE Trans. Intell. Transp. Syst. 22 (11), 6700–6715. http://dx.doi.org/10.1109/TITS. 2020.2994126.

• Zhang, Y., Ma, Q., Qu, J., Zhou, R., 2024a. Effects of driving style on takeover performance during automated driving: Under the influence of warning system factors. Appl. Ergon. 117, 104229. http://dx.doi.org/10.1016/j.apergo.2024.104229.

• Zhang, K., Zhou, F., Wu, L., Xie, N., He, Z., 2024b. Semantic understanding and prompt engineering for large-scale traffic data imputation. Inf. Fusion 102, 102038. http://dx.doi.org/10.1016/j.inffus.2023.102038.

• Zhao, J., Liu, P., Li, Z., 2024. Exploring the impact of trip patterns on spatially aggregated crashes using floating vehicle trajectory data and graph convolutional networks. Accid. Anal. & Prev. 194, 107340. http://dx.doi.org/10.1016/j.aap.2023. 107340.

• Zhou, Z., Huang, H., Li, B., Zhao, S., Mu, Y., 2024. SafeDrive: Knowledge-and data-driven risk-sensitive decision-making for autonomous vehicles with large language models. http://dx.doi.org/10.1109/ICCE-Taiwan49838.2020.9258199, arXiv preprint arXiv:2412.13238.

• Zhu, J., Ma, Y., Zhang, Y., Zhang, Y., Lv, C., 2023. Takeover quality prediction based on driver physiological state of different cognitive tasks in conditionally automated driving. Adv. Eng. Inform. 57, 102100. http://dx.doi.org/10.1016/j.aei.