category: literaturenote citekey: wangensuringsotifenhancedobject2025 title: "Ensuring SOTIF: Enhanced object detection techniques for autonomous driving" authors: "Wang, Sifen; Wang, Zhangyu; Hong, Sheng; Wang, Pengcheng; Zhang, Shaowei" year: 2025 date: 2025-00-00 2025 doi: 10.1016/j.aap.2025.108094 publication: "Accident Analysis & Prevention" url: "https://doi.org/10.1016/j.aap.2025.108094" zotero_key: 8G98D6BT zotero_storage: 268HFEKX collections: transpordi_planeerimine folder: 001_artiklid firstAuthor: "Wang, Sifen"

status: converted

Contents lists available at ScienceDirect

Accident Analysis and Prevention

journal homepage: www.elsevier.com/locate/aap

Ensuring SOTIF: Enhanced object detection techniques for autonomous driving☆

Sifen Wang a,b , Zhangyu Wang ^a , Sheng Hong ^c , Pengcheng Wang c,* , Shaowei Zhang ^d

^a School of Transportation Science and Engineering, Beihang University, China
^b Beijing Institute of Control Engineering, Beijing 100190, China
^c School of Cyber Science and Technology, Beihang University, Beijing, China
^d Dongfeng Motor Corporation Research and Development Institute, Wuhan, China

ARTICLE INFO

Keywords: Object detection SOTIF Prediction extension box Autonomous Driving

ABSTRACT

Neural networks' insufficient interpretability can lead to unguaranteed Safety of the Intended Functionality (SOTIF) issues when perceptual results are not always met in autonomous driving applications. To address the safety shortcomings in the current object detection process, this study proposes an object detection algorithm to enhance the accuracy of the perception system's detection. We utilize the classical one-stage object detection algorithm YOLO v5 as the baseline in this study and evaluate our proposed model. A prediction extension box is added to the classical YOLO v5 model, which considers the coverage range and redundancy of real targets, guaranteeing the safety of image perception. The proposed object detection algorithm has been shown to increase the coverage range of detected targets, which significantly enhances perception safety in the autonomous driving process.

1. Introduction

As automotive technology progresses, automotive systems are integrating more sophisticated intelligence, navigating a broader spectrum of road conditions and confronting increasingly intricate decisionmaking challenges (Deng et al. 2023, Prakash et al. 2024). The conventional approach to functional safety, which was once adequate, now struggles to address the unpredictability and complexity of real-world driving scenarios. Consequently, there is a growing emphasis on addressing the Safety of the Intended Functionality (SOTIF), a concept that encompasses the safety considerations beyond traditional safety measures to ensure that the automated vehicle (AV)'s intended functionalities perform as expected under a wide array of conditions, including those that may not have been anticipated during the design phase (Li et al. 2022, Zhang et al. 2021).

The SOTIF framework is designed to mitigate risks associated with autonomous driving that arise from issues such as design flaws or operational constraints, ensuring they are kept within tolerable limits (Saberi et al. 2020, Wang et al.,2024a; Wang et al., 2024b). In the realm of autonomous driving, deep learning neural networks play a pivotal role in perception modules, which are responsible for interpreting the

vehicle's surroundings. Despite their widespread use, these networks often suffer from a lack of transparency, which can lead to uncertainties in the accuracy of perception. This inconsistency poses significant challenges for maintaining the high safety standards required in autonomous driving systems (Adee et al. 2021). To address these challenges, it is crucial to enhance the interpretability of neural networks and refine perception algorithms to reliably meet the stringent demands of autonomous driving applications.

Scholars from a variety of countries have conducted investigations and research to tackle the SOTIF issues that autonomous driving encounters. For example, Michelmore et al. (2018) explored the use of uncertainty in end-to-end controllers for self-driving cars and suggested new evaluations for it. Wu et al. (2023) investigated the effectiveness and performance of reinforcement learning in autonomous driving scenarios. They proposed a new reinforcement learning approach based on an uncertainty perception model, demonstrating excellent performance in various autonomous driving scenarios and proving the feasibility and effectiveness of the method. Furthermore, Khatun et al. (2020) investigated the current state of scene-based hazard analysis and risk assessment methods. They proposed an extended approach that integrates the relationship and interaction between functional safety and

E-mail address: pcwang@buaa.edu.cn (P. Wang).

^☆ This article is part of a special issue entitled: 'Safety validation' published in Accident Analysis and Prevention.

^* Corresponding author.

SOTIF, demonstrating excellent performance in the context of lateral brake assistance systems. Meyer et al. (2022) expanded the established model-based system engineering process with additional use cases and scenarios. Peng et al. (2024) presented the Self Surveillance and Self-Adaption System as a systematic approach to online minimize the SOTIF risk, which provides a systematic solution for monitoring, quantification, and mitigation of inherent and external risks. Wang et al. (2024a); Wang et al. (2024b) presented a silent testing method to specifically address the validation of AV perception, utilizing a lane marking detection algorithm, and showed its effectiveness in evaluating AV perception and validating AV systems.

Autonomous driving relies heavily on object detection to ensure the safety and efficiency of vehicle operations. In safety–critical domains like autonomous driving, errors by the object detector may endanger pedestrians and other vulnerable road users. A class of related works that deal with safety focused object detection has been investigated. Volk et al. (2020) proposed a safety metric that incorporates all these parameters and returns a single easily interpretable safety assessment score for object perception. Bansal et al. (2021) introduced the Risk Ranked Recall metrics for object detection systems, which include three risk ranks based on an objective cyber-physical model for the risk of collision. Ceccarelli and Montecchi (2023) presented object detection measures that reward the correct identification of objects that were most dangerous and most likely to affect driving decisions. Lyssenko et al. (2024) developed a safety-enhanced loss function modification that incorporates pedestrian-specific criticality estimation during the training process. This approach utilizes a reachable set-based time-to-collision metric derived from motion analysis, combined with spatial distance measurements, to effectively quantify and mitigate potential collision risks in worst-case scenarios.

While advancements have been made in enhancing safety and risk assessment for object detection systems, there is still a need for more detailed research and development of specific improvement methods. In this paper, we introduce an enhanced algorithm for predicting bounding boxes in image object detection, seamlessly integrated with the YOLO v5 algorithm to bolster the safety of the detection process.

The primary innovation and contribution of this study lies in a novel method for extending prediction bounding boxes to improve the utility of the object detection for collision avoidance in AVs. In contrast to conventional approaches, the proposed method offers superior precision and augmented perception capabilities for automated vehicles. This study elucidates the application of this method for object detection within images. Furthermore, through empirical analysis, we substantiate the efficacy of this method in assessing the perceptual abilities of automated vehicles. This advancement not only contributes to the field but also furnishes technical support and assurance for the progression towards intelligent transportation systems.

The structure of this paper follows the following: Section 2 explains the improved algorithm for image prediction boxes. The proposed methods are experimentally analyzed in Section 3, and Section 4 wraps up the paper with a summary of findings and suggestions for future research directions.

2. Extended prediction box for image object detection

2.1. Basic idea of SOTIF

The SOTIF standard provides guidance on system analysis, specification, design, and the identification and evaluation of potential functional insufficiencies and conditions. It also addresses risks associated with SOTIF. A scenario-based method is proposed for verification and validation. The scenarios are categorized into four groups: known & not hazardous (1), known & hazardous (2), unknown & hazardous (3), and unknown & not hazardous (4). The main objective is to identify unknown & hazardous scenarios (3) and convert them into known & hazardous scenarios (2) through simulations, vehicle testing, and longterm assessments. Additionally, known & hazardous scenarios (2) can be transformed into known safety scenarios (1) by enhancing functional performance or restricting their use, while detected unknown & not hazardous scenarios (4) can be classified as known & not hazardous scenarios (1). By applying these methods, the distribution of scenarios can transition from Fig. 1a to Fig. 1b. The area size indicates the number of scenarios rather than the associated risks. SOTIF's role is to provide evidence that the risk level is sufficiently low. While the number of scenarios contributes to this evidence, it is not the sole factor. The potential severity of damage and the likelihood of hazardous situations impacting the intended function are not represented in these categories.

The process for developing SOTIF is outlined as follows. Initially, relevant elements are defined, and hazard events are identified. Following this, hazard identification and risk assessment are conducted based on the identified hazards. Once a hazard event is recognized, it must be determined whether the associated risk is acceptable. If the risk is deemed unacceptable, the next step involves identifying and evaluating the trigger conditions for the hazard from a causal perspective. If the risk from the trigger conditions is deemed unreasonable, modifications and improvements to the function are necessary. Conversely, if the risk is considered reasonable, a verification and validation strategy is formulated for both known and unknown hazard scenarios to demonstrate that the residual risk at the vehicle level is below the acceptable threshold. The release of SOTIF should only occur when the evaluated risk is sufficiently low.

In this study, we aim to reduce the unknown hazard domains in image recognition, i.e., area 3 in Fig. 1b. Therefore, our goal is to minimize the possibility of image misrecognition as much as possible. Based on this consideration, we propose an extended detection box theory to cover the images as comprehensively as possible in order to avoid misrecognition. More details will be provided in the following section.

2.2. Analysis of relation between the predication box and the ground truth box

In autonomous driving, object detection systems are in high demand, as erroneous detections or miscalculations in spatial positioning can lead to catastrophic outcomes. The inherent unpredictability of deep learning models, often likened to a "black-box" due to their opaque decisionmaking processes, introduces a layer of uncertainty that can escalate the risk of system of the test and failures at the algorithmic level. Particularly in image object detection, the representation of positional data through bounding boxes is critical; any misestimation in the dimensions or placement of these boxes can lead to dangerous incidents such as collisions or scrapes. To mitigate these risks and strengthen the SOTIF framework, it is imperative to refine the detection algorithms.

This section introduces an innovative algorithm designed to enhance the accuracy of image prediction boxes, which is a critical component in the object detection process. In the context of the YOLO (You Only Look Once) algorithm, the loss function is a composite of three key elements: the classification loss, which ensures the correct identification of objects; the localization loss, which fine-tunes the precision of the bounding box coordinates; and the confidence loss, which measures the model's certainty in its predictions. Among these, the localization loss plays a pivotal role in refining the exactness of the prediction box positioning, thereby reducing the potential for SOTIF-related hazards and ensuring a higher degree of safety in autonomous driving applications.

The Intersection Over Union (IOU) loss function is defined as Eq. (1). A and B are ground truth box and prediction box respectively. G-IOU LOSS, D-IOU LOSS and C-IOU LOSS are further improvements to IOU loss function.

Fig. 1. The goal of SOTIF.

$$IOU = \frac{A \cap B}{A \cup B}$$

$$IOU \ LOSS = -\ln(IOU)$$ (1)

C-IOU also considers the aspect ratio of prediction box, which is defined as Eq. (2). b and bgt represent prediction box and ground truth box respectively, and ρ is used to calculate the Euclidean distance between two points. C represents the diagonal distance of the smallest closed area. α is adjust parameters, and v measures the aspect ratio. When the aspect ratio of prediction box and ground truth box is consistent, v = 0.

$$C - IOULOSS = 1 - IOU + \frac{\rho^2(b, b^{gr})}{C^2} + av$$

$$a = \frac{v}{(1 - IOU) + v}$$

$$v = \frac{4}{\pi^2} \left(\arctan \frac{w^{gr}}{h^{gr}} - \arctan \frac{w}{h}\right)^2$$ (2)

Through the developmental process of IOU mentioned above, it is evident that, in the pursuit of aligning predicted boxes with actual boxes, various IOU metrics have been proposed to characterize the matching relationship between two bounding boxes. The loss results may be similar for different predicted boxes. In such cases, the loss function deems the situations as identical. However, from a safety perspective, similar-sized loss functions could indicate various safety risks.

To ensure expected functional safety, it is crucial to predict target boxes and category recognition during autonomous driving target detection. The prediction of bounding boxes is closely tied to IOU. Different IOUs exhibit different optimization tendencies. IOUs oriented towards "efficiency" may not adequately consider the safety aspects of bounding boxes.

Taking pedestrian detection as an example and considering the performance and safety of IOU simultaneously, it can be categorized into the following four scenarios, as illustrated in Fig. 2. In this representation, the red solid line box denotes the prediction box, while the gray

Fig. 2. Comparison for definitions of IOU and safety based on SOTIF. Fig. 3. Schematic diagram of prediction box expansion.

dotted line box represents the ground truth box. In the first scenario, the prediction box is away from the ground truth box, resulting in no overlap and an IOU of 0, indicating a high level of unsafety. In the second scenario, the prediction box and the ground truth box mostly coincide, yielding a good IOU; however, there is still a slight safety deficiency since a portion of the detection target remains uncovered, potentially leading to collisions in subsequent decision-making and control due to insufficient redundancy. In the third scenario, although the prediction box completely encompasses the ground truth box, it extends too far beyond it, resulting in a small IOU. Nevertheless, safety is ensured, even though it may impact decision-making and planning efficiency. In the fourth scenario, both IOU and safety are optimal, with the prediction box almost perfectly coinciding with the ground truth box, achieving a balance between efficiency and safety.

To ensure safety during object detection, it is advisable to expand the size of the prediction box to encompass the ground truth box, as shown in Fig. 3. Specifically, for a prediction box, its width and height are w and h respectively, and its center point is O; After expansion, the width and height of the new prediction box are *w*ʹ and *h*ʹ respectively, and the center point is *O*ʹ .

The extended mapping relationship needs to meet the requirement of Eq. (3):

$$\begin{array}{l} k' = k_w w \ h' = k_h h \end{array} \tag{3}$$

where kw is the width expansion factor and kh is the height expansion factor.

Before the depth neural network outputs the final detection result, the IOU of the generated candidate prediction box must exceed the threshold, i.e.

$$IOU > a$$ (4)

where α is the IOU threshold and the super parameter of the depth neural network.

The upper bound of the expansion coefficient is directly related to α. As shown in Fig. 4, the half length of ground truth box A is ra, and the half length of prediction box B is rb, and the distance between the length of ground truth box A and prediction box B is r.

Given that the IoU of the ground-truth bounding box A and the predicted bounding box B satisfies a threshold, we have:

$$IOU = \frac{A \cap B}{A \cup B} \geqslant \alpha \tag{5}$$

Converting this to the ratio of widths, we have:

$$\frac{r + r_b}{2r_a + r_b - r} \geqslant \alpha \tag{6}$$

To ensure coverage of the ground-truth bounding box, for the expansion coefficient, the following must always hold:

$$r + k_w r_b \geqslant 2r_a \tag{7}$$

Thus, in the worst-case scenario, the minimum width expansion coefficient that can be taken is:

$$k_w = \frac{2}{\alpha} - 1 \tag{8}$$

Similarly, for the height expansion coefficient, the minimum height expansion coefficient is

$$k_h = \frac{2}{\alpha} - 1 \tag{9}$$

On the other hand, the expansion coefficient does not account for shrinking the predicted bounding box. Hence, we have

$$1 \leq k_w \leq \frac{2}{\alpha} - 1$$

$$1 \leq k_h \leq \frac{2}{\alpha} - 1$$ (10)

2.3. Analysis of the extension prediction box

This section attempts to expand the predicted bounding box to obtain a safety margin. In fact, if the expansion algorithm adopts the theoretical upper bound, i.e., when α is 0.5, all predicted bounding boxes need to be expanded threefold to completely cover. However, this is practically unacceptable since the IoU between most predicted and ground-truth bounding boxes is much greater than 0.5. For example, based on Eq. (10), when the threshold α is 0.8, the expansion coefficient only needs to be 1.50, and when the threshold α is 0.9, the expansion coefficient only needs to be 1.22. For a well-trained model, the average IoU between the predicted and ground-truth bounding boxes is often greater than 0.9,

and in this case, the expansion coefficient can be controlled within 1.22.

Therefore, in response to the essential requirements of SOTIF, a reasonable method is needed to determine the specific value of the expansion coefficient. To measure the coverage of the ground-truth bounding box, we utilize the IOG (Intersection over Ground Truth) as an indicator, defined in prior work (Bansal et al. 2021) as shown in the following formula:

$$IOG = \frac{|A \cap B|}{|A|} \tag{11}$$

where A is ground truth box; B is the prediction box.

We can find that the value range of IOG should be [0, 1]. When the prediction box and the ground truth box are completely disjoint, i.e., IOG is 0. Conversely, when the prediction box completely contains the ground truth box, i.e., IOG is 1. From a safety perspective, an IOG = 0 indicates a lack of safety, while IOG = 1 signifies that safety requirements are fully met.

Regarding the expansion coefficient, it is essential to strike a balance. On one hand, we aim for an appropriate value to ensure sufficient coverage of the target bounding box, thereby enhancing safety redundancy. On the other hand, an excessively large expansion coefficient may compromise efficiency, making it crucial to avoid uncontrolled growth. To facilitate proper training, the loss function is designed as follow.

$$L_{\text{ex}}(s, l, g, k_w, kh) = \frac{1}{N} \left( L'{\text{loc}}(s, l, g, k_w, k_h) + \lambda P(k_w, k_h) \right)$$ (12)

where *L*ex is the loss function of bounding extension box; s is the matching index, which is a 0–1 variable, s = 1 indicates that the bounding box matching is successful, and s = 0 indicates that the matching is failed; l and g are prediction box and ground truth box; N is the total number of samples in the training; *L*ʹ loc is the coverage loss of prediction box; P is the expansion coefficient penalty function, and λ is the weight parameter of the penalty function.

The prediction box coverage loss function *L*ʹ loc is defined as in Eq. (13). The value range is mapped to [0, + ∞). When the prediction box completely covers the ground truth box, the loss function value is 0.

$$L'_{loc}(s, l, g, k_w, k_h) = -\ln(IOG(s, l, g, k_w, k_h))$$ (13)

In order to improve the efficiency and the boundedness of the loss function, change Eq. (7) into Eq. (8). At this time, the value range is mapped to [0*,*1]. When the prediction box completely covers the ground truth box, the loss function also takes a value of 0.

$$L'_{loc}(s, l, g, k_w, k_h) = 1 - IOG(s, l, g, k_w, k_h)$$ (14)

Because too large boundary expansion coefficient will reduce efficiency, it needs to be subject to soft constraints. The expansion coefficient penalty function is shown in Eq. (15).

$$P(k_w, k_h) = (k_w - 1)^{m_1} + (k_h - 1)^{m_1} + (\gamma - k_w)^{m_2} + (\gamma - k_h)^{m_2}$$ (15)

where γ is the theoretical upper bound under the corresponding threshold α; *m*1 and *m*2 are exponential parameters, m¹ = 2, m² = − 4.

2.4. Application of the proposed method based on YOLO v5

This study utilizes the YOLO v5 algorithm network structure as presented in previous studies (Jiang et al. 2022, Wang et al. 2022). Based on the classical YOLO v5 network, the extending bounding box is integrated, as shown in Fig. 5. Similar to the classical YOLO v5 architecture, the proposed network is mainly divided into three parts: Backbone, Neck, and Prediction. At the input stage, this network employs Mosaic data augmentation, which enhances training efficiency and robustness by stitching together different images. This approach im-Fig. 4. Schematic diagram of boundary box. proves the accuracy of recognition during the detection process,

Fig. 5. YOLO v5 network architecture based on the extension predicting box.

ensuring correct identification even in certain low-frequency scenarios. The Backbone part extracts feature information from the input images through continuous convolution and other operations, utilizing Focus and CSP structures. The Neck part employs FPN and PAN structures; FPN uses a top-down approach to pass global semantic feature information downwards through a larger receptive field, while PAN uses a bottom-up approach to acquire more precise localization information with a smaller receptive field (Yu et al. 2023). The combination of these two structures allows for more accurate semantic and localization information. The Prediction part is responsible for completing the predictions, outputting the classification and location information of the detected objects.

After processing through the FPN and PAN in the Neck part, the predicted bounding boxes are obtained as shown in Fig. 6. The features output by the PAN are then processed through the bounding box extension feature extraction network. These features, along with the predicted bounding box results, are inputted into the new network. After further processing by the network, the final extension coefficients are obtained.

It should be noted that while newer versions of YOLO (e.g., YOLOv8 and YOLOv11) are available, we selected YOLOv5 for several reasons (Jegham et al. 2024, Khekan et al. 2025). First, YOLOv5 is a mature and stable version with extensive community support, well-documented implementations, and a proven track record in various applications. This makes it a reliable choice for our research. Second, YOLOv5 provides a flexible framework that allows for easy customization and integration of additional features, such as the extended prediction boxes proposed in our work. Third, YOLOv5 still delivers competitive performance in terms of accuracy and speed, particularly when optimized for specific tasks like autonomous driving.

This section discusses the calculation principles of the IOU and IOG formulas between predicted boxes and ground truth boxes. The purpose of using IOG came from SOTIF principles, which emphasize reducing "hazardous omissions" (missed detections). It should be noted that the

Fig. 6. YOLOv5 Detection Results.

critical distinction between IoU and IoG, particularly the asymmetric nature of IoG, which focuses on the coverage of the ground truth box by the prediction. While IoG is effective for ensuring that dangerous objects are sufficiently detected, as shown in (Bansal, et al. 2021), its application to precision may lead to inflated values, especially when the prediction box is excessively large. In fact, to avoid the excessively large predication box, we added the expansion coefficient penalty function in Eq. (12). Based on this consideration, we use different metrics during training and testing phases for specific purposes. We employ IOG during training stage to prioritize safety. For example, imagine a pedestrian crossing the road, even if our detection box is slightly larger than ideal, we want to ensure we never miss detecting them. IOG helps achieve this by focusing on covering the ground truth object. The IOG formula, as shown in Eq. (11), means measuring how much of the actual object our prediction box covers, rather than how precisely our box matches it. In testing stage, we switch back to the standard IOU metric, which is align with the traditional evaluation method. We want to verify that our detections are not just safe, but also precise. Using IOU prevents the system from getting credit for overly large boxes that might appear successful under IOG but aren't practically useful.

Subsequently, we will characterize the coverage relationship between predicted boxes and ground truth boxes and use specific image recognition models to validate the effectiveness of the proposed methods in the next section.

3. Experiment analysis

3.1. Dataset

In this experiment, we conducted the experiment on the KITTI dataset, which synchronizes the capture of scenes in mid-sized cities, rural areas, and highways using cameras in conjunction with a LiDAR scanner and a positioning system (Geiger et al. 2012). This dataset contains 7481 training and 7518 test images. Due to the fact that the ground truth labels are available only for training data, we split the original training set into the training part consisting of 5985 images, the testing part consisting of 748 images, the validating part consisting of 748 images. The KITTI dataset contains a total of nine label types: Car, Van, Truck, Pedestrian, Person_sitting, Cyclist, Tram, Misc, and DontCare. Among these labels, DontCare refers to areas that are labeled but not of primary concern and are therefore excluded during the data preprocessing stage. The categories Pedestrian and Person_sitting is quite similar and do not differ significantly in the context of autonomous driving scenarios, so these two labels are merged into a single label, Pedestrian. Similarly, the Truck and Misc labels are combined into one label, Truck. After processing, the nine labels are reduced to six for training: Car, Cyclist, Truck, Van, Pedestrian, and Tram.

Once the training is complete, the weight file can be called to perform object detection in images. It can be observed that YOLOv5 accurately identifies vehicles, pedestrians, and other objects in street scenes. The network also performs well in recognizing vehicles that are partially occluded by others on the street, and it can detect distant pedestrians, even when only a few pixels are visible.

3.2. Analysis of the extension coefficients

In the previous discussion, we derived the theoretical upper bound of the extension coefficient, as shown in Eq. (10). It can be observed that smaller values of α correspond to much larger values of k. When α takes the value of 1, the worst-case scenario requires an extension factor of 19 times. As α increases, the value of k decreases significantly; when takes the value of 0.9, the worst-case scenario only requires an extension factor of 1.22 to completely cover the actual bounding box. In common deep learning algorithms for object detection, the value typically is greater than or equal to 0.5.

After analyzing the theoretical upper bound, nine different thresh-

olds were selected to analyze the detection results, with the results shown in Table 1. Here, k represents the theoretical upper bound of the extension coefficient, *kw,*max represents the maximum value of the width extension coefficient in the detection results, *kh,*max represents the maximum value of the height extension coefficient in the detection results, *kw,*ave represents the average value of the width extension coefficient in the detection results, and *kh,*ave represents the average value of the height extension coefficient in the detection results.

Fig. 7 presents the relationship between the extension coefficient and IOU. From the figure, it can be observed that when the IOU is low, the theoretical upper bound for the extension is quite large, while the maximum extension value in the detection dataset is significantly lower than the theoretical value. This indicates that even at low IOU levels, the output of the object detection network will still cover parts of the target, and worst-case scenarios are rare. As the IOU threshold increases, the difference between the maximum value and the theoretical value gradually decreases, suggesting that it becomes easier to encounter situations where the value slightly exceeds the threshold, leading to an increased occurrence of theoretical worst-case IOU scenarios. When the IOU continues to rise, the difference between the theoretical value and the maximum value becomes even smaller, both approaching 1. However, setting an excessively high IOU threshold can severely impact recall, which is why very high IOU thresholds are seldom used. On the other hand, it is noteworthy that the average extension coefficient in the detection dataset is consistently smaller than the maximum value, indicating that poor scenarios are relatively rare. In other words, in most cases, only a slight extension of the bounding box is necessary to achieve better coverage of the detected targets.

3.3. Analysis of the detection results

This section presents the comparison of prediction box and extended prediction box, which uses the YOLO v5 objection detection model as the baseline.

The model's performance is evaluated using various technical metrics, such as Precision, Recall, mean Average Precision (mAP). These metrics provide a quantitative assessment of the model's effectiveness and efficiency, which are discussed in the following sections.

Precision (P) in classification measures the accuracy of positive predictions and indicates the proportion of correctly identified positive instances among all instances predicted as positive. It is calculated as the ratio of true positives to the total of true positives and false positives, as shown in the equation.

$$P = \frac{TP}{TP + FP} \tag{16}$$

Recall (R) measures the proportion of correctly identified positive cases out of all actual positive instances, as outlined in the following equation:

$$R = \frac{TP}{TP + FN} \tag{17}$$

In Eqs. (16) and (17)), FP refers to the number of detections found only in the results and not in the ground truth, while TP indicates the detections present in both the ground truth and the results. FN represents detections that exist only in the ground truth and are absent in the results.

Precision reflects the model's ability to distinguish negative samples. The higher the precision, the stronger the model's ability to distinguish negative samples. Recall reflects the model's ability to identify positive samples. The higher the recall, the stronger the model's ability to identify positive samples.

The average precision (AP) is calculated individually for each class (Padilla et al. 2020, Wang and Cheung 2022), which is the average value of the highest precision under different recall conditions. The calculation formula is as follows:

Table 1 Extension Coefficients and Threshold.

α	0.1	0.2	0.3	0.4	0.5	0.6	0.7	0.8	0.9
k	19.000	9.000	5.667	4.000	3.000	2.333	1.857	1.500	1.222
kw,max	3.921	2.692	2.448	2.413	2.175	2.032	1.598	1.411	1.119
kh,max	2.736	2.312	2.039	1.764	1.511	1.499	1.326	1.214	1.101
kw,ave	1.079	1.072	1.070	1.065	1.061	1.057	1.041	1.034	1.024
kh,ave	1.054	1.051	1.044	1.041	1.041	1.036	1.032	1.024	1.015

Fig. 7. Extension Coefficient vs. IOU Threshold.

$$AP = \int_0^1 P(R)dR = \sum P(i)\Delta R(i)$$ (18)

The mean AP (mAP) is a metric used to measure the accuracy of object detectors over all classes in a specific database. It calculates the average precision at different recall levels, offering a thorough evaluation of the model's ability to accurately identify objects in a specific context. A higher mAP score indicates improved algorithm performance. The mAP is simply the average AP over all classes, that is

$$mAP = \frac{1}{N} \sum_{i=1}^{N} AP_i$$ (19)

In this equation, N denotes the total number of classes, and APⁱ represents the Average Precision for each class.

Based on Eq. (19), the model training results are presented in Table 2. From the table, it is evident that the mAP of the trained YOLO v5 model without extended predication box is 84.71 %. Employing the extended predication box on YOLO v5 model, the mAP is 90.59 %. We can find that our proposed prediction box extension method could significantly improve the object detection precision.

It should be noted that to better evaluate the precisions between YOLO v5 model without extended predication box and YOLO v5 model with extended predication box, this study adopts IOU as the criterion for measuring the accuracy of detecting corresponding objects in the specific data set. Due to the fact that each value of IOU threshold provides a different AP metric, and thus the threshold used must always be indicated. In this study, the IOU threshold is set as 0.5.

As shown in Fig. 8, after expansion, the left side of the vehicle is adequately covered, but there is some redundancy on the right side. Given the vehicle's proximity, prediction box extension offers more precise edge detection and higher perception confidence, thus holding a dominant role in the detection process. The resulting outcomes ensure both accuracy and safety.

Furthermore, to verify the detection capability for SOTIF, the averaged IOU and IOG metrics is calculated after model inference. In specific, each bounding box of the ground truth is matched with the best prediction result according to the IOU or IOG between them. Then, these IOU or IOG scores for all ground truths are averaged. As shown in Fig. 9, based on the original YOLOv5 object detection algorithm, the IOU of the predicted bounding boxes is 0.912, while the IOG is 0.934. In fact, the IOG is almost always greater than or equal to the IOU, as indicated in Eq. (20). After applying the bounding box extension algorithm, the IOU changed to 0.898, which is a slight decrease of 0.014 compared to the predicted bounding boxes; however, the IOG increased to 0.972, which is an significant increase of 0.038 compared to the predicted bounding boxes. Thus, it can be generally concluded that the proposed method is effective to enhance the crucial metric of SOTIF.

$$IOU = \frac{|A \cap B|}{|A \cup B|} \leqslant \frac{|A \cap B|}{|A|} = IOG$$ (20)

As previous studies (Rani et al. 2024, Rashmi and Chaudhry, 2024), we know that the predicted bounding boxes generated by YOLOv5 are accurate in most cases, which means that the required extension coefficients are generally close to 1. Larger extension coefficients are only needed for targets where the bounding box coverage is relatively poor.

4. Conclusion

The advancement of information technology and deep learning has resulted in significant advancements in autonomous driving technology. Yet, its practical application is still confronted with a myriad of challenges. The intricacy of driving scenarios and unforeseen elements present difficulties for the perception, decision-making, and control components of autonomous vehicles. In the field of autonomous driving, deep learning algorithms are extensively utilized for perceptual decision-making. Despite extensive theoretical research, the inner workings of deep learning algorithms are not fully transparent, and incorrect predictions can have catastrophic consequences.

To tackle these challenges, the SOTIF framework has been developed. It aims to minimize safety risks that stem from design shortcomings or performance limitations during the autonomous driving process to a tolerable level. Our research, within the SOTIF framework, concentrates on the safety of object detection within the perception phase of autonomous driving. The objective is to enhance the accuracy of target detection, providing a robust foundation for subsequent decisionmaking and control processes, and thereby strengthening SOTIF compliance in autonomous driving. Recognizing that detection

Table 2 Model training results.

Model	Car	Cyclist	Truck	Van	Tram	Pedestrian	mAP
YOLO v5(%)	89.24	82.13	88.51	87.76	89.21	71.38	84.71
without extended predication box YOLO v5(%) with extended predication box	94.17	89.28	94.44	91.69	93.76	80.25	90.59

Fig. 8. Object detection with prediction box extension method based on YOLO v5.

Fig. 9. Comparison of averaged IOU and IOG to verify SOTIF.

algorithms often focus on maximizing the IOU to improve accuracy, this approach may neglect safety considerations for certain targets. Our paper introduces an algorithm that expands the prediction boxes generated by the YOLO v5 neural network. This expansion ensures more comprehensive target coverage and maintains the safety of the image perception process. This enhancement adds an extra layer of safety and redundancy to the detection process, safeguarding the overall driving experience. By integrating these algorithms with a perception fusion method, we achieve a comprehensive perception result that evaluates risks across the prediction process. Empirical testing has shown that our proposed prediction extension box with IOG metric enhances the precision compared to traditional algorithms without extension box, resulting in a significant increase in detection safety and reliability.

It should be noted that this study aims to use the extending predication boxes to improve the precision of object detection. As we know that higher object detection precision could reduce the likelihood of false negatives or missed detections, which is a major cause of collisions in real world scenarios. Hence, the proposed extending prediction boxes could reduce unknown hazardous areas by improving the precision of detected objects, which is an critical step to enhance the automated vehicle's SOTIF.

As future work, first, we are currently working towards object detectors in the safety critical domain. Such an object detector with safety parameters model assigns a criticality score to each object, based on ground truth and estimated object distance, colliding trajectory, and time to collisions. Second, we plan to explore adaptive extending bounding box based on object type, velocity, and environmental context. This could help reduce false positives while maintaining a high level of safety. Because extending bounding boxes could lead to regions without obstacles being erroneously marked as hazardous, effectively introducing false positives.

CRediT authorship contribution statement

Sifen Wang: Writing – original draft. Zhangyu Wang: Formal analysis, Conceptualization. Sheng Hong: Visualization, Validation. Pengcheng Wang: Writing – review & editing. Shaowei Zhang: Methodology, Software.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgement

This work was supported by the National Key Research and Development Program of China, Grant #2022YFB2503002.

Data availability

No data was used for the research described in the article.

wangensuringsotifenhancedobject2025.md 52 KB

Historia Czysty

status: converted

Accident Analysis and Prevention