Ensemble Learning for Multi-Task Road Safety Attributes Classication

Abstract

6 Road tra c crashes claim 1.19 million lives annually and remain the leading cause of death for 7 people aged 529 years. Safe road infrastructure is among the most eective countermeasures for 8 reducing this burden. The International Road Assessment Programme (iRAP) oers a globally 9 recognised framework that requires trained coders to manually identify up to 50 physical road at10 tributes every 10-metres from survey images for road infrastructure assessment and improvements. 11 While the iRAP methodology provides a robust and globally validated framework for infrastruc12 ture safety assessment, the current reliance on manual coding can be resource-intensive and may 13 limit scalability, particularly in low- and middle-income contexts. Although computer vision ap14 proaches have been proposed for automated attribute classi cation, few studies have systematically 15 examined ensemble learning to enhance performance across multiple safety attributes. This paper 16 addresses this gap by evaluating ensemble voting strategies for road attributes classi cation task. 17 Results show that ensemble methods outperform individual baseline models. 18 Keywords: automated road assessment, ensemble learning, iRAP, multi-task classi cation, vision 19 transformers, road safety

20 1 Introduction 21 Road crashes remain a critical global public health challenge, claiming 1.19 million lives each 22 year. The burden falls disproportionately on low- and middle-income countries, which account 23 for 92 per cent of all road tra c fatalities despite having a fraction of the world's registered 24 vehicles (WHO, 2023). A substantial share of road crashes is attributable to inadequate road 25 infrastructure: poor geometric design, insu cient delineation, lack of pedestrian facilities, and 26 absence of appropriate roadside treatments are well-established risk factors for crash occurrence 27 and severity (Papadimitriou et al., 2019) . Designing safer roads and prioritising infrastructure 28 investment therefore requires accurate, large-scale road infrastructure assessment. 29 The International Road Assessment Programme (iRAP) provides a Star Rating System to assess 30 the inherent safety of roads, assigning ratings from 1 star (highest risk) to 5 stars (lowest risk) for 31 dierent road users (iRAP Star Rating Manual, 2024). To perform road infrastructure assessment, 32 trained and accredited human coders inspect geo-referenced road survey videos and record values 33 for more than 50 physical road attributes for each 100-metre segment in accordance with the iRAP 34 Coding Manual (iRAP Coding Manual, 2025). This attribute coding underpins iRAP's Star Rating 35 models, which translate infrastructure characteristics into evidence-based risk scores. These scores 36 are used by governments and road agencies worldwide to develop Safer Roads Investment Plans and 37 prioritise cost-eective safety interventions. However, the manual coding process remains costly, 38 time-consuming, and susceptible to coder variability, thereby limiting the scalability of formal road 39 assessments, particularly in low- and middle-income countries. 40 Recent research has begun to address this bottleneck through automated computer vision ap41 proaches. Deep learning models trained on road survey imagery have demonstrated promising 42 performance in classifying road attributes (Ka£an et al., 2024; Shahid et al., 2025b). However, 43 in our preliminary studies (Shahid et al., 2025b) we found that these road safety attributes are 44 structurally diverse, ranging from easily recognisable geometric features to more complex transient, 45 low-frequency scene elements that challenge any individual deep learning model. As a result, no 46 single deep learning model reliably excels across this full diversity of classi cation tasks simulta47 neously. 48 Ensemble learning oers a promising way to overcome this limitation. By combining predictions 49 from multiple independently trained models, ensemble methods can exploit complementary error

50 patterns and reduce task-speci c failure modes (Ganaie et al., 2022). Despite the well-established 51 e cacy of ensemble approaches in multi-task classi cation settings, their systematic application to 52 iRAP road safety attribute classi cation has not been reported in the literature. 53 To address this gap, this paper makes the following contributions: 54 1. We present the rst study to systematically apply and compare ensemble learning strategies 55 for automated multi-task iRAP road safety attribute classi cation from road survey videos, 56 covering 23 attributes across a large-scale dataset of 199,233 10-meter road segments. 57 2. We implement and evaluate various ensemble voting strategies spanning training-free ap58 proaches (Majority Vote, Con dence Vote), validation-calibrated methods (Weighted Av59 erage Probability, Task-Speci c Best), and a meta-learning strategy (Stacking), evaluated 60 against individual backbone models to quantify the bene t of each aggregation approach. 61 3. All strategies are assessed using comprehensive performance metrics providing a holistic view 62 of classi cation quality.

63 2 Methodology 64 Dataset Details 65 We evaluate our approach on a multi-task dataset comprising 199,233 annotated 10-meter road 66 segments extracted from survey videos covering 2,300 km of public roads in Bosnia and Herzegovina. 67 The dataset was created by aligning coded iRAP attributes (50+ attributes per segment) from an 68 o cial coding campaign with corresponding video frames using GPS endpoint information, as 69 described in (Shahid et al., 2025b). We use a subset of T = 23 iRAP classes and split the segments 70 into 186,796 training, 6,593 validation, and 5,844 test samples while keeping all segments from the 71 same video in the same partition to avoid data leakage.

72 Model Pipeline 73 Each 10-metre road segment is represented not by a single frame but by a short temporal sequence. 74 Concretely, the input to the model is a batch of such sequences, X ∈ RB×F ×C×H×W , where F = 3 75 frames are sampled at temporal osets [−4, −1, 0], corresponding to a spatial context window 76 of [−40 m, −10 m, 0 m]. This oset con guration was determined through a systematic ablation 77 study comparing dierent frame oset combinations. To pass all frames through a single backbone 78 in one forward pass, the batch is reshaped to X ′ ∈ R(B·F )×C×H×W , treating each frame as an 79 independent image. 80 The classi cation target consists of T = 23 road attributes, each with Ct ∈ {3, . . . , 7} classes. QT 81 Because the joint prediction space t=1 Ct is intractably large, the problem is decomposed into T 82 independent classi cation tasks, each solved by a dedicated head attached to a shared backbone. 83 Two backbone architectures are evaluated for this role BEiT and ConvNextV2 (Table 1).

Table 1: Pre-Trained models used with their con gurations Model Size Resolution Params (Million) Pre-training & Fine-tuning BEiT (Bao et al., 2021) Base 384 × 384 86.7 ImageNet-22kImageNet-22k ConvNextV2 (Woo et al., 2023) Base 384 × 384 88.7 ImageNet-22kImageNet-1K

84 Both backbones share the same feature extraction strategy and are kept frozen throughout training, 85 acting as multi-scale feature extractors. For each input frame xf ∈ R3×H×W , feature maps from all 86 intermediate backbone stages are spatially collapsed viaPNglobal average pooling and concatenated 87 channel-wise into a single vector of dimension D = i=1 Ci , where Ci is the channel depth at 88 stage i. The three per-frame vectors are then averaged to produce a single temporally-fused 89 representation z ∈ RD for the segment: F 1 X z= GAP(Cat[Backbone(xf )]) (1) F f =1

90 where GAP denotes global average pooling and Cat denotes channel-wise concatenation of multi91 scale features.

92 This fused vector z is then fed independently to each of the T task heads. Each head is a three-layer 93 MLP with LayerNorm and GELU activations, producing the logits for its assigned attribute:      (3) (2) (1) ŷt = Wt GELU LN Wt GELU LN Wt z (2) (·) 94 where Wt are task-speci c weight matrices and LN is LayerNorm. 95 During training, each backbone and its attached heads are ne-tuned independently. The model is 96 evaluated on the validation set after every epoch, and training stops early when validation macro97 F1 fails to improve for three consecutive epochs; the best-performing checkpoint is retained. At 98 inference, both saved checkpoints are loaded with all weights frozen and applied independently to (t) 99 the test set. Each model m ∈ {1, 2} produces a per-task softmax probability vector pm ∈ ∆Ct for 100 every segment and every attribute t ∈ {1, . . . , T }. These M × T probability vectors are the inputs 101 to the ensemble aggregation layer, which combines them using the strategies described in Section 2 102 to produce the nal attribute predictions ŷ (t) across all T = 23 tasks. The complete two-phase 103 pipeline is illustrated in Figure 1.

104 Training Details 105 Both models are ne-tuned independently under an identical hyperparameter con guration to 106 ensure a fair comparison and controlled ensemble diversity. Each model is trained for 20 epochs 107 with early stopping based on validation macro-F1, with a patience of 3 epochs to prevent over-

tting while retaining the best-performing checkpoint. Optimisation uses AdamW with β1 = 0.9, 109 β2 = 0.999, and ε = 10−8 on a single NVIDIA GeForce RTX 4080 GPU. A decoupled weight decay 110 of λ = 5×10−3 is applied to non-bias, non-normalisation parameters; bias and batch-normalisation 111 parameters are excluded from weight decay. The learning rate is set to 2 × 10−5 for both the 112 backbone and task-head parameters. Training employs mixed-precision arithmetic (FP16) via 113 PyTorch's automatic mixed-precision (AMP) with gradient scaling to prevent under ow. Gradients 114 are accumulated over 2 steps. A drop-path rate of 0.2 and a dropout rate of 0.4 are applied for 115 regularisation.

116 Ensemble Strategies 117 In this paper we have evaluated various ensemble voting strategies. Let M =2 be the number (t) 118 of ensemble members, T the number of tasks, and pm ∈ ∆Ct the softmax probability vector 119 from model m for task t on a given sample. Majority Vote and Con dence Vote constitute the 120 hard-voting strategies, as they operate on discrete class predictions, whereas Weighted Average 121 Probability, Task-Speci c Best, and Stacking are soft-voting strategies that operate directly on 122 predicted class probabilities.

123 Training-Free Strategies 124 1. Majority Vote (MV). Majority voting is the most widely used hard-voting combination 125 rule. Each model independently predicts the most probable class label for a given sample, and 126 the ensemble selects the class that receives the plurality of votes across all M models. Formally, (t) (t) 127 each model casts a hard vote ĉm = arg maxc pm,c , and the ensemble selects the plurality class 128 (Kokkinos and Margaritis, 2014): M h i (t) X ŷMV = arg max 1 ĉ(t) m =c (3) c m=1

129 Majority voting is training-free and requires no validation data; its limitation is that it ignores 130 the con dence of individual predictions, treating all votes as equally reliable regardless of model 131 certainty (Ganaie et al., 2022). 132 2. Con dence Vote (CV). Con dence voting is a hard-voting variant that resolves disagree133 ments by deferring to the most con dent model rather than the plurality (Stanovov et al., 2020). 134 When all models agree, the shared prediction is used; when they disagree, the model with the 135 highest peak softmax probability is selected: (t) (t) ŷCV = ĉm∗ , m∗ = arg max max p(t) m,c (4) m c

136 This rule is particularly eective when base models are well-calibrated, so that high con dence 137 reliably signals correct predictions.

Phase 1: Fine-Tuning (each backbone trained independently) Frames [t−4, t−1, t] Frames [t−4, t−1, t]

Pre-trained BEiT Pre-trained ConvNextV2

Multi-scale Feature Extraction Multi-scale Feature Extraction

D D GAP → Concat → z∈R GAP → Concat → z∈R

D D Temporal Mean Pooling→ z̄∈R Temporal Mean Pooling→ z̄∈R

Task Heads Task Heads

(t) Softmax p ∈∆Ct , ∀ t Softmax p (t) ∈∆Ct , ∀ t

Best Checkpoint Savedby val. macro-F1 Best Checkpoint Savedby val. macro-F1

Phase 2: Inference & Ensemble Aggregation Test Frames [t−4, t−1, t] Test Frames [t−4, t−1, t]

Best Checkpoint Best Checkpoint

(t) (t) p1 ∈∆Ct , ∀ t p2 ∈∆Ct , ∀ t

Ensemble Aggregation (Section 2)

(t) Final Predictions ŷ ∈{1, . . . , Ct } for all T =23 attributes

Figure 1: Two-phase pipeline for multi-task iRAP road safety attribute classi cation. Both models independently produce per-task softmax distributions p(t) m , which ensemble strategies aggregate into nal predictions across all T =23 attributes.

138 Validation-Calibrated Strategies 139 Let V denote a labelled validation set dierent from both training and test sets. Backbone weights 140 are frozen; V is used only to derive scalar mixing coe cients. 141 3. Weighted Average Probability (WAP). Weighted soft voting extends the average probabil142 ity rule by assigning higher weight to models with stronger overall validation performance (Dogan 143 and Birant, 2019). Global per-model accuracy am on V serves as the mixing weight: M (t) X ŷWAP = arg max am · p(t) m,c , am = Accm (V) (5) c m=1

144 4. Task-Speci c Best (TSB). In a multi-task setting, dierent models may excel on dierent 145 subtasks due to varying task a nities and the eects of negative transfer (Standley et al., 2020). 146 TSB exploits this by selecting, for each task independently, the single best model according to

147 per-task validation accuracy: (t) (t) ŷTSB = ĉm∗ (t) , m∗ (t) = arg max Acc(t) m (V) (6) m

148 Meta-Learning Strategy 149 5. Stacking (ST). Stacked generalisation, introduced by Wolpert (1992), is a meta-learning 150 approach in which a second-level learner is trained to combine the outputs of the base models. 151 Unlike voting rules, stacking learns a meta-level combiner that adapts to base model outputs and 152 is particularly eective when base learners are diverse (Dºeroski and šenko, 2004). We trained a 153 per-task logistic regression meta-learner gt on concatenated base model probability vectors from 154 V . The meta-feature vector for task t concatenates the M softmax probability vectors, yielding a 155 meta-input of dimension M × Ct : h i (t) (t) (t) u(t) = p1 ; p2 ; . . . ; pM ∈ RM ·Ct (7)

156 The meta-learner gt is trained to minimise the cross-entropy loss on V :       (t) X ŷST = gt u(t) , gt = arg min LCE g u(t) , y (t) (8) g (u,y)∈V

157 StandardScaler normalisation is applied to u(t) before tting gt to prevent feature scale dominance.

158 Evaluation Metrics 159 All ensemble strategies described above were evaluated using a comprehensive set of metrics that 160 assess discriminative performance across multiple complementary dimensions. The de nitions and 161 equations for accuracy, precision, recall, and F1-score are provided in our previous work (Shahid 162 et al., 2025a). For completeness, we brie y restate the notation used throughout the evaluation. 163 Let N be the number of test samples, T the number of attribute classi cation tasks, Ct the number (t) (t) 164 of classes for task t, yi the ground-truth label, and ŷi the predicted class for the i-th sample on 165 task t. The remaining metrics and their mathematical formulations are de ned as follows: 166 Cohen's κ. Measures agreement between predictions and ground truth, correcting accuracy for 167 chance agreement: po − pe κ= (9) 1 − pe 168 where po is the observed accuracy and pe is the expected agreement under the hypothesis that 169 predictions and labels are independent: X npred · ntrue pe = c c (10) c N2

170 with npred c and ntrue c denoting the marginal counts of class c in predictions and ground truth, 171 respectively.

172 3 Results and Discussion 173 Table 2 reports comprehensive metrics across the six-metric evaluation suite for both individual 174 backbone models and all ensemble strategies. Figure 2 provides a complementary multi-metric 175 view of all ensemble strategies via radar charts. 176 The results show that Majority Vote (MV), which relies solely on hard label plurality, closely mir177 rors the weaker BEiT backbone (macro-F1: 0.6027 vs. 0.6023) and fails to exploit ConvNextV2's 178 superior probability estimates, con rming that hard-decision aggregation loses calibration infor179 mation critical for imbalanced multi-class tasks. Con dence Vote (CV) recovers this information 180 by weighting votes by predicted class probability, matching ConvNextV2's overall performance 181 (macro-F1: 0.7020) while providing slight recall improvements (0.7258 vs. 0.7259). 182 Among validation-calibrated strategies, Weighted Average Probability (WAP) achieves the highest 183 macro-F1 (0.7063), macro-recall (0.7283), Cohen's κ (0.6611), and MCC (0.6664) of all deploy184 able methods. Stacking (LR) attains the highest overall accuracy (0.8673) and macro-precision 185 (0.7076) but at the cost of reduced recall (0.6956) and macro-F1 (0.6906), re ecting its tendency