Assessing the Sensitivity of Scenario Selection to Different Data Sources in the Validation Phases of Connected and Automated Vehicles

Abstract

This paper investigates a functional scenario sampling methodology for the validation of Connected and Automated Vehicle (CAV) systems within a scenario-based testing framework. The approach relies on data-driven labelling of critical scenarios and compares two input sources: simulated scenarios generated by a traffic simulator and predicted scenarios derived from machine learning models trained on field data. A four-phase evaluation framework is proposed, including scenario emulation, verification against field data, scenario reduction, and impact assessment on an adaptive cruise control (ACC) use case. The comparison highlights the sensitivity of the scenario selection process to input data, revealing divergences in extrapolated scenarios and potential biases. Results show that both approaches are consistent in observed conditions but differ beyond available data, leading to variations in selected scenarios. This work highlights the importance of input data quality for robust and reliable CAV validation.