Bridging Models and Modalities: Fusing Models Created from Diverse Multimodal Datasets

Abstract

Predictive generalization is a central challenge in transport choice modeling, especially when models are applied to new settings. One important source of this challenge is out-of-distribution (OOD) prediction, which can arise when data for the same decision problem are collected from different experimental settings. Such settings often produce heterogeneous datasets, including modalityrich and modality-lean data, that provide complementary but incomplete views of behavior. To leverage information from these datasets, we propose a general framework that extends model fusion methods to combine predictive distributions from models trained on heterogeneous datasets. Using two stated choice datasets, we find that PSIS-LOO stacking performs best overall. The results further show that the benefit of model fusion depends on the target domain, and that stacking improves performance mainly by correcting severe prediction errors in OOD cases.