Transformer-based Generative Modelling on Activity Sequences with Time-of-Day Choices

Abstract

In this study, a generative Multi-Dimensional Transformers (MDT) framework is introduced for activity-based travel demand modelling to enable the joint modelling of daily activity sequences and Time-of-Day (TOD) choices. The limitations of traditional econometric models in capturing high-dimensional correlations and sequential dependencies are addressed by this approach. Built on the Transformer architecture, multi-head attention and masking operations are leveraged to capture nonlinear relationships while customized embedding strategies and activity-temporal sequential matrices are designed to support variable-length sequences. Then the 2019 Household Travel Survey of Shanghai is utilized for the empirical application where real-world activity sequence patterns and TOD distributions are successfully estimated and simulated. Furthermore, the heterogeneous effects of exogenous variables such as gender and household structure on activitytemporal decisions are revealed. Overall, the MDT framework provide a scalable generative tool for simulating complex activity-travel behaviours and supporting transport policy evaluation.