NextGen Assisted Utility Specification: Multitask Reinforcement Learning for Transfer Across Datasets

Abstract

Discrete choice model specification is a time-consuming task for modellers, who often specify and estimate multiple models while balancing fit, parsimony, and behavioural plausibility. We present Delphos, an Reinforcement learning agent that learns transferable specification strategies across transport choice datasets. Delphos frames specification as sequential decision-making problem in which it applies a sequence of modelling actions and receives rewards from an estimation environment based on goodness-of-fit and convergence. To transfer modelling decisions across datasets with different variable sets, we use a DeepSet-Q architecture that encodes specifications as sets of modelling terms and conditions decisions on a dataset context vector. This enables a shared policy to be trained in a multitask setting and to generalise its recommendations to unseen but related datasets. Delphos is trained on three mode choice datasets and increasingly proposes specifications that outperform the baselines over training. When then applied to an unseen dataset, the trained agent specifies competitive and behaviourally plausible models in less than 10 minutes on a CPU.