Gradient-Assisted Model Specification

Abstract

This paper proposes a gradient-assisted approach to model specification in discrete choice analysis. The key idea is that, although the sample-level gradient of the log-likelihood is zero at the maximum likelihood optimum, respondent-level gradients are not, and therefore contain valuable information about underlying heterogeneity. Using a well known Stated Preference (SP) dataset, we show that these gradients reveal systematic variation in preferences and can be exploited in several ways. First, they help identify omitted systematic heterogeneity and guide improved multinomial logit specifications. Second, in mixed logit models, gradients are strongly related to posterior means and can be introduced directly into random coefficient distributions, leading to major improvements in fit and a substantial reduction in residual heterogeneity. Third, gradients provide a natural basis for clustering respondents and improving class allocation in latent class models. Across all model structures, the use of gradients yields better fit, more behaviourally meaningful heterogeneity patterns, and lower willingness-to-pay measures. The paper also highlights endogeneity concerns as an important next step for future research.