Perturbed utility Markovian choice model: choice probability generation function and estimation

Abstract

This paper proposes the perturbed utility Markovian choice model (PUMCM), where sequential decisions of individuals are modeled as a Markov decision process that maximizes a perturbed utility at each state. A class of choice probability generation functions is characterized, whose gradient directly yields the optimal policy. An efficient single-level estimation approach is then developed by leveraging the invertibility of the gradient mapping of the choice probability generation function. Notably, the proposed estimation method eliminates the need for the computationally intensive bilevel estimation that is commonly used in existing Markovian choice models. Further, our approach is robust in the sense that it allows both positive and negative parameters, which is demonstrated through numerical experiments. To the best of our knowledge, both PUMCM and its estimation are novel and complement to their static counterpart of perturbed utility-based choice models.