Key Takeaway: This paper introduces uniform stability of posteriors, a concept that holds uniformly across all data, and shows that exponential family priors produce p-stable posteriors.

Abstract

Infinitesimal sensitivities of the posterior distribution P(Â·X) and posterior quantities [rho](P) w.r.t. the choice of the prior P are considered. In a very general setting, the posterior P(Â·x) and posterior quantities [rho](P) are treated as functions of the prior P on the space of probability measures. Stability then amounts to checking if these functions satisfy Lipschitz condition of order 1. For parametric prior families, an intuitive criterion of p-stability is proposed and a general result on p-stability of posteriors is established. This result is then used to show that exponential family priors produce p-stable posteriors. In another interesting development, the notion of uniform stability of posteriors, i.e., stability w.r.t. to the prior that hold uniformly over all data, is introduced. Sufficient conditions are obtained under which posteriors are uniformly stable in the total variation metric and the weak convergence metrics.