Skip to contents

Diagnoses how strongly the posterior depends on the prior, and whether prior and data conflict, without refitting the model. The prior (or the likelihood) is "power-scaled", i.e. raised to a power alpha, and the resulting change in the posterior is measured (Kallioinen et al., 2023).

Usage

prior_sensitivity(
  fit,
  alpha = c(0.8, 1.25),
  threshold = 0.05,
  z_threshold = 2,
  pars = NULL,
  scale_priors = NULL
)

Arguments

fit

A stanreg object from rstanarm::stan_glm(), rstanarm::stan_glmer() or fit_model_with_prior().

alpha

Power-scaling factors for the finite perturbations. Values below 1 weaken, above 1 strengthen the prior (or likelihood). The default, c(0.8, 1.25), is a symmetric change on the log scale.

threshold

Influence threshold used for the diagnosis. The default 0.05 corresponds to a prior (or likelihood) contributing about 10% of the posterior precision in a normal model.

z_threshold

Absolute conflict_z at or above which an influential prior is diagnosed as conflicting with the data.

pars

Parameters to report (default: all).

scale_priors

Parameters whose priors are power-scaled (default: all). Use this to isolate one prior, e.g. scale_priors = "sigma".

Value

An object of class bmb_prior_sensitivity with elements summary (one row per parameter), perturbation (one row per component, alpha and parameter, with Pareto \(\hat{k}\)), priors (the priors used, after autoscaling), and settings. Has print() and plot() methods.

Details

Local sensitivity. The derivative of a posterior expectation with respect to log(alpha) at alpha = 1 equals a posterior covariance (Giordano et al., 2018). For each parameter \(\theta\) with posterior mean \(\mu\) and standard deviation \(\sigma\), bmbeR reports

  • prior_mean_sens = \(Cov(\theta, \log p(\theta)) / \sigma\): how many posterior SDs the mean moves per unit change in log(alpha);

  • prior_sd_sens = \(Cov((\theta-\mu)^2, \log p(\theta)) / (2\sigma^2)\): the relative change in the posterior SD;

  • lik_mean_sens, lik_sd_sens: the same with the log-likelihood.

In a conjugate normal model with prior \(N(m_0, s_0^2)\) and likelihood \(N(\bar y, s^2)\), -2 * prior_sd_sens is the share of posterior precision contributed by the prior, -2 * lik_sd_sens the share contributed by the data, and conflict_z = prior_mean_sens / (2 * sqrt(prior_sd_sens * lik_sd_sens)) equals \((m_0 - \bar y)/\sqrt{s_0^2 + s^2}\), the classic prior predictive check of prior-data conflict (Box, 1980). Outside that model these are approximations. See the "Prior sensitivity" article on the package website for the derivations.

Diagnosis (per parameter). The prior's influence is max(|prior_mean_sens|, |prior_sd_sens|): a prior matters if it moves the posterior mean or adds precision. The likelihood's informativeness is |lik_sd_sens|. (lik_mean_sens is reported but not used: for a symmetric posterior, tempering prior and likelihood together leaves the mean unchanged, so lik_mean_sens = -prior_mean_sens.)

  • "-": the prior's influence is below threshold; the likelihood dominates.

  • "weak likelihood": the prior is influential and the likelihood is not informative; the data say little about this parameter.

  • "prior-data conflict": prior influential, likelihood informative, and |conflict_z| >= z_threshold; prior and data disagree.

  • "informative prior": prior influential and likelihood informative, but they agree; fine if the prior is justified.

This refines the diagnosis of Kallioinen et al. (2023), which flags potential conflict whenever both prior and likelihood sensitivity are high, by separating informative-but-compatible priors from conflicting ones.

Finite perturbations. For each value in alpha, the posterior under the power-scaled prior (and likelihood) is approximated by Pareto-smoothed importance sampling (Vehtari et al., 2024). The Pareto \(\hat{k}\) diagnostic indicates whether the approximation is reliable. Weakening (alpha < 1) is the harder direction: for an approximately normal posterior the importance weights have a Pareto tail with \(k \approx 1 - \alpha\) or heavier, so values much below 0.8 are often unreliable. Use sensitivity_analysis() to assess large perturbations by refitting.

Posterior contraction is \(1 - \sigma^2_{post}/\sigma^2_{prior}\) (Schad et al., 2021): values near 0 mean the data barely updated the prior. It is NA when the prior variance is infinite (flat, Cauchy, Student-t with df <= 2).

The intercept is analysed at the predictor means, which is the parameter rstanarm places its prior on. Supported: models from rstanarm::stan_glm() and rstanarm::stan_glmer() with normal, Student-t, Cauchy, Laplace, flat or exponential priors, fitted without QR = TRUE.

Multilevel models. For rstanarm::stan_glmer() fits, the priors on the intercept, the regression coefficients and the auxiliary parameter are power-scaled and reported. The group-level covariance prior (decov) is not power-scaled, so the result describes sensitivity to the population-level priors with the group-level prior held fixed (selective power-scaling; Kallioinen et al., 2023).

References

Kallioinen, N., Paananen, T., Bürkner, P.-C., & Vehtari, A. (2023). Detecting and diagnosing prior and likelihood sensitivity with power-scaling. Statistics and Computing, 34, 57. doi:10.1007/s11222-023-10366-5

Giordano, R., Broderick, T., & Jordan, M. I. (2018). Covariances, robustness, and variational Bayes. Journal of Machine Learning Research, 19(51), 1–49. https://jmlr.org/papers/v19/17-670.html

Vehtari, A., Simpson, D., Gelman, A., Yao, Y., & Gabry, J. (2024). Pareto smoothed importance sampling. Journal of Machine Learning Research, 25(72), 1–58. https://jmlr.org/papers/v25/19-556.html

Box, G. E. P. (1980). Sampling and Bayes' inference in scientific modelling and robustness. JRSS A, 143(4), 383–430. doi:10.2307/2982063

Schad, D. J., Betancourt, M., & Vasishth, S. (2021). Toward a principled Bayesian workflow in cognitive science. Psychological Methods, 26(1), 103–126. doi:10.1037/met0000275

Examples

# \donttest{
data(kidiq, package = "rstanarm")
fit <- rstanarm::stan_glm(kid_score ~ mom_iq, data = kidiq,
                          prior = rstanarm::normal(2, 0.05),
                          chains = 2, iter = 1000, refresh = 0)
ps <- prior_sensitivity(fit)
ps
#> <bmb_prior_sensitivity> power-scaling diagnostics (1000 draws, threshold 0.05)
#>     variable   mean    sd contraction prior_mean_sens prior_sd_sens
#>  (Intercept) 86.842 1.089       1.000          -0.175        -0.123
#>       mom_iq  1.568 0.047       0.115           8.085        -0.168
#>        sigma 23.238 0.878       0.998           3.712        -0.075
#>  lik_mean_sens lik_sd_sens conflict_z           diagnosis
#>          0.059      -0.297     -0.459   informative prior
#>         -8.086      -0.206     21.723 prior-data conflict
#>         -3.757      -0.378     11.046 prior-data conflict
#> 
#> Flagged parameters:
#>   (Intercept)          informative prior
#>   mom_iq               prior-data conflict
#>   sigma                prior-data conflict
#> 
#> Finite perturbations: posterior mean shift, in posterior SDs, when the prior or
#> likelihood is raised to the power alpha (* = unreliable importance sampling;
#> refit with sensitivity_analysis() instead):
#>             prior a=0.8 prior a=1.25 lik a=0.8 lik a=1.25
#> (Intercept) -0.04       +0.03        +0.02     -0.04     
#> mom_iq      -1.50       +1.56        +1.47     -1.59     
#> sigma       -0.65       +0.75        +0.68     -0.69     
#> Intercept diagnostics refer to the intercept at the predictor means.
plot(ps)

# }