Assembles the information that the Bayesian Analysis Reporting
Guidelines (BARG; Kruschke, 2021) ask authors to report, marks each item
as ok, check (needs attention) or missing (not yet done), and
computes the pieces that are cheap to obtain from the fit itself:
convergence diagnostics, a posterior summary, posterior predictive
p-values, PSIS-LOO and power-scaling prior sensitivity.
Arguments
- fit
A
stanregobject fromrstanarm::stan_glm(),rstanarm::stan_glmer()orfit_model_with_prior().- prior_check
Optional result of
prior_predictive_check().- sensitivity
Optional result of
prior_sensitivity()orsensitivity_analysis(). IfNULL,prior_sensitivity()is run.- performance
Optional result of
evaluate_model_performance()on held-out data.- prob
Probability mass of the reported credible intervals.
- seed
Seed for the posterior predictive simulations. R's random number stream is left unchanged.
- x
A
bmb_reportobject.- type
"text"(as printed) or"markdown".- digits
Significant digits in the posterior summary.
- ...
Unused.
Value
An object of class bmb_report with elements items (a data
frame with one row per checklist item: item, status, detail),
posterior (posterior summary table), priors, convergence, loo,
ppc, sensitivity and prob.
Details
Supply the results of steps that require extra work, such as
prior_predictive_check() and evaluate_model_performance(), to
complete the checklist.
Posterior predictive p-values use the mean, SD, minimum and maximum of the outcome. For binary outcomes they use the mean and, with at least 20 observations, a binned calibration statistic, because any model with an intercept reproduces the mean. The calibration statistic groups observations by fitted probability and sums the squared standardised differences between observed and expected successes, a posterior predictive version of the Hosmer-Lemeshow statistic; only large values indicate misfit.
Held-out performance is marked check when the coverage of the predictive
intervals differs from their nominal level (prob in
evaluate_model_performance()) by more than two binomial standard errors.
Use format(report, type = "markdown") (or knit the object in R Markdown
or Quarto) for a Markdown version of the checklist.
References
Kruschke, J. K. (2021). Bayesian analysis reporting guidelines. Nature Human Behaviour, 5, 1282–1291. doi:10.1038/s41562-021-01177-7
Gelman, A., Meng, X.-L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733–760. https://www3.stat.sinica.edu.tw/statistica/j6n4/j6n41/j6n41.htm
Examples
# \donttest{
data(kidiq, package = "rstanarm")
fit <- fit_model_with_prior(kidiq, kid_score ~ mom_iq + mom_hs,
chains = 4, iter = 1000, refresh = 0)
workflow_report(fit)
#> Bayesian analysis report (BARG checklist; Kruschke, 2021)
#> ============================================================
#> [ok] 1. Model
#> kid_score ~ mom_iq + mom_hs; gaussian family, identity link; 434
#> observations; 3 coefficients
#> [ok] 2. Priors
#> rstanarm defaults (weakly informative, autoscaled): (Intercept) ~
#> normal(86.8, 51); mom_iq ~ normal(0, 3.4); mom_hs ~ normal(0,
#> 124); sigma ~ exponential(rate = 0.049)
#> [ -- ] 3. Prior predictive check
#> Not supplied: run prior_predictive_check().
#> [ok] 4. MCMC computation
#> 4 chains x 1000 iterations (500 warm-up), seed 1234. Max R-hat
#> 1.006, min bulk-ESS 2120, min tail-ESS 1080, 0 divergences.
#> [ok] 5. Posterior summary
#> Means, medians, SDs and 95% central credible intervals for 4
#> parameters (see $posterior).
#> [ok] 6. Posterior predictive check
#> Posterior predictive p-values: mean 0.49, sd 0.55, min 0.82, max
#> 0.71
#> [ok] 7. Cross-validation (PSIS-LOO)
#> elpd_loo = -1876.1 (SE 14.3), p_loo = 4.1; 0 observation(s) with
#> Pareto k > 0.70
#> [ok] 8. Prior sensitivity
#> Power-scaling: no parameter is sensitive to the prior.
#> [ -- ] 9. Held-out predictive performance
#> Not supplied: run evaluate_model_performance() on test data
#> (optional if LOO suffices).
#> [ok] 10. Software
#> R 4.6.2; rstanarm 2.32.2; rstan 2.32.7; bmbeR 2.1.0
#>
#> Posterior summary:
#> variable mean median sd q2.5 q97.5 p_positive
#> (Intercept) 25.700 25.800 5.9000 13.900 37.200 1.000
#> mom_iq 0.564 0.563 0.0603 0.449 0.683 1.000
#> mom_hs 5.980 6.040 2.2300 1.660 10.400 0.998
#> sigma 18.200 18.200 0.6460 16.900 19.500 1.000
#>
#> 0 item(s) need attention, 2 not yet done.
# }