Skip to contents

Assembles the information that the Bayesian Analysis Reporting Guidelines (BARG; Kruschke, 2021) ask authors to report, marks each item as ok, check (needs attention) or missing (not yet done), and computes the pieces that are cheap to obtain from the fit itself: convergence diagnostics, a posterior summary, posterior predictive p-values, PSIS-LOO and power-scaling prior sensitivity.

Usage

workflow_report(
  fit,
  prior_check = NULL,
  sensitivity = NULL,
  performance = NULL,
  prob = 0.95,
  seed = 1234
)

# S3 method for class 'bmb_report'
format(x, type = c("text", "markdown"), digits = 3, ...)

Arguments

fit

A stanreg object from rstanarm::stan_glm(), rstanarm::stan_glmer() or fit_model_with_prior().

prior_check

Optional result of prior_predictive_check().

sensitivity

Optional result of prior_sensitivity() or sensitivity_analysis(). If NULL, prior_sensitivity() is run.

performance

Optional result of evaluate_model_performance() on held-out data.

prob

Probability mass of the reported credible intervals.

seed

Seed for the posterior predictive simulations. R's random number stream is left unchanged.

x

A bmb_report object.

type

"text" (as printed) or "markdown".

digits

Significant digits in the posterior summary.

...

Unused.

Value

An object of class bmb_report with elements items (a data frame with one row per checklist item: item, status, detail), posterior (posterior summary table), priors, convergence, loo, ppc, sensitivity and prob.

Details

Supply the results of steps that require extra work, such as prior_predictive_check() and evaluate_model_performance(), to complete the checklist.

Posterior predictive p-values use the mean, SD, minimum and maximum of the outcome. For binary outcomes they use the mean and, with at least 20 observations, a binned calibration statistic, because any model with an intercept reproduces the mean. The calibration statistic groups observations by fitted probability and sums the squared standardised differences between observed and expected successes, a posterior predictive version of the Hosmer-Lemeshow statistic; only large values indicate misfit.

Held-out performance is marked check when the coverage of the predictive intervals differs from their nominal level (prob in evaluate_model_performance()) by more than two binomial standard errors.

Use format(report, type = "markdown") (or knit the object in R Markdown or Quarto) for a Markdown version of the checklist.

References

Kruschke, J. K. (2021). Bayesian analysis reporting guidelines. Nature Human Behaviour, 5, 1282–1291. doi:10.1038/s41562-021-01177-7

Gelman, A., Meng, X.-L., & Stern, H. (1996). Posterior predictive assessment of model fitness via realized discrepancies. Statistica Sinica, 6(4), 733–760. https://www3.stat.sinica.edu.tw/statistica/j6n4/j6n41/j6n41.htm

Examples

# \donttest{
data(kidiq, package = "rstanarm")
fit <- fit_model_with_prior(kidiq, kid_score ~ mom_iq + mom_hs,
                            chains = 4, iter = 1000, refresh = 0)
workflow_report(fit)
#> Bayesian analysis report (BARG checklist; Kruschke, 2021)
#> ============================================================
#> [ok]    1. Model
#>           kid_score ~ mom_iq + mom_hs; gaussian family, identity link; 434
#>           observations; 3 coefficients
#> [ok]    2. Priors
#>           rstanarm defaults (weakly informative, autoscaled): (Intercept) ~
#>           normal(86.8, 51); mom_iq ~ normal(0, 3.4); mom_hs ~ normal(0,
#>           124); sigma ~ exponential(rate = 0.049)
#> [ -- ]  3. Prior predictive check
#>           Not supplied: run prior_predictive_check().
#> [ok]    4. MCMC computation
#>           4 chains x 1000 iterations (500 warm-up), seed 1234. Max R-hat
#>           1.006, min bulk-ESS 2120, min tail-ESS 1080, 0 divergences.
#> [ok]    5. Posterior summary
#>           Means, medians, SDs and 95% central credible intervals for 4
#>           parameters (see $posterior).
#> [ok]    6. Posterior predictive check
#>           Posterior predictive p-values: mean 0.49, sd 0.55, min 0.82, max
#>           0.71
#> [ok]    7. Cross-validation (PSIS-LOO)
#>           elpd_loo = -1876.1 (SE 14.3), p_loo = 4.1; 0 observation(s) with
#>           Pareto k > 0.70
#> [ok]    8. Prior sensitivity
#>           Power-scaling: no parameter is sensitive to the prior.
#> [ -- ]  9. Held-out predictive performance
#>           Not supplied: run evaluate_model_performance() on test data
#>           (optional if LOO suffices).
#> [ok]    10. Software
#>           R 4.6.2; rstanarm 2.32.2; rstan 2.32.7; bmbeR 2.1.0
#> 
#> Posterior summary:
#>     variable   mean median     sd   q2.5  q97.5 p_positive
#>  (Intercept) 25.700 25.800 5.9000 13.900 37.200      1.000
#>       mom_iq  0.564  0.563 0.0603  0.449  0.683      1.000
#>       mom_hs  5.980  6.040 2.2300  1.660 10.400      0.998
#>        sigma 18.200 18.200 0.6460 16.900 19.500      1.000
#> 
#> 0 item(s) need attention, 2 not yet done.
# }