Runs model_metrics() on each fit in a list and binds the
rows, with a model column and a metric_basis column saying
whether each row is in-sample, out-of-bag or on newdata. It takes
a single spatial_fit too. No model is refitted: the fitted values
are read with fitted(), and newdata is scored with
predict(). compare_models() adds information criteria
and residual Moran's I to the same table.
Arguments
- fits
A
spatial_fitobject, or a named list of them (e.g.list(GWR = gwr_obj, Bayesian = bayes_obj)). The names are used as the model labels and every element must have one; an unnamed list is an error, and so are duplicated names.modelis the key the comparison table is assembled on, so two fits sharing a name cannot be told apart in the output.- newdata
Optional sf object for out-of-sample evaluation. Must contain the response variable and all predictors. If NULL, in-sample metrics are computed.
- ...
Passed to
predict()whennewdatais given. Withoutnewdatanothing reads them, so any argument here is an error, which catches a misspeltnewdata.
Value
A data.frame with one row per model and columns for
model name, all regression metrics, and metric_basis: what the
row's metrics were computed on, "in-sample" (fitted values),
"out-of-bag" (an rf_fit's fitted values, see "What the
metrics are computed on") or "newdata". Rows with different
bases do not compare like for like. An element that is not a
spatial_fit is skipped, with a logged warning, and has no row;
a list in which no element is a spatial_fit is an error.
Details
The metrics are those of model_metrics(): see its sections
"What the metrics are computed on" and "Percentage errors on responses
with zeros".
See also
Other in-sample checks:
compare_models(),
model_metrics(),
print.morans_i(),
residual_morans_i()
Examples
library(sf)
set.seed(1)
pts <- st_as_sf(
data.frame(x = 5e5 + runif(60, 0, 1000), y = 5e6 + runif(60, 0, 1000),
a = rnorm(60)),
coords = c("x", "y"), crs = 32632
)
# An offset keeps the response away from zero, where MAPE is undefined.
pts$z <- 10 + 2 * pts$a + rnorm(60, 0, 0.3)
# Fit on 40 rows and keep 20 back, so the second call is out of sample.
fit <- fit_rf_model(pts[1:40, ], "z", "a", num_trees = 50, seed = 1)
evaluate_insample(fit) # out-of-bag for a forest
#> model n RMSE MAE MAPE SMAPE R2 Adj_R2 n_MAPE
#> 1 rf_fit 40 0.409009 0.3236812 3.425689 3.390414 0.9541711 NA 40
#> n_SMAPE metric_basis
#> 1 40 out-of-bag
evaluate_insample(fit, newdata = pts[41:60, ]) # on the 20 held-out rows
#> model n RMSE MAE MAPE SMAPE R2 Adj_R2 n_MAPE
#> 1 rf_fit 20 0.5080838 0.3836504 3.56621 3.649174 0.9022065 NA 20
#> n_SMAPE metric_basis
#> 1 20 newdata