A wrapper over cv_spatial() that refits a ranger
forest on each training fold. By default it holds out whole spatial
blocks, so most neighbours of a held-out point are not in the training
set; the forest's out-of-bag error holds out single rows. The folds you
pass decide what the score measures (see "Fold methods" in
make_folds()).
Usage
cv_rf(
data_sf,
response_var,
predictor_vars,
folds = NULL,
k = 5,
seed = 123,
parallel = FALSE,
block_size = NULL,
auto_range = FALSE,
boundary = NULL,
pointize = "auto",
metrics = NULL,
...
)Arguments
- data_sf
An sf object.
- response_var
Response column name.
- predictor_vars
Predictor column names.
- folds
Optional fold definitions, in any of three shapes: a
make_folds()return value; a bare list oflist(train =, test =)pairs of..row_idvalues; or a vector of fold labels, one per row, which becomes leave-that-label-out splits. The label vector is how folds built by another package are used here, sinceblockCV::cv_spatial()'s$folds_listholds two unnamed vectors per fold and is refused by name. That function returns a label vector as$folds_ids. Train and test must be disjoint: a fold that trains on its own test rows is not a cross-validation split and is refused with an error. IDs naming no row in the prepared data are dropped with a logged count. Built viablock_kfoldwhenNULL, as incv_spatial(). Withfolds = NULLthe score estimates the error at the distance the block grid built here puts between the held-out points and their training data, which overstates the error of a map where the samples cover the area it predicts over. The excess is how much the model's error grows from the map's distances to the data to the block grid's, and is small for a model whose error changes little with the distance from the data.fold_separation()withprediction_pointscompares the held-out distances with those of the locations the model will predict at, andmake_folds(method = "knndm", prediction_points = )builds folds matched to them.- k
Number of folds when
foldsisNULL. Default 5.- seed
RNG seed. Default 123. It seeds fold construction and, through a per-fold draw, each fold's forest, so two different seeds give different results even on identical
folds. To grow every fold's forest from one fixed ranger seed instead, callcv_spatialwith your ownfit_fnwrappingfit_rf_model(seed = ).- parallel
Passed to
cv_spatial. DefaultFALSE. Under forked workers each fold's forest runs single-threaded unlessnum_threadsis passed explicitly, soparallel = 4means four threads in total, where it would otherwise mean four times the session'smc.cores.- block_size, auto_range, boundary
Passed to
cv_spatial.block_sizeandauto_range = TRUEsize the blocks built whenfoldsisNULL; beside suppliedfoldsthey have nothing to size and raise a warning of classspatialkit_fold_args_ignored.- pointize
How non-POINT geometry is reduced to a point before fitting; passed to
cv_spatial. Default"auto".- metrics
Optional scoring function of your own, passed to
cv_spatial; see Your own metrics there.- ...
Passed to
fit_rf_modelon every fold:num_trees,mtry,importance,include_coordsand so on.data_sf,response_var,predictor_varsand.already_preppedare set by this function and must not be passed here (every fold would fail with "matched by multiple actual arguments").seedis this function's own argument and never reachesfit_rf_model()through here; seeseedabove for growing every fold's forest from one fixed seed.
Value
The cv_spatial() result. MAPE and
SMAPE skip rows whose denominator is zero; n_MAPE and
n_SMAPE count the rows they used (see "Percentage errors on
responses with zeros" in model_metrics()).
Comparing the forest with and without coordinates
Whether include_coords = TRUE lowers the error depends on how far
from the training data the predictions are (see "Coordinates as
predictors" in fit_rf_model), so it is a comparison to make
on your own data, at the block size the map will be used at. Score the
two forests on the same folds: build the folds once with
make_folds and pass the result as folds to two
calls, one with include_coords = TRUE and one without. Each fold
then holds out the same rows in both, so the two overall rows
compare like with like and the two fold_metrics tables pair up
fold by fold. A difference in overall that is small beside the
spread of the per-fold differences is not evidence for either forest.
The example below runs the comparison. compare_models_cv()
takes one set of forest arguments (rf_args) per call, so it scores
only one of the two forests.
See also
fit_rf_model() for the forest's arguments, and
plot_cv_metrics() to draw the per-fold scores.
Other cross-validation:
compare_models_cv(),
cv_bayes(),
cv_gwr(),
cv_spatial(),
plot_calibration(),
plot_cv_metrics()
Examples
library(sf)
set.seed(1)
n <- 200
dat <- st_as_sf(
data.frame(x = 5e5 + runif(n, 0, 1000), y = 5e6 + runif(n, 0, 1000),
a = rnorm(n)),
coords = c("x", "y"), crs = 32632
)
# An offset keeps the response away from zero, where MAPE is undefined.
dat$z <- 10 + 2 * dat$a + rnorm(n, 0, 0.3)
cv_rf(dat, "z", "a", k = 4)$overall
#> cv_rf(): no folds supplied -- using spatial block k-fold CV (k=4).
#> RMSE MAE MAPE SMAPE R2 Adj_R2 n_pred n_MAPE n_SMAPE
#> 1 0.417048 0.3264843 3.424117 3.405887 0.9587525 NA 200 200 200
# The forest with and without the coordinates, on the same blocked
# folds. The second call logs the include_coords caution once per
# session. This response has no spatial structure, so here the
# coordinates can only cost accuracy.
folds <- make_folds(dat, k = 4, method = "block_kfold", seed = 1)
plain <- cv_rf(dat, "z", "a", folds = folds, num_trees = 200)
with_xy <- cv_rf(dat, "z", "a", folds = folds, num_trees = 200,
include_coords = TRUE)
c(plain = plain$overall$RMSE, with_xy = with_xy$overall$RMSE)
#> plain with_xy
#> 0.4208380 0.7656067
# Fold by fold: positive where the coordinates lowered the error.
plain$fold_metrics$RMSE - with_xy$fold_metrics$RMSE
#> [1] -0.5017832 -0.2412238 -0.3906285 -0.1638759