Refits a geographically weighted regression from scratch on each training
fold and scores it on the held-out fold, so the reported error is the
model's at locations it did not see. Use it for an accuracy figure for a
GWR: the in-sample \(R^2\) that model_metrics() reports on a
gwr_fit says little, because a local regression with a small bandwidth
can track the training points almost exactly. The bandwidth is selected
again in each fold unless bandwidth fixes it, so the selection itself is
inside the cross-validation, with no tuning on the full data first.
Usage
cv_gwr(
data_sf,
response_var,
predictor_vars,
folds = NULL,
k = 5,
seed = 123,
adaptive = TRUE,
bandwidth = NULL,
kernel = c("bisquare", "gaussian", "tricube", "boxcar", "exponential"),
boundary = NULL,
pointize = "auto",
block_size = NULL,
auto_range = FALSE,
parallel = FALSE,
metrics = NULL
)Arguments
- data_sf
An sf layer with the response and the predictors.
- response_var
Name of the response column.
- predictor_vars
Character vector of predictor column names.
- folds
Optional fold definitions, in one of three shapes: a
make_folds()result; a bare list oflist(train =, test =)pairs of..row_idvalues; or a vector of fold labels, one per row, which becomes leave-that-label-out splits. The label vector is how folds built by another package are used here:blockCV::cv_spatial()returns one as$folds_ids, while its$folds_listholds two unnamed vectors per fold and is refused by name. Train and test must be disjoint; a fold that trains on its own test rows is not a cross-validation split and is refused with an error. IDs naming no row of the prepared data are dropped with a logged count (expected when rows were removed for missing values; a sign the folds came from other data when they were not). WhenNULL, the folds aremake_folds(method = "block_kfold"), built withk,seed,boundary,block_sizeandauto_range. Withfolds = NULLthe score estimates the error at the distance the block grid built here puts between the held-out points and their training data, which overstates the error of a map where the samples cover the area it predicts over. The excess is how much the model's error grows from the map's distances to the data to the block grid's, and is small for a model whose error changes little with the distance from the data.fold_separation()withprediction_pointscompares the held-out distances with those of the locations the model will predict at, andmake_folds(method = "knndm", prediction_points = )builds folds matched to them.- k
Number of folds to build when
foldsisNULL; not read whenfoldsis supplied. Default 5.- seed
Seed for building the folds when
foldsisNULL, and for the one seed drawn per fold before it is fitted. With a seed, the caller's random stream is restored afterwards. Default 123.- adaptive
Logical; use an adaptive bandwidth, a number of neighbours, rather than a fixed distance. Default
TRUE.- bandwidth
Optional bandwidth, applied to every fold. For
adaptive = TRUEa whole number of neighbours; foradaptive = FALSEa distance in the units of the projected CRS the folds are fitted in (lon/lat input is projected first, so a value in degrees is read as metres).NULL(default) selects a bandwidth in each fold from that fold's training rows, by the searchfit_gwr_model()runs (its section "Bandwidth selection": AICc, or the leave-one-location-out sum of squares when observations share a location), and reports it infold_metrics$bandwidth.- kernel
The kernel that weights each location's neighbours:
"bisquare"(default),"gaussian","tricube","boxcar"or"exponential", as infit_gwr_model().- boundary
Optional polygon layer (sf or sfc). Data without a CRS are aligned to it, and the blocks built when
foldsisNULLare clipped to it (seemake_folds()and spatialkit-geometry).- pointize
How a feature that is not a point is reduced to one before fitting: a mode of
coerce_to_points(). Default"auto".- block_size
Optional minimum block edge length, in the units of the CRS the folds are built in, for the blocks built when
foldsisNULL; seemake_folds(). It is a size you choose and is compared with the autocorrelation range only when one can be estimated, in which case a smaller size is warned about. Beside suppliedfoldsit has nothing to size and raises a warning of classspatialkit_fold_args_ignored.- auto_range
Logical, default
FALSE.TRUEwithfolds = NULLestimates the autocorrelation range and uses it as the minimum block size.TRUEbeside suppliedfoldsraises the same warning asblock_size.- parallel
Accepted so that every
cv_*()function takes the same arguments, but the GWR folds always run one after another in this R process, where the warningspredict()raises for each fold are collected into one. A value asking for more than one core raises a warning saying so. DefaultFALSE.- metrics
Optional scoring function of your own, a
function(y, yhat)returning a named numeric vector; its names become columns offold_metrics(per fold) andoverall(pooled) beside the built-in ones. See the section "Your own metrics" incv_spatial()for the contract.
Value
A list with the elements cv_spatial() returns (overall,
fold_metrics, predictions, folds, n_folds_attempted,
n_folds_succeeded, fold_status, orphan_rows, n_unknown_ids and
n_dropped; see its Value for each, and for the five fold statuses),
plus:
formulaThe model formula, as text.
adaptiveWhether the bandwidth was adaptive.
fold_metrics also carries each fold's bandwidth. Adj_R2 is NA in
both overall and fold_metrics: the pooled predictions have no single
parameter count, and a GWR's effective parameter count is not its
predictor count.
A held-out location whose local regression cannot be estimated has
yhat NA, and one beyond the reach of its fold's fit is predicted and
scored; each case raises one warning for the whole run (classes
"spatialkit_gwr_not_estimable" and "spatialkit_gwr_extrapolation"),
with the totals over the folds that had any. See predict.gwr_fit().
Where a fold's fit took the response about a centre (a response at a
large offset; see fit_gwr_model()), predictions also carries
y_centre (that centre, 0 in the rows of a fold fitted on the raw
response), yhat_centred and y_train_mean_centred (yhat and
y_train_mean less the centre, which keep digits that rounding at the
response's size would lose), and the metrics are those of the centred values, as
model_metrics() computes them.
Details
Folds default to spatial blocks (make_folds(method = "block_kfold")),
not random ones. A random split leaves a held-out point's neighbours in
the training set, so its score is the error of predicting among the
samples; blocks hold out whole areas, and folds says what their score
estimates. Use cv_bayes() for the same treatment of a Bayesian GP,
cv_rf() for a random forest, and compare_models_cv() to score several
backends on one set of folds.
Percentage errors on responses with zeros
MAPE divides by the observed value and SMAPE by
\(|y| + |\hat{y}|\), so neither is defined where its denominator is zero.
Neither returns Inf or NaN. Both are averaged over the rows
whose denominator is non-zero, and are NA when no row qualifies.
Non-zero is judged at the scale of the data: a denominator no larger
than 100 machine epsilons times the largest one counts as zero, so the
rule does not depend on the units of the response.
The n_MAPE and n_SMAPE columns record how many rows that was;
the n column (n_pred in a cross-validation result) counts
finite observation/prediction pairs. Read a percentage error next to that
count: when n_MAPE is smaller, MAPE is an average over a
subset of the data, whatever its value.
Any response taking exact zeros is affected: counts, rainfall, abundance,
claim amounts. On a zero-inflated response with 62 zeros out of 120,
MAPE averages the 58 non-zero rows and n_MAPE is 58.
SMAPE drops the rows where observation and prediction are both
near zero, which on a well-fitted zero-inflated model are the rows it got
right, so it averages the harder rows only and reads worse than the fit
deserves. n_SMAPE shows how many rows it kept; it does not correct
the average.
RMSE, MAE and \(R^2\) use every finite row; prefer them
whenever the response can be zero. For a Bayesian GP fit,
cv_bayes() also reports CRPS and interval coverage, which
are proper scoring rules and are defined at zero.
See also
Other cross-validation:
compare_models_cv(),
cv_bayes(),
cv_rf(),
cv_spatial(),
plot_calibration(),
plot_cv_metrics()
Examples
library(sf)
set.seed(1)
n <- 60
dat <- st_as_sf(
data.frame(x = 5e5 + runif(n, 0, 1000), y = 5e6 + runif(n, 0, 1000),
elev = rnorm(n)),
coords = c("x", "y"), crs = 32632
)
dat$price <- 10 + 0.01 * (st_coordinates(dat)[, 1] - 5e5) +
2 * dat$elev + rnorm(n)
cv <- cv_gwr(dat, "price", "elev", k = 3, bandwidth = 30)
#> cv_gwr(): no folds supplied -- using spatial block k-fold CV (k=3).
cv$overall # pooled over the held-out rows of every fold
#> RMSE MAE MAPE SMAPE R2 Adj_R2 n_pred n_MAPE n_SMAPE
#> 1 1.970881 1.640318 11.65992 11.14015 0.620296 NA 60 60 60
cv$fold_metrics # and fold by fold
#> fold n_train n_test n_pred RMSE MAE MAPE SMAPE R2
#> 1 1 39 21 21 1.34471 1.167010 7.532286 7.606738 0.6918865
#> 2 2 41 19 19 2.31357 1.985458 14.905985 13.676105 0.5117606
#> 3 3 40 20 20 2.16090 1.809408 12.910182 12.441061 0.6691330
#> Adj_R2 n_MAPE n_SMAPE bandwidth
#> 1 NA 21 21 30
#> 2 NA 19 19 30
#> 3 NA 20 20 30