Reduces the layer to 2-D points in a projected CRS and drops the rows a
model cannot use: rows with a missing or non-finite value in the response
or a predictor, and rows without a location. Every fitter and every
cv_*() function prepares its data here.
Usage
prep_model_data(
data_sf,
response_var,
predictor_vars,
boundary = NULL,
pointize = c("auto", "surface", "point_on_surface", "centroid", "line_midpoint",
"bbox_center"),
require_response = TRUE
)Arguments
- data_sf
An sf object. Rows without a location are dropped (see Details).
- response_var
Name of the response column.
- predictor_vars
Predictor column names. May be
character(0)for an intercept-only model (fit_bayesian_spatial_model()supports one; the GWR and random-forest backends do not and reject it themselves).- boundary
Optional sf/sfc whose CRS the data are projected to. It is not a clip: rows that fall outside it are kept. Subset
data_sfyourself (for example withsf::st_filter()) to restrict the rows. One that mixes XY and XYZ (or XYM) geometries is read in x and y.- pointize
How non-point geometries become points, passed to
coerce_to_points()as itsmode:"auto"(default),"centroid","point_on_surface","surface"(an alias for"point_on_surface"),"line_midpoint"or"bbox_center". Its errors, such as the one of classspatialkit_features_degeneratefor a polygon s2 cannot take a centroid of, are raised in this function's name and give the rows ofdata_sf.- require_response
Logical; if
FALSEthe response column is not required to be present (useful for out-of-sample prediction where the response is unknown). DefaultTRUE.
Value
An sf object with 2-D (XY) POINT geometry, without the rows that
were dropped. The attribute "dropped" records what was removed
and two marks on the rows that were kept, in a list with these fields:
nRows dropped. Every fit stores it as
$info$n_dropped.n_geometryHow many of them were dropped for having no location.
whichTheir positions in
data_sf.row_idTheir
..row_idvalues when the layer carries that column, elseNULL.reasonOne per dropped row:
"geometry","missing"or"non_finite", in that order of precedence when several apply.by_columnFor each modelling column, and for
"(geometry)", how many of the dropped rows it is missing or non-finite in; a row can count under several.response_std_diffThe mean response of the dropped rows minus that of the kept rows, in standard deviations of the response over both;
NAwhen the response is not numeric, is itself missing in every dropped row, or takes one value up to the rounding of its values.n_rowsThe number of rows returned, which the record was made for.
constant_predictorsThe predictors that take one value in every returned row (
character(0)when none does).n_duplicate_locationsThe returned rows whose coordinates repeat those of an earlier row.
Details
Non-POINT geometries (MULTIPOINT included) are reduced to representative
points by coerce_to_points(), so that coordinates line up
one row per observation. Any Z or M coordinate (POINT Z from a GPS, a
GeoPackage or KML) is dropped, because every backend works in 2-D
distance.
A row has no location when its geometry is empty, a coordinate is not finite or lies beyond 1e150, or, in a geographic CRS, a vertex lies at a latitude beyond 90 degrees (see spatialkit-geometry, "Rows without a location"). The 1e150 bound is read on the coordinates as given and again after the projection, because a move between CRSs can bring a coordinate beyond it back finite, and that point is not a place the caller gave.
The CRS is projected whenever one can be established
(ensure_projected()). A layer without a CRS is read by the
lon/lat test of spatialkit-geometry ("Layers without a CRS"):
coordinates the test takes for degrees are read as EPSG:4326 and
projected, with a warning, and others are passed through as they are.
A small planar survey inside the lon/lat envelope is read as degrees, so
set the CRS on data_sf if the data are planar.
The response may not appear in predictor_vars. Using it as its own
predictor is leakage no backend catches (an out-of-bag \(R^2\) near 1 in
the random forest, a silently reduced design matrix in GWR, duplicated
rows and a phantom <none> entry in the GWR selection table), so it
is refused here.
Column names must be syntactically valid R names
(make.names(x) == x). Every backend builds a model formula from
these names, and a name R parses as an expression would fit another model
than the one requested while the fit still recorded the name you asked
for: "B5-B4" is B5 - B4, and "log(a)" is a function
call. Rename the column (for example with make.names()) before
fitting.
The "dropped" record describes the rows this call returned and
does not survive subsetting: clean[i, ], like
dplyr::filter(), slice() or arrange() of it, is a
plain layer with no "dropped" attribute, and a fit given such a
subset with .already_prepped = TRUE reports n_dropped = 0
even when the parent layer dropped rows. sf::st_drop_geometry()
keeps the record, since the rows are the same; see
[.spatialkit_rows for what binding such data frames does.
Warnings
When preparing data for a fit (require_response = TRUE) three
conditions are raised as R warnings, each with a class so it can be caught
by name with tryCatch() or withCallingHandlers():
spatialkit_rows_droppedRows were dropped. The message gives the number and share of rows, the count by reason and by column, and, for a numeric response, how far the mean response of the dropped rows sits from that of the kept rows. A fit on the rows that remain describes those rows only; when the difference is large the rows are not missing at random and the fit is biased towards the complete ones.
spatialkit_response_impreciseThe response lies so far from zero that doubles there are spaced more than a millionth of its standard deviation apart (a standard deviation of 1 at about 1e10, \(2^{33}\), or beyond). Its values are rounded to that spacing, and so is every mean and prediction computed at that size: the fit, its scores and any choice made on them read that rounding. The rows are kept; subtract a constant from the response first.
spatialkit_constant_predictorA predictor takes a single value in every row that remains. Its coefficient cannot be told from the intercept: a GWR window is rank-deficient, a Bayesian slope is its prior, and a forest ignores it. The predictor is kept; drop it from
predictor_vars.
The fitters and the cv_*() functions pass these warnings on.
compare_models_cv(),
cv_block_size_sweep() and
select_features_forward() run several cross-validations
of one layer, each of which prepares it, and raise
spatialkit_rows_dropped and spatialkit_constant_predictor
once per distinct text for the run and
spatialkit_response_imprecise once per response column;
cv_spatial() raises the last once as well when its
fit_fn prepares each fold's rows again.
With require_response = FALSE (how every predict() method
prepares newdata) none is raised: the dropped rows are written
to the log and to the record, and the prediction for each comes back
NA, so nothing is lost without a trace. Repeated locations are
legitimate (repeat visits to one site) and are counted in the record and
the log, never warned about.
See also
Other data preparation:
[.spatialkit_rows(),
coerce_to_points(),
ensure_projected(),
harmonize_crs(),
spatialkit-geometry
Examples
library(sf)
dat <- st_as_sf(
data.frame(x = 1:5, y = 5:1,
resp = c(1, 2, NA, 4, 5),
pred = c(1, 2, 3, 4, Inf)),
coords = c("x", "y"), crs = 32632
)
# Drops rows 3 (NA) and 5 (Inf), and warns that it did, with the counts.
clean <- suppressWarnings(prep_model_data(dat, "resp", "pred"))
attr(clean, "dropped")
#> $n
#> [1] 2
#>
#> $n_geometry
#> [1] 0
#>
#> $which
#> [1] 3 5
#>
#> $row_id
#> NULL
#>
#> $reason
#> [1] "missing" "non_finite"
#>
#> $by_column
#> resp pred
#> 1 1
#>
#> $response_std_diff
#> [1] 1.460593
#>
#> $constant_predictors
#> character(0)
#>
#> $n_duplicate_locations
#> [1] 0
#>
#> $n_rows
#> [1] 3
#>