Gives each location of newdata the value of the cell it lies in, read
from a summarize_by_cell() result: the pooled value when the summary
holds one, the plain mean otherwise, with the interval for one
observation when the summary holds it. Beside each value it returns the
location's distance from the nearest observation, since a cell's value
describes the part of the cell where its observations lie.
Arguments
- newdata
The locations: an sf layer. A feature that is not a point is reduced to one with
coerce_to_points(, "auto"); a collection holding a polygon s2 cannot take a centroid of in longitude and latitude (edges that cross), or whose centroid GEOS puts outside it, stops the call with the error of classspatialkit_features_degenerate, raised in this function's name and naming its rows ofnewdata. No column of it is read. With no CRS whilepointshas one, the call stops. A layer whose geometries mix dimensions or carry M (XYM or XYZM) is read in x and y (step 1).- values
The
summarize_by_cell()result forpoints. It must hold the ID column and a cell value ofresponse_var: the pooled value (estimate = "pooled") or the mean (the defaultagg_funs). Withseeds = NULLit must also carry the cell polygons, which it does when it was made withcells_sf. Avalueswith no row, or whose IDs are allNA, stops the call, with or withoutseeds. Its polygons are read in x and y when they mix dimensions or carry M (XYM or XYZM).- points
The assigned layer
valueswas computed from: an sf layer with the ID column andresponse_var, and at least one scored row.- response_var
Name of the summarised column wanted: the response, or a predictor.
- id_col
Name of the ID column, found as
summarize_by_cell()finds it: whenpointshas no column of that name, the first of"poly_id","polygon_id"and"cell_id"it has. It cannot have the name of another column of the result (value,estimate,obs_lo,obs_hi,filled,filled_from,dist_nearest,dist_ratio).- seeds
NULL, or the seeds the cells were built on: an sf POINT layer with a column named as the ID column that gives each seed's cell. For cells fromtess <- build_tessellation(seeds)that isseeds$cell_id <- tess$indexwhen the ID column is"cell_id". For seeds a seeding function returned and the seeds of aselect_resolution()result,indexgives each seed the cell it generates. Other points are indexed by the cell that holds them, and one outside the boundary isNA, or the ID of the nearest cell, which can be the cell of another seed, when it lies within a thousandth of the median cell width of it; with the cell polygons invaluesa seed that isNAis given its cell when step 3 can tell which it is (it is not when the points inside several cells no seed names find it as their nearest seed), and without them the locations nearest it have none. Passed throughget_voronoi_seeds(method = "provided", seeds = )before the tessellation, points placed by hand are seeds (seecreate_voronoi_polygons()). Two seeds at different locations with one ID are an error; seeds that share a location share its cell, asindexgives them.NAmarks a seed without a cell. When no seed with a location has the ID of a cell ofvalues(an ID column from another tessellation, IDs that are allNA, or the IDs of cells on seeds with an empty point or a coordinate that is not finite or lies beyond 1e150, as given or once placed in the working CRS, such as metres given EPSG:4326), counting the cells step 3 gives seeds outside the polygons, the call stops, since no location could take its own cell's value; one seed with such an ID and a location is enough, and a seed whose IDvalueshas no row for stands for a cell without a value. Without a CRS besidepointswith one, the seeds are read as step 1 says.
Value
A data frame with one row per row of newdata, in its order:
- the ID column
The cell the location lies in, under the name of the ID column;
NAfor a location in no cell.valueThe cell's value, or for a filled row the value of the cell in
filled_from.estimate"pooled"or"mean": which value of the summaryvalueis.obs_lo,obs_hiThe interval for one observation, when
valuesholds it.filledTRUEwhen the location has no cell, or its cell neither a scored row nor a value, and the value is that of the cell of the nearest scored row.filled_fromFor a filled row, the ID of the cell the value came from;
NAotherwise.dist_nearestDistance to the nearest location of a scored row, in the units of the working CRS.
dist_ratiodist_nearestover the spacing;NAwhen the scored rows sit on one location.
A row of newdata with an empty geometry, or with a coordinate that is
not finite or lies beyond 1e150 as given or in the working CRS, has NA
in every column. The two ID
columns have the class of the ID column of values; when a cell that
values has no row for has an ID that class cannot hold (a string
beside numbers), both are character.
What produced the frame is in attr(, "values_at"), a list of class
"spatialkit_values_at" with a print method: assignment ("seeds"
or "polygons"), crs (the working CRS), column (response_var),
estimate, obs_level (the level of the interval columns; NULL
without them, and NA when values holds them and no longer says at
what level), n (the scored rows), locations (their distinct
locations), spacing (what dist_ratio divides by; NA with one
location),
filled (how many rows of newdata were filled) and count_rule (the
summary's record of how the cell count was set, or NULL). It describes
the call: a subset of the frame's rows keeps it as it is.
The function raises no classed warning of its own; the warning that
says how seeds without a CRS were read (step 1) and the one that
counts the seeds outside the polygons of values left without a cell
(step 3) are plain ones. It passes on the conditions that
coerce_to_points(), a CRS transformation and
assign_features_to_polygons() raise, with two exceptions. The error of class spatialkit_features_degenerate with which
coerce_to_points() refuses a polygon s2 cannot take a centroid of, or
a centroid GEOS puts outside its feature, is raised in this function's
name, with the same class and fields, naming newdata or points and
its rows. The warning
assign_features_to_polygons() raises when no location at all falls in
a polygon is neither raised nor logged, since every such row has
filled = TRUE, which says the same row by row.
Details
The function works in six steps.
The working CRS. The CRS the summary measured in: the one in
attr(values, "count_rule")when that record names a projected CRS, else the one inattr(values, "pooled"), else the CRS ofpointswhen it is projected and the oneensure_projected()picks for it when it is not.points,newdataandseedsare transformed to it.newdata,seedsandpointsare read in x and y: a layer whose points, or the points taken from its other features, do not all have the same dimensions (some with a Z or M coordinate) is read without its Z and M, since sf reads a layer in the dimension of its first point. So is a layer of points, polygons or lines innewdataorpointswhose geometries carry M (XYM or XYZM), which GEOS does not read. So are the polygons ofvalueswhen they do not all have the same dimensions. Polygons ofvalueswhose geometries carry M are read in x and y as well. When neitherpointsnornewdatahas a CRS the coordinates are used as they are, unless those ofpointslook like longitude and latitude: both are then read as EPSG:4326, assummarize_by_cell()readspoints, withensure_projected()'s warning;newdataorseedswith an x outside -180 to 360 or a y beyond 90 north or south cannot be in degrees, and stop the call.seedswithout a CRS besidepointswith one are read asbuild_tessellation()reads a boundary without a CRS, with a warning: beside a projected CRS, coordinates that look like longitude and latitude are taken as EPSG:4326 and transformed, and others are taken to be in the CRS ofpoints; beside a geographic CRS, coordinates within the ranges of longitude and latitude are taken to be in it, and others stop the call.newdatawithout a CRS besidepointswith one stops the call.The scored rows. The rows of
pointswith a finiteresponse_var, a cell ID and a location: a point whose coordinates are finite and within 1e150, as given and in the working CRS. Their distinct locations are the observations every distance below is taken from.The cell of a location. With
seeds, the cell of the nearest seed. That is the Voronoi cell before its clip to the cover, so every location has one, unless its nearest seed has anNAID: a location nearest such a seed has no cell.build_tessellation()gives each seed of a layer it takes for seeds (seecreate_voronoi_polygons()) the cell it generates, a seed outside a boundary that is not convex (a k-means centre over a bay) included, andNAonly to a seed whose cell the clip removed whole. Other points, such as points placed by hand, are indexed by the cell that holds them, so one outside a boundary that is not convex isNAalthough its cell is built. One that lies in no cell but within a thousandth of the median cell width of a cell takes the ID of the nearest cell instead, which beside a boundary that is not convex can be the cell of another seed; two seeds with one ID stop the call (seeseeds). Whenvaluescarries the cell polygons, a seed that isNAis given its cell here: a seed with anNAID that lies in no polygon ofvaluestakes the ID of the polygon, among those no seed names, that a point inside it finds as its nearest seed. That is the cell the seed generates, since every point of a Voronoi cell is nearer its own seed than any other. A seed that the points inside several such polygons find as their nearest seed generates at most one of them (the seeds of the others are missing fromseedsor have no location), and nothing says which: it keeps itsNAID, with a warning, so the locations nearest it have no cell and are filled (step 5). A seed with anNAID inside a polygon keeps it, and so does one whose cellvaluesdoes not hold. Withoutseeds, the polygon ofvaluesthat holds the location, byassign_features_to_polygons()with its defaulttie_breakand the summary's record of the count, so a location on the edge between two cells the profile scored is placed as that function places such a row (see "Ties"); a location in no polygon has no cell. Polygons without a CRS are read as that function read them for the rows ofpoints: the locations are handed to it in the CRS ofpoints, whatever CRSnewdataarrived in. Where that join runs on the sphere and s2 refuses a polygon ofvalues(see "Joins in longitude and latitude" inassign_features_to_polygons()), the call stops with an error of class"spatialkit_cells_degenerate"that gives the rows ofvaluess2 refuses (rowson the condition), names the cause and says to transformvaluesto a projected CRS.The value. The cell's pooled value of
response_var,..pooled_<prefix><response_var>with the prefixresp_orpred_, whenvalueshas that column; otherwise its mean,<prefix>mean_<response_var>. The columnestimatesays which. For a column summarised as response and as predictor the response's columns are read. The interval is the cell's..obs_lo_<prefix><response_var>and..obs_hi_<prefix><response_var>whenvalueshas them.The fill. A location with no cell, or whose cell holds no scored row and has no value in
values, takes the value and interval of the cell of its nearest scored row.filledis thenTRUEandfilled_fromis that cell's ID. The ID column still holds the location's own cell, as step 3 finds it, so the two columns together say where a location lies and where its value came from. A cell that holds a scored row is not filled. Wherevalueshas no value for such a cell (the summary could not compute the pooled columns, or has no row for the cell), its locations havevalue = NAandfilled = FALSE; a filled value isNAin the same case, when the cell it is taken from has none.The distance.
dist_nearestis the distance from the location to the nearest location of a scored row, in the units of the working CRS.dist_ratiois that distance over the layer's spacing: the median, over the distinct locations of the scored rows, of the distance to the nearest other one.
Cell IDs are matched as text. The ID column of values is character when
the summary joined a cells_sf whose IDs were of another class than the
assigned layer's, while points and seeds keep the class they had; the
IDs of all three are written out as text before any match, and both ID
columns returned have the class of the ID column of values wherever
that class can hold every ID (see Value).
Ties
A distance to a seed, a centre or a scored location counts as the same as
the least when its square is no more than the square of the least distance
lengthened by a margin, plus one part in ten million of the least squared
distance. The margin is one part in 1e10 of the largest coordinate of the
seeds, centres or scored locations searched, at most a millionth of the
diagonal of their bounding box and at least four times the spacing of
doubles at that coordinate. A location the same distance from two seeds takes the one
that comes first in seeds. For the seeds of a select_resolution()
result that is the lower-numbered centre, the rule resolution_profile()
scores with. When the summary's record of the count holds the centres the
profile scored and seeds are those centres (the seeds a selection
carries, or those a seeding function returns for it), in any order and any
CRS, the distances are taken to the centres as the record holds them, a tie
goes to the lower-numbered centre, and each centre gives the ID of the seed
that stands at it. Seeds count as the centres when they are as many and
each lies at a different centre, within a thousandth of its distance from
the next nearest one, so a seed that came back from another CRS with its
coordinates rounded, or seeds sorted by their cell, do not settle a tie
another way. A location on the edge between two such cells then takes the
cell assign_features_to_polygons() gives a row there, with seeds and
without. Other seeds, among them a subset of the centres, are searched as
given. A location the same distance from two locations of scored rows takes
the one in the lower column (x values within one part in ten million of the
diagonal of the scored locations' bounding box, or within one part in 1e10
of their largest coordinate (at most a millionth of that diagonal,
and at least four times the spacing of doubles at that coordinate) where
that is more, counting as one), then the
one lower in y, y values read the same way; among scored rows at one
location that lie in different cells, the cell that comes first in the
order of the IDs (ascending for numbers, the C-locale order for strings,
the order of the levels for a factor), and the same among locations in one
column and one row (two records of one station that differ only in the
last bits of their coordinates, which are the same distance from any
location, however near, while they lie within the margin of each
other). Neither rule depends on the order of
the rows of points, or on whether the FNN package is installed. A
tie is a tie between distances computed from the coordinates in the working
CRS, so on gridded locations the same layer in another projected CRS can
put a location on a boundary in the other cell; another unit or origin of
the same CRS moves only the last bits of the distances and settles the ties
the same way, with one exception. The margin grows with the largest
coordinate (about 0.46 mm at a UTM northing near 4,550 km, about half a
micrometre once the origin lies next to data 5 km across), so a location
whose distances to two of them differ by less than the margin with one
origin and by more with another is a tie with the first, settled as
above, and takes the nearer with the second.
Away from the observations
The value and the interval describe locations among the observations.
The interval's two ends are quantiles of leave-one-out errors, each the
error of predicting an observation from the others around it, so it says
nothing of a location far from every observation: a gap in the layer, or
ground beyond its edge. Away from the observations no interval here is
calibrated. dist_nearest and dist_ratio say how far each location is
from the data. No threshold is applied to either and no location is
flagged or left out. area_of_applicability() and fold_separation()
are the tools for deciding where a prediction applies.
The interval's coverage falls as dist_ratio grows; "How this was
checked" gives the figures.
Without the FNN package
The function makes three nearest-neighbour searches: each location's
nearest scored location, each location's nearest seed (and that of a
point inside each cell no seed names, when a seed with an NA ID lies
outside every cell; see step 3), and each scored location's nearest
other one (for the spacing). With FNN each uses
a kd-tree. Without it each compares every pair, and a search that would
compare more than getOption("spatialkit.exact_pairs_max", 2e9) pairs
stops the call first, with an error of class spatialkit_work_limit
that names the search (the fields work, limit and what). The search
for the spacing does not depend on the number of locations asked for:
above about 44,700 distinct scored locations the call stops without
FNN however few they are. The result is the same with and without
the package.
How this was checked
A simulation shows how the interval's coverage falls with dist_ratio
(dev/sim-pooled-values.R
in the package's GitHub repository; "How this was checked" in
summarize_by_cell() describes it). Each of 200 layers has 1,500
clustered observations in 333 cells, 300 further observations held out
at random among them and 300 held out as a block, which leaves a gap in
the layer. Over the held-out observations of all layers, the interval at
obs_level = 0.90 held 91.5 percent of those with a dist_ratio below
1, 89.2 percent from 1 to 2, 83.9 percent from 2 to 4, 77.9 percent from
4 to 8 and 69.5 percent above 8. It held 89.6 percent of the observations
held out at random and 75.1 percent of those in the block.
See also
summarize_by_cell(), which computes values;
build_tessellation(), whose index gives each seed's cell;
area_of_applicability() and fold_separation() for where a
prediction applies.
Other assign and summarise:
assign_features_to_polygons(),
kriging_adequacy(),
summarize_by_cell()
Examples
library(sf)
set.seed(1)
n <- 400
east <- runif(n, 0, 1e4)
north <- runif(n, 0, 1e4)
all_pts <- st_as_sf(
data.frame(x = 5e5 + east, y = 5e6 + north,
income = 30 + 5 * sin(east / 2000) + 5 * cos(north / 1500) +
rnorm(n)),
coords = c("x", "y"), crs = 32632
)
pts <- all_pts[1:350, ]
newpts <- all_pts[351:400, ]
# The count from a size, the cells, the observations in them, the values,
# all at the defaults: pooled values at 4.5 observations per cell.
prof <- resolution_profile(pts, "income")
sel <- select_resolution(prof)
seeds <- get_voronoi_seeds(method = "kmeans", n = sel, sample_points = pts)
tess <- build_tessellation(seeds, method = "voronoi", quiet = TRUE)
asg <- assign_features_to_polygons(pts, tess$cells, polygon_id_col = "cell_id")
out <- summarize_by_cell(asg, "income", id_col = "cell_id",
cells_sf = tess$cells, obs_level = 0.90)
# The value at other locations, each placed in the cell of its nearest
# seed. `tess$index` gives the cell of each seed.
seeds$cell_id <- tess$index
at <- cell_values_at(newpts, out, asg, "income", id_col = "cell_id",
seeds = seeds)
head(at)
#> cell_id value estimate obs_lo obs_hi filled filled_from dist_nearest
#> 1 39 28.44976 pooled 25.99484 30.80471 FALSE NA 238.4435
#> 2 4 37.27735 pooled 34.82244 39.63230 FALSE NA 115.6546
#> 3 58 23.90035 pooled 21.44544 26.25530 FALSE NA 271.2353
#> 4 29 38.39229 pooled 35.93738 40.74724 FALSE NA 367.1291
#> 5 75 21.00320 pooled 18.54829 23.35815 FALSE NA 227.2415
#> 6 19 34.78451 pooled 32.32960 37.13946 FALSE NA 351.8853
#> dist_ratio
#> 1 0.9403784
#> 2 0.4561211
#> 3 1.0697034
#> 4 1.4478917
#> 5 0.8961997
#> 6 1.3877727
attr(at, "values_at")
#> Cell values at other locations: income, the pooled value
#> cell : of the nearest seed (the Voronoi cell before its clip to the cover)
#> interval : 90% for ONE OBSERVATION (obs_lo, obs_hi), centred on the value
#> scored rows : 350 rows at 350 locations, measured in EPSG:32632
#> spacing : 253.6, the median distance from a scored location to the
#> nearest other; dist_ratio is dist_nearest over it
#> filled : no location
#> count : 78 cells, set by size (the summary's record, in `count_rule`)
#> Away from the observations no interval here is calibrated.
# Without the seeds a location is placed in the polygon that holds it. One
# beyond the layer's edge lies in no polygon: it is filled from the cell of
# the nearest observation, and `dist_ratio` says how far that is.
beyond <- st_sf(geometry = st_sfc(st_point(c(5e5 + 2e4, 5e6 + 5e3)), crs = 32632))
cell_values_at(rbind(st_sf(geometry = st_geometry(newpts[1:2, ])), beyond),
out, asg, "income", id_col = "cell_id")
#> cell_id value estimate obs_lo obs_hi filled filled_from dist_nearest
#> 1 39 28.44976 pooled 25.99484 30.80471 FALSE NA 238.4435
#> 2 4 37.27735 pooled 34.82244 39.63230 FALSE NA 115.6546
#> 3 NA 25.39825 pooled 22.94334 27.75320 TRUE 74 10204.7354
#> dist_ratio
#> 1 0.9403784
#> 2 0.4561211
#> 3 40.2456524