Skip to contents

Gives each location of newdata the value of the cell it lies in, read from a summarize_by_cell() result: the pooled value when the summary holds one, the plain mean otherwise, with the interval for one observation when the summary holds it. Beside each value it returns the location's distance from the nearest observation, since a cell's value describes the part of the cell where its observations lie.

Usage

cell_values_at(
  newdata,
  values,
  points,
  response_var,
  id_col = "poly_id",
  seeds = NULL
)

Arguments

newdata

The locations: an sf layer. A feature that is not a point is reduced to one with coerce_to_points(, "auto"); a collection holding a polygon s2 cannot take a centroid of in longitude and latitude (edges that cross), or whose centroid GEOS puts outside it, stops the call with the error of class spatialkit_features_degenerate, raised in this function's name and naming its rows of newdata. No column of it is read. With no CRS while points has one, the call stops. A layer whose geometries mix dimensions or carry M (XYM or XYZM) is read in x and y (step 1).

values

The summarize_by_cell() result for points. It must hold the ID column and a cell value of response_var: the pooled value (estimate = "pooled") or the mean (the default agg_funs). With seeds = NULL it must also carry the cell polygons, which it does when it was made with cells_sf. A values with no row, or whose IDs are all NA, stops the call, with or without seeds. Its polygons are read in x and y when they mix dimensions or carry M (XYM or XYZM).

points

The assigned layer values was computed from: an sf layer with the ID column and response_var, and at least one scored row.

response_var

Name of the summarised column wanted: the response, or a predictor.

id_col

Name of the ID column, found as summarize_by_cell() finds it: when points has no column of that name, the first of "poly_id", "polygon_id" and "cell_id" it has. It cannot have the name of another column of the result (value, estimate, obs_lo, obs_hi, filled, filled_from, dist_nearest, dist_ratio).

seeds

NULL, or the seeds the cells were built on: an sf POINT layer with a column named as the ID column that gives each seed's cell. For cells from tess <- build_tessellation(seeds) that is seeds$cell_id <- tess$index when the ID column is "cell_id". For seeds a seeding function returned and the seeds of a select_resolution() result, index gives each seed the cell it generates. Other points are indexed by the cell that holds them, and one outside the boundary is NA, or the ID of the nearest cell, which can be the cell of another seed, when it lies within a thousandth of the median cell width of it; with the cell polygons in values a seed that is NA is given its cell when step 3 can tell which it is (it is not when the points inside several cells no seed names find it as their nearest seed), and without them the locations nearest it have none. Passed through get_voronoi_seeds(method = "provided", seeds = ) before the tessellation, points placed by hand are seeds (see create_voronoi_polygons()). Two seeds at different locations with one ID are an error; seeds that share a location share its cell, as index gives them. NA marks a seed without a cell. When no seed with a location has the ID of a cell of values (an ID column from another tessellation, IDs that are all NA, or the IDs of cells on seeds with an empty point or a coordinate that is not finite or lies beyond 1e150, as given or once placed in the working CRS, such as metres given EPSG:4326), counting the cells step 3 gives seeds outside the polygons, the call stops, since no location could take its own cell's value; one seed with such an ID and a location is enough, and a seed whose ID values has no row for stands for a cell without a value. Without a CRS beside points with one, the seeds are read as step 1 says.

Value

A data frame with one row per row of newdata, in its order:

the ID column

The cell the location lies in, under the name of the ID column; NA for a location in no cell.

value

The cell's value, or for a filled row the value of the cell in filled_from.

estimate

"pooled" or "mean": which value of the summary value is.

obs_lo, obs_hi

The interval for one observation, when values holds it.

filled

TRUE when the location has no cell, or its cell neither a scored row nor a value, and the value is that of the cell of the nearest scored row.

filled_from

For a filled row, the ID of the cell the value came from; NA otherwise.

dist_nearest

Distance to the nearest location of a scored row, in the units of the working CRS.

dist_ratio

dist_nearest over the spacing; NA when the scored rows sit on one location.

A row of newdata with an empty geometry, or with a coordinate that is not finite or lies beyond 1e150 as given or in the working CRS, has NA in every column. The two ID columns have the class of the ID column of values; when a cell that values has no row for has an ID that class cannot hold (a string beside numbers), both are character.

What produced the frame is in attr(, "values_at"), a list of class "spatialkit_values_at" with a print method: assignment ("seeds" or "polygons"), crs (the working CRS), column (response_var), estimate, obs_level (the level of the interval columns; NULL without them, and NA when values holds them and no longer says at what level), n (the scored rows), locations (their distinct locations), spacing (what dist_ratio divides by; NA with one location), filled (how many rows of newdata were filled) and count_rule (the summary's record of how the cell count was set, or NULL). It describes the call: a subset of the frame's rows keeps it as it is.

The function raises no classed warning of its own; the warning that says how seeds without a CRS were read (step 1) and the one that counts the seeds outside the polygons of values left without a cell (step 3) are plain ones. It passes on the conditions that coerce_to_points(), a CRS transformation and assign_features_to_polygons() raise, with two exceptions. The error of class spatialkit_features_degenerate with which coerce_to_points() refuses a polygon s2 cannot take a centroid of, or a centroid GEOS puts outside its feature, is raised in this function's name, with the same class and fields, naming newdata or points and its rows. The warning assign_features_to_polygons() raises when no location at all falls in a polygon is neither raised nor logged, since every such row has filled = TRUE, which says the same row by row.

Details

The function works in six steps.

  1. The working CRS. The CRS the summary measured in: the one in attr(values, "count_rule") when that record names a projected CRS, else the one in attr(values, "pooled"), else the CRS of points when it is projected and the one ensure_projected() picks for it when it is not. points, newdata and seeds are transformed to it. newdata, seeds and points are read in x and y: a layer whose points, or the points taken from its other features, do not all have the same dimensions (some with a Z or M coordinate) is read without its Z and M, since sf reads a layer in the dimension of its first point. So is a layer of points, polygons or lines in newdata or points whose geometries carry M (XYM or XYZM), which GEOS does not read. So are the polygons of values when they do not all have the same dimensions. Polygons of values whose geometries carry M are read in x and y as well. When neither points nor newdata has a CRS the coordinates are used as they are, unless those of points look like longitude and latitude: both are then read as EPSG:4326, as summarize_by_cell() reads points, with ensure_projected()'s warning; newdata or seeds with an x outside -180 to 360 or a y beyond 90 north or south cannot be in degrees, and stop the call. seeds without a CRS beside points with one are read as build_tessellation() reads a boundary without a CRS, with a warning: beside a projected CRS, coordinates that look like longitude and latitude are taken as EPSG:4326 and transformed, and others are taken to be in the CRS of points; beside a geographic CRS, coordinates within the ranges of longitude and latitude are taken to be in it, and others stop the call. newdata without a CRS beside points with one stops the call.

  2. The scored rows. The rows of points with a finite response_var, a cell ID and a location: a point whose coordinates are finite and within 1e150, as given and in the working CRS. Their distinct locations are the observations every distance below is taken from.

  3. The cell of a location. With seeds, the cell of the nearest seed. That is the Voronoi cell before its clip to the cover, so every location has one, unless its nearest seed has an NA ID: a location nearest such a seed has no cell. build_tessellation() gives each seed of a layer it takes for seeds (see create_voronoi_polygons()) the cell it generates, a seed outside a boundary that is not convex (a k-means centre over a bay) included, and NA only to a seed whose cell the clip removed whole. Other points, such as points placed by hand, are indexed by the cell that holds them, so one outside a boundary that is not convex is NA although its cell is built. One that lies in no cell but within a thousandth of the median cell width of a cell takes the ID of the nearest cell instead, which beside a boundary that is not convex can be the cell of another seed; two seeds with one ID stop the call (see seeds). When values carries the cell polygons, a seed that is NA is given its cell here: a seed with an NA ID that lies in no polygon of values takes the ID of the polygon, among those no seed names, that a point inside it finds as its nearest seed. That is the cell the seed generates, since every point of a Voronoi cell is nearer its own seed than any other. A seed that the points inside several such polygons find as their nearest seed generates at most one of them (the seeds of the others are missing from seeds or have no location), and nothing says which: it keeps its NA ID, with a warning, so the locations nearest it have no cell and are filled (step 5). A seed with an NA ID inside a polygon keeps it, and so does one whose cell values does not hold. Without seeds, the polygon of values that holds the location, by assign_features_to_polygons() with its default tie_break and the summary's record of the count, so a location on the edge between two cells the profile scored is placed as that function places such a row (see "Ties"); a location in no polygon has no cell. Polygons without a CRS are read as that function read them for the rows of points: the locations are handed to it in the CRS of points, whatever CRS newdata arrived in. Where that join runs on the sphere and s2 refuses a polygon of values (see "Joins in longitude and latitude" in assign_features_to_polygons()), the call stops with an error of class "spatialkit_cells_degenerate" that gives the rows of values s2 refuses (rows on the condition), names the cause and says to transform values to a projected CRS.

  4. The value. The cell's pooled value of response_var, ..pooled_<prefix><response_var> with the prefix resp_ or pred_, when values has that column; otherwise its mean, <prefix>mean_<response_var>. The column estimate says which. For a column summarised as response and as predictor the response's columns are read. The interval is the cell's ..obs_lo_<prefix><response_var> and ..obs_hi_<prefix><response_var> when values has them.

  5. The fill. A location with no cell, or whose cell holds no scored row and has no value in values, takes the value and interval of the cell of its nearest scored row. filled is then TRUE and filled_from is that cell's ID. The ID column still holds the location's own cell, as step 3 finds it, so the two columns together say where a location lies and where its value came from. A cell that holds a scored row is not filled. Where values has no value for such a cell (the summary could not compute the pooled columns, or has no row for the cell), its locations have value = NA and filled = FALSE; a filled value is NA in the same case, when the cell it is taken from has none.

  6. The distance. dist_nearest is the distance from the location to the nearest location of a scored row, in the units of the working CRS. dist_ratio is that distance over the layer's spacing: the median, over the distinct locations of the scored rows, of the distance to the nearest other one.

Cell IDs are matched as text. The ID column of values is character when the summary joined a cells_sf whose IDs were of another class than the assigned layer's, while points and seeds keep the class they had; the IDs of all three are written out as text before any match, and both ID columns returned have the class of the ID column of values wherever that class can hold every ID (see Value).

Ties

A distance to a seed, a centre or a scored location counts as the same as the least when its square is no more than the square of the least distance lengthened by a margin, plus one part in ten million of the least squared distance. The margin is one part in 1e10 of the largest coordinate of the seeds, centres or scored locations searched, at most a millionth of the diagonal of their bounding box and at least four times the spacing of doubles at that coordinate. A location the same distance from two seeds takes the one that comes first in seeds. For the seeds of a select_resolution() result that is the lower-numbered centre, the rule resolution_profile() scores with. When the summary's record of the count holds the centres the profile scored and seeds are those centres (the seeds a selection carries, or those a seeding function returns for it), in any order and any CRS, the distances are taken to the centres as the record holds them, a tie goes to the lower-numbered centre, and each centre gives the ID of the seed that stands at it. Seeds count as the centres when they are as many and each lies at a different centre, within a thousandth of its distance from the next nearest one, so a seed that came back from another CRS with its coordinates rounded, or seeds sorted by their cell, do not settle a tie another way. A location on the edge between two such cells then takes the cell assign_features_to_polygons() gives a row there, with seeds and without. Other seeds, among them a subset of the centres, are searched as given. A location the same distance from two locations of scored rows takes the one in the lower column (x values within one part in ten million of the diagonal of the scored locations' bounding box, or within one part in 1e10 of their largest coordinate (at most a millionth of that diagonal, and at least four times the spacing of doubles at that coordinate) where that is more, counting as one), then the one lower in y, y values read the same way; among scored rows at one location that lie in different cells, the cell that comes first in the order of the IDs (ascending for numbers, the C-locale order for strings, the order of the levels for a factor), and the same among locations in one column and one row (two records of one station that differ only in the last bits of their coordinates, which are the same distance from any location, however near, while they lie within the margin of each other). Neither rule depends on the order of the rows of points, or on whether the FNN package is installed. A tie is a tie between distances computed from the coordinates in the working CRS, so on gridded locations the same layer in another projected CRS can put a location on a boundary in the other cell; another unit or origin of the same CRS moves only the last bits of the distances and settles the ties the same way, with one exception. The margin grows with the largest coordinate (about 0.46 mm at a UTM northing near 4,550 km, about half a micrometre once the origin lies next to data 5 km across), so a location whose distances to two of them differ by less than the margin with one origin and by more with another is a tie with the first, settled as above, and takes the nearer with the second.

Away from the observations

The value and the interval describe locations among the observations. The interval's two ends are quantiles of leave-one-out errors, each the error of predicting an observation from the others around it, so it says nothing of a location far from every observation: a gap in the layer, or ground beyond its edge. Away from the observations no interval here is calibrated. dist_nearest and dist_ratio say how far each location is from the data. No threshold is applied to either and no location is flagged or left out. area_of_applicability() and fold_separation() are the tools for deciding where a prediction applies.

The interval's coverage falls as dist_ratio grows; "How this was checked" gives the figures.

Without the FNN package

The function makes three nearest-neighbour searches: each location's nearest scored location, each location's nearest seed (and that of a point inside each cell no seed names, when a seed with an NA ID lies outside every cell; see step 3), and each scored location's nearest other one (for the spacing). With FNN each uses a kd-tree. Without it each compares every pair, and a search that would compare more than getOption("spatialkit.exact_pairs_max", 2e9) pairs stops the call first, with an error of class spatialkit_work_limit that names the search (the fields work, limit and what). The search for the spacing does not depend on the number of locations asked for: above about 44,700 distinct scored locations the call stops without FNN however few they are. The result is the same with and without the package.

How this was checked

A simulation shows how the interval's coverage falls with dist_ratio (dev/sim-pooled-values.R in the package's GitHub repository; "How this was checked" in summarize_by_cell() describes it). Each of 200 layers has 1,500 clustered observations in 333 cells, 300 further observations held out at random among them and 300 held out as a block, which leaves a gap in the layer. Over the held-out observations of all layers, the interval at obs_level = 0.90 held 91.5 percent of those with a dist_ratio below 1, 89.2 percent from 1 to 2, 83.9 percent from 2 to 4, 77.9 percent from 4 to 8 and 69.5 percent above 8. It held 89.6 percent of the observations held out at random and 75.1 percent of those in the block.

See also

summarize_by_cell(), which computes values; build_tessellation(), whose index gives each seed's cell; area_of_applicability() and fold_separation() for where a prediction applies.

Other assign and summarise: assign_features_to_polygons(), kriging_adequacy(), summarize_by_cell()

Examples

library(sf)
set.seed(1)
n <- 400
east  <- runif(n, 0, 1e4)
north <- runif(n, 0, 1e4)
all_pts <- st_as_sf(
  data.frame(x = 5e5 + east, y = 5e6 + north,
             income = 30 + 5 * sin(east / 2000) + 5 * cos(north / 1500) +
               rnorm(n)),
  coords = c("x", "y"), crs = 32632
)
pts    <- all_pts[1:350, ]
newpts <- all_pts[351:400, ]

# The count from a size, the cells, the observations in them, the values,
# all at the defaults: pooled values at 4.5 observations per cell.
prof  <- resolution_profile(pts, "income")
sel   <- select_resolution(prof)
seeds <- get_voronoi_seeds(method = "kmeans", n = sel, sample_points = pts)
tess  <- build_tessellation(seeds, method = "voronoi", quiet = TRUE)
asg   <- assign_features_to_polygons(pts, tess$cells, polygon_id_col = "cell_id")
out   <- summarize_by_cell(asg, "income", id_col = "cell_id",
                           cells_sf = tess$cells, obs_level = 0.90)

# The value at other locations, each placed in the cell of its nearest
# seed.  `tess$index` gives the cell of each seed.
seeds$cell_id <- tess$index
at <- cell_values_at(newpts, out, asg, "income", id_col = "cell_id",
                     seeds = seeds)
head(at)
#>   cell_id    value estimate   obs_lo   obs_hi filled filled_from dist_nearest
#> 1      39 28.44976   pooled 25.99484 30.80471  FALSE          NA     238.4435
#> 2       4 37.27735   pooled 34.82244 39.63230  FALSE          NA     115.6546
#> 3      58 23.90035   pooled 21.44544 26.25530  FALSE          NA     271.2353
#> 4      29 38.39229   pooled 35.93738 40.74724  FALSE          NA     367.1291
#> 5      75 21.00320   pooled 18.54829 23.35815  FALSE          NA     227.2415
#> 6      19 34.78451   pooled 32.32960 37.13946  FALSE          NA     351.8853
#>   dist_ratio
#> 1  0.9403784
#> 2  0.4561211
#> 3  1.0697034
#> 4  1.4478917
#> 5  0.8961997
#> 6  1.3877727
attr(at, "values_at")
#> Cell values at other locations: income, the pooled value
#>   cell        : of the nearest seed (the Voronoi cell before its clip to the cover)
#>   interval    : 90% for ONE OBSERVATION (obs_lo, obs_hi), centred on the value
#>   scored rows : 350 rows at 350 locations, measured in EPSG:32632
#>   spacing     : 253.6, the median distance from a scored location to the
#>                 nearest other; dist_ratio is dist_nearest over it
#>   filled      : no location
#>   count       : 78 cells, set by size (the summary's record, in `count_rule`)
#> Away from the observations no interval here is calibrated.

# Without the seeds a location is placed in the polygon that holds it.  One
# beyond the layer's edge lies in no polygon: it is filled from the cell of
# the nearest observation, and `dist_ratio` says how far that is.
beyond <- st_sf(geometry = st_sfc(st_point(c(5e5 + 2e4, 5e6 + 5e3)), crs = 32632))
cell_values_at(rbind(st_sf(geometry = st_geometry(newpts[1:2, ])), beyond),
               out, asg, "income", id_col = "cell_id")
#>   cell_id    value estimate   obs_lo   obs_hi filled filled_from dist_nearest
#> 1      39 28.44976   pooled 25.99484 30.80471  FALSE          NA     238.4435
#> 2       4 37.27735   pooled 34.82244 39.63230  FALSE          NA     115.6546
#> 3      NA 25.39825   pooled 22.94334 27.75320   TRUE          74   10204.7354
#>   dist_ratio
#> 1  0.9403784
#> 2  0.4561211
#> 3 40.2456524