Puts uncertainty intervals on anything computed from a weekly panel (coefficients, contributions, average or marginal ROI) by resampling blocks of consecutive rows, refitting, and recomputing. Resampling single rows would destroy the autocorrelation that carryover creates and make the intervals too narrow; resampling blocks keeps it within each block (Künsch, 1989).
Usage
block_bootstrap(
data,
statistic,
times = 200L,
block_length = NULL,
by = NULL,
level = 0.9,
seed = NULL,
panel = c("shared", "independent"),
index = NULL,
circular = FALSE,
cores = 1L
)Arguments
- data
A data frame, rows in time order (within each group when
byis supplied), unlessindexnames a column to order them by. It should hold everythingstatisticneeds: typically the KPI, the transformed media, the controls and the raw spend.- statistic
A function of one data frame returning a named numeric vector. It is called once on
datafor the point estimate and once per replicate on a resampled copy. Replicates are matched to the estimate by name, so the order of the elements may change between calls; a replicate that returns a name the estimate does not have counts as failed, and a name it leaves out is missing for that replicate.- times
Number of bootstrap replicates, at most 1,000,000.
- block_length
Rows per block, or periods per block when
byis supplied withpanel = "shared". The default,ceiling(n^(1/3))withnthe number of rows ofdata, is the usual rate-optimal order for the moving block bootstrap. It is often too short for media data; see "Choosing the block length".- by
Optional character vector of grouping columns, for a panel such as one series per geography. How the groups are resampled is set by
panel.- level
Confidence level of the percentile intervals.
- seed
Optional random seed, one whole number, for reproducible intervals. Every replicate's rows are drawn and the point estimate is computed after seeding, and the caller's random number stream is restored afterwards. Each replicate's call to
statisticstarts from a seed of its own, drawn before any replicate is evaluated, so a statistic that draws random numbers is reproducible too.- panel
How groups are resampled when
byis supplied."shared", the default, draws one set of blocks of periods and applies it to every group, so a shock that hit every geography in the same week stays together in each replicate."independent"draws separate blocks for each group. That treats the groups as unrelated series, and for a statistic pooled across groups it gives intervals that are too narrow whenever the groups share shocks.- index
Optional name of a numeric,
DateorPOSIXctcolumn holding each row's period. Rows are put in period order within each group, and withpanel = "shared"they are matched across groups by period, so the groups may cover different periods. Withoutindex, rowtof every group is taken to be the same period, andpanel = "shared"needs every group to have the same number of rows.- circular
If
TRUE, blocks wrap around from the end of each series to its start (the circular block bootstrap of Politis and Romano, 1992), so every row is equally likely to be drawn. The defaultFALSEis the moving block bootstrap, which draws the first and lastblock_length - 1rows of each series less often than the rest.- cores
Number of cores over which the replicates are evaluated. Values above 1 use
parallel::mclapply()on Unix-alikes and a socket cluster elsewhere. Every replicate's rows, and the seed its call tostatisticstarts from, are drawn in this session before any replicate is evaluated, so the result does not depend oncores. The workers of a socket cluster are new R sessions that load mediamix and nothing else, so therestatisticmust call other packages withpkg::fun()and must not rely on functions or data defined in your session.
Value
An object of class mm_bootstrap: a data frame with one row per
element of the statistic and columns term, estimate (on the original
data), lower, upper (percentile interval) and std_error, with the
replicate matrix attached as the attribute "replicates" (one row per
replicate, one column per term). Replicates in which statistic failed
are dropped and counted in a message that quotes the first error. Each
term's interval and standard error use the replicates in which that term
is finite; missing or infinite values stay in the replicate matrix and are
counted per term in a message.
Subsetting the rows keeps the replicate matrix in step with the terms.
Removing one of the five columns returns a plain data frame, as do
as.data.frame() and, with the broom column names std.error, conf.low
and conf.high, tidy().
Details
Transform the media before bootstrapping and put the transformed columns
in data. Resampled blocks are glued end to end, so re-running the adstock
filter on resampled rows would carry spend across block boundaries that
were never adjacent in time; the transformed columns already hold each
week's correct carryover.
The interval reflects sampling variation given the model specification,
including the transform parameters. It does not include the uncertainty in
choosing those parameters unless statistic re-tunes them, which is
possible but slow.
Every replicate's rows are drawn before statistic is first called, so a
statistic that sets its own seed, as a cross-validated fit often does,
cannot make the resamples repeat. Each replicate's call to statistic
then starts from a seed drawn for it in advance, which is what makes the
replicates the same in a worker process as in this one. Random numbers
that statistic draws on the replicates do not advance the caller's
stream.
Panels
With by and the default panel = "shared", each replicate draws one set
of blocks of periods and takes every group's rows for those periods, the
moving blocks bootstrap for panels of Gonçalves (2011). Shocks common to
every group, such as a national holiday or a competitor's launch, then stay
together, which they must for a statistic pooled across groups. Each
group's rows are passed to statistic together, in time order, one group
after another.
Groups are matched period by period. With index, a group has no row for
a period it does not cover, so group sizes vary from replicate to
replicate. Without index, row t of each group is period t, and groups
of different lengths are an error: supply index, or use
panel = "independent".
With panel = "independent", every group draws its own blocks and keeps
its own size. That is only right when the groups share no shocks.
Choosing the block length
A block must be long enough to hold the dependence that matters for the
statistic. In media data that is mostly carryover, so a block at least as
long as the longest channel's effective_window() is a sensible floor.
The default ceiling(n^(1/3)) is a rate rather than a length chosen for
the data, and it is often shorter: 6 for 156 weekly rows.
Percentile intervals from a block bootstrap of a short series cover less than their nominal level, and the shortfall grows with the autocorrelation. For the mean of 156 values from an AR(1) process, nominal 90% intervals from 200 replicates covered the true mean in about 81% of 2,000 simulated series when the autocorrelation was 0.5 and blocks were 6 rows long, and in about 71% when it was 0.8, rising to about 75% with blocks of 13 rows. Circular blocks gave much the same figures. Read the intervals as a lower bound on the uncertainty.
The block length is checked against the series it resamples: longer than half the shortest series gives a warning, because few distinct resamples remain, and as long as the series is an error, because every replicate would repeat the data.
References
Gonçalves, S. (2011). The moving blocks bootstrap for panel linear regression models with individual fixed effects. Econometric Theory, 27(5), 1048–1082. doi:10.1017/S0266466610000630
Künsch, H. R. (1989). The jackknife and the bootstrap for general stationary observations. The Annals of Statistics, 17(3), 1217–1241. doi:10.1214/aos/1176347265
Lahiri, S. N. (2003). Resampling Methods for Dependent Data. Springer. doi:10.1007/978-1-4757-3803-2
Politis, D. N. and Romano, J. P. (1992). A circular block-resampling procedure for stationary data. In R. LePage and L. Billard (eds.), Exploring the Limits of Bootstrap, 263–270. Wiley.
Examples
data(mm_weekly)
north <- mm_weekly[mm_weekly$geo == "north", ]
channels <- c("tv", "video", "search", "social", "display")
truth <- attr(mm_weekly, "truth")
# Transform once, on the full series, then bootstrap the model on top.
spec <- lapply(stats::setNames(channels, channels), function(ch) list(
adstock = list(decay = truth$decay[[ch]]),
saturation = list(half_max = truth$half_max[[ch]],
shape = truth$shape[[ch]])))
media <- transform_plan(north[channels], spec)
names(media) <- paste0(channels, "_t")
d <- cbind(north, media, week = seq_len(nrow(north)))
average_roi <- function(d) {
fit <- lm(revenue ~ tv_t + video_t + search_t + social_t + display_t +
week + price + seasonality + holiday, data = d)
contrib <- colSums(sweep(as.matrix(d[paste0(channels, "_t")]), 2,
coef(fit)[paste0(channels, "_t")], `*`))
stats::setNames(contrib / colSums(d[channels]), channels)
}
boot <- block_bootstrap(d, average_roi, times = 50, block_length = 13,
seed = 1)
boot
#>
#> ── Block bootstrap
#> 50 replicates, blocks of 13 rows, 90% percentile intervals
#> term estimate lower upper std_error
#> tv 1.310 1.10874 1.882 0.2579
#> video 1.924 0.01862 3.495 1.1227
#> search 2.794 1.58295 4.258 0.8991
#> social 3.614 0.51808 5.213 1.4353
#> display 1.517 -0.03334 5.217 1.6852
# A panel: every geography takes the same blocks of weeks.
pooled_mean <- function(d) c(revenue = mean(d$revenue))
block_bootstrap(mm_weekly, pooled_mean, times = 50, block_length = 13,
by = "geo", index = "date", seed = 1)
#>
#> ── Block bootstrap
#> 50 replicates, blocks of 13 periods shared across groups, 90% percentile
#> intervals
#> term estimate lower upper std_error
#> revenue 26271 25485 26682 370.5