harnessr keeps two caches in memory for the life of the R session: the
documents gr_ingest() has already extracted and cleaned, keyed on the file
and every setting that changes the result, and the vectors the built-in
"api" embedder has already fetched (see gr_embed()). This empties them.
The next ingestion of a file reads and cleans it again, and the next ranking
sends its embeddings requests again, which are counted against max_calls
and max_cost_usd like any others.
Details
Each cache is bounded: when what it holds passes its budget (256 MB of documents, 128 MB of vectors) the entries used least recently are dropped, so a long session or a Shiny server does not need this to stay within memory. It is for starting again from nothing: to give the memory back at once, or to time or test a run as a fresh session would see it.
Not to be confused with gr_cache_clear(), which deletes the model
responses a gr_cache() keeps in a directory on disk. Neither touches
the other's cache: this leaves every stored response in place, and
gr_cache_clear() leaves every document and vector in memory.
gr_options(cache_documents = FALSE, cache_embeddings = FALSE) turns the
two memory caches off instead.
See also
gr_cache_clear() for the on-disk response cache, gr_options()
for cache_documents and cache_embeddings, gr_ingest(), gr_embed()
Examples
doc <- gr_ingest(gr_example())
#> Using cached ingestion for this document + settings.
gr_flush_caches("documents")
gr_flush_caches()