Build a gr_answer, the object every reader must return

Exported because it is part of the extension API: gr_read() rejects anything that does not inherit "gr_answer", so a custom reader registered with gr_register_reader() cannot be written without this. Using it also means your reader reports partial, evidence and notes the same way the built-ins do, so gr_compare() can put it in the same table.

Usage

new_answer(
  text,
  reader,
  question,
  chunks_used,
  trace,
  evidence = NULL,
  partial = FALSE,
  notes = list(),
  chunks_sent = NULL,
  premise = NULL
)

Arguments

text

The answer string. Use "NOT_IN_DOCUMENT" when the chunks did not contain the answer; gr_compare() counts that separately from a failure.

reader

Your reader's name, as registered.

question

The question, carried through for the record.

chunks_used

Integer chunk_ids that CONTRIBUTED to the answer, not every chunk you sent.

trace

The gr_trace passed to your reader. Pass it through; do not create a new one, or your calls will not appear in the run's totals.

evidence

Optional data frame of supporting spans; build it with the columns chunk_id, text, page, section, score.

partial

TRUE if anything degraded: a failed call, a dropped chunk, a truncated prompt. Callers are told to check this before trusting $answer, so setting it honestly matters more than it looks.

notes

Named list of whatever your reader wants to report.

chunks_sent

Chunk ids the reader actually put in front of the model, when that is more than chunks_used. Used only to tell a fabricated citation from a faithful one: a per-chunk reader drops the chunks that answered "not in this excerpt", and citing one of those is not an invention. Defaults to chunks_used.

premise

Whether, and how, the question's premise was checked; see premise in gr_read_spec(). NULL (the default) when your reader did not check it, which leaves the field out of the answer. NA or FALSE when it checked and the model reported nothing contradicting it. Otherwise the sentence in which the model said what the document states instead, which should then be text as well, so that $answer says it and is_not_found() is FALSE. Any other value, TRUE among them, is an error.

Value

A gr_answer.

See also

gr_register_reader(), gr_answer, gr_read(), new_chunks()

Other reading functions: gr_answer, gr_check_reader(), gr_read(), gr_read_spec(), gr_reader_signature(), gr_readers(), gr_register_reader(), is_not_found()

Examples

# The minimum a custom reader has to return.
tr <- gr_trace()
a <- new_answer("Revenue was 45.2 million.", "my_reader", "What was revenue?",
                chunks_used = 1L, trace = tr, notes = list(strategy = "first chunk"))
a$partial
#> [1] FALSE
a$notes$strategy
#> [1] "first chunk"