The active tokenizer

The active tokenizer

Usage

gr_tokenizer()

Value

The name of the tokenizer currently in use, visibly.

See also

gr_set_tokenizer(), gr_count_tokens()

Other cost and token functions: gr_budget(), gr_count_tokens(), gr_estimate_cost(), gr_model_info(), gr_model_limits(), gr_models(), gr_register_model(), gr_set_tokenizer(), gr_truncate_tokens()

Examples

gr_tokenizer()
#> [1] "heuristic"