Evaluation rubric
Taxbot is evaluated on whether it retrieves the right South African authorities and whether answers remain faithful
to those sources. Public scores are not yet published. Status: harness_pending.
Retrieval
Whether the gold legislation, SARS guidance or case law appears in the retrieved working set for curated professional questions on the production chat path.
Published score: Pending
Answer quality
Hard checks on calculations, dates and cited provisions, plus rubric scoring on grounded analysis. A pass requires core score thresholds with no fatal period or rule errors.
Published score: Pending
Machine-readable status
results.json reports harness_pending until scores are published.