Why are surface n-gram overlap metrics such as ROUGE and BLEU considered inadequate for evaluating answer quality in production RAG systems like FieldDesk?
Day 20: Evaluate Answer Quality and Groundedness
Evaluate Answer Quality and Groundedness
Loading lesson content...
What is the primary engineering rationale for enforcing the Veto Rule in the RAG generation scoring function above?
In PolicyDesk, an automated LLM-as-a-judge evaluation harness decomposes an assistant answer into four atomic claims. Claims 1, 2, and 3 are logically entailed by retrieved corporate handbook chunks. Claim 4 asserts that employees traveling internationally receive an unapproved tax-free cash allowance of 500 euros, a statement absent from all retrieved evidence. How should the groundedness evaluator score this response?