All Lessons
Why is evaluating an enterprise RAG system solely by inspecting the final generated text (the 'vibe check') fundamentally untrustworthy?
1 / 1
beginner 45 min
Day 3: Create a Gold Set with Evidence Labels
Create a Gold Set with Evidence Labels
Based on Anthropic Claude documentation and the CCA-F exam blueprint. Updated August 2026.
Loading lesson content...
In the FieldDesk gold set item above, why does the ground_truth_evidence field explicitly specify chunk_id instead of merely citing MAN-X840-D.pdf as the target document?
1 / 1
The evaluation harness runs on PolicyDesk across 500 benchmark queries. Retrieval metrics show Recall@5 is 0.97 and MRR is 0.92, but end-to-end Faithfulness is only 0.69 due to the LLM introducing ungrounded assertions during synthesis. Based on this decoupled evaluation, what is the correct engineering priority?
1 / 1