All Lessons

Why must an evaluation dataset be frozen before the team begins tuning prompt templates and retrieval configurations for Helpdesk Copilot?

1 / 1
advanced 45 min

Day 17: Build an Evaluation Set Before Tuning Further

Build an Evaluation Set Before Tuning Further

Loading lesson content...

Review the evaluation case definition below. Why does this case structure specify required_facts and forbidden_claims rather than asserting against a single canonical answer string?

1 / 1

The engineering team compiles 100 reviewed evaluation cases across five ticket categories. Why should these cases be partitioned into a development set and an untouched holdout set?

1 / 1