admin
f7ad9ba51f
- introduce Gold Standard evaluation corpus - document decision taxonomy - define prompt-engineering methodology - add regression workflow - establish Prompt Version 2 baseline - validate decision_simple, decision_deferred and decision_none
15 lines
875 B
Markdown
15 lines
875 B
Markdown
# evil_meeting
|
|
|
|
Tests a deliberately difficult meeting with interruptions, corrections, topic switches, changed positions, absent referenced people, and near-decisions.
|
|
|
|
Every utterance is designed to trigger a common extraction failure. The meeting mentions Omar and Platform, but neither is a participant. It includes an explicit non-decision on migration and a real decision only on excluding FR-7 from Friday's batch.
|
|
|
|
Typical LLM mistakes:
|
|
|
|
- Extracting a migration decision even though the group says no migration decision today.
|
|
- Assigning Dana a todo even though she retracts it.
|
|
- Treating Omar as a participant or technical owner.
|
|
- Claiming Platform approved something despite being absent.
|
|
- Losing the correction from "old export" to "nightly CSV job" and from API export to CSV export.
|
|
- Treating Dana's opinion about rollout appearance as a fact or decision.
|