Establish prompt engineering baseline with Gold Standard tests

- introduce Gold Standard evaluation corpus - document decision taxonomy - define prompt-engineering methodology - add regression workflow - establish Prompt Version 2 baseline - validate decision_simple, decision_deferred and decision_none
2026-07-30 12:13:10 +02:00
parent 07b0d80113
commit f7ad9ba51f
43 changed files with 1288 additions and 45 deletions
@@ -0,0 +1,14 @@
+# evil_meeting
+
+Tests a deliberately difficult meeting with interruptions, corrections, topic switches, changed positions, absent referenced people, and near-decisions.
+
+Every utterance is designed to trigger a common extraction failure. The meeting mentions Omar and Platform, but neither is a participant. It includes an explicit non-decision on migration and a real decision only on excluding FR-7 from Friday's batch.
+
+Typical LLM mistakes:
+
+- Extracting a migration decision even though the group says no migration decision today.
+- Assigning Dana a todo even though she retracts it.
+- Treating Omar as a participant or technical owner.
+- Claiming Platform approved something despite being absent.
+- Losing the correction from "old export" to "nightly CSV job" and from API export to CSV export.
+- Treating Dana's opinion about rollout appearance as a fact or decision.