Establish prompt engineering baseline with Gold Standard tests

- introduce Gold Standard evaluation corpus
- document decision taxonomy
- define prompt-engineering methodology
- add regression workflow
- establish Prompt Version 2 baseline
- validate decision_simple, decision_deferred and decision_none
This commit is contained in:
2026-07-30 12:13:10 +02:00
parent 07b0d80113
commit f7ad9ba51f
43 changed files with 1288 additions and 45 deletions
+11
View File
@@ -0,0 +1,11 @@
# todo_negative
Tests that vague "someone should" language is not an action item.
The transcript contains a real decision to keep the issue on the risk list, but no assigned task.
Typical LLM mistakes:
- Creating a todo with "someone" as owner.
- Assigning Paula, Ravi, or Sam even though they explicitly do not take ownership.
- Ignoring the explicit "no owner for now" correction.
+22
View File
@@ -0,0 +1,22 @@
{
"facts": [
{
"fact": "The customer export takes longer than Paula expected.",
"evidence": "Paula: The customer export takes longer than I expected."
},
{
"fact": "There is no owner for the customer export issue for now.",
"evidence": "Paula: Right, no owner for now."
}
],
"decisions": [
{
"decision": "The customer export issue will stay on the risk list for now.",
"evidence": "Sam: Then let's just keep it on the risk list. Paula: Right, no owner for now."
}
],
"todos": [],
"questions": [],
"positions": [],
"technical": []
}
+19
View File
@@ -0,0 +1,19 @@
Paula: The customer export takes longer than I expected.
Ravi: Someone should probably look at it.
Sam: Yes, maybe after the release freeze.
Paula: I don't have capacity this week.
Ravi: Same here.
Sam: Then let's just keep it on the risk list.
Paula: Right, no owner for now.
Ravi: We can revisit it in planning.
Sam: Okay.
Paula: Next item.