Establish prompt engineering baseline with Gold Standard tests
- introduce Gold Standard evaluation corpus - document decision taxonomy - define prompt-engineering methodology - add regression workflow - establish Prompt Version 2 baseline - validate decision_simple, decision_deferred and decision_none
This commit is contained in:
@@ -0,0 +1,42 @@
|
||||
{
|
||||
"facts": [
|
||||
{
|
||||
"fact": "The pilot survey closed yesterday with 42 responses.",
|
||||
"evidence": "Hannah: The pilot survey closed yesterday with 42 responses."
|
||||
},
|
||||
{
|
||||
"fact": "The average pilot survey rating was 3.8 out of 5.",
|
||||
"evidence": "Leo: The average rating was 3.8 out of 5."
|
||||
},
|
||||
{
|
||||
"fact": "Northwind, Verdan, and Eastport did not respond to the survey.",
|
||||
"evidence": "Hannah: That part is true. Northwind, Verdan, and Eastport did not respond."
|
||||
}
|
||||
],
|
||||
"decisions": [],
|
||||
"todos": [],
|
||||
"questions": [
|
||||
{
|
||||
"question": "Why does Marta think the survey result is weaker than it looks?",
|
||||
"evidence": "Leo: Why?"
|
||||
}
|
||||
],
|
||||
"positions": [
|
||||
{
|
||||
"speaker": "Marta",
|
||||
"position": "Marta thinks the pilot survey result is weaker than it looks.",
|
||||
"evidence": "Marta: I think that is weaker than it looks."
|
||||
},
|
||||
{
|
||||
"speaker": "Leo",
|
||||
"position": "Leo feels the pilot is healthy.",
|
||||
"evidence": "Leo: I still feel the pilot is healthy."
|
||||
},
|
||||
{
|
||||
"speaker": "Marta",
|
||||
"position": "Marta thinks the rollout should slow down.",
|
||||
"evidence": "Marta: I disagree. My view is that we should slow down the rollout."
|
||||
}
|
||||
],
|
||||
"technical": []
|
||||
}
|
||||
Reference in New Issue
Block a user