Establish prompt engineering baseline with Gold Standard tests
- introduce Gold Standard evaluation corpus - document decision taxonomy - define prompt-engineering methodology - add regression workflow - establish Prompt Version 2 baseline - validate decision_simple, decision_deferred and decision_none
This commit is contained in:
@@ -0,0 +1,73 @@
|
||||
{
|
||||
"facts": [
|
||||
{
|
||||
"fact": "Tenant FR-7 still used the nightly CSV job yesterday.",
|
||||
"evidence": "Alex: Fine. So fact: tenant FR-7 still used the nightly CSV job yesterday."
|
||||
},
|
||||
{
|
||||
"fact": "Omar is the customer contact, not the technical owner.",
|
||||
"evidence": "Bea: Omar is the customer contact, not a participant here and not the technical owner."
|
||||
},
|
||||
{
|
||||
"fact": "Platform is the technical owner, but nobody from Platform is in the meeting.",
|
||||
"evidence": "Chen: The technical owner is still Platform, but nobody from Platform is in this call."
|
||||
},
|
||||
{
|
||||
"fact": "Friday's rollout batch still includes DE-2 and NL-4.",
|
||||
"evidence": "Alex: Good. Back to the portal rollout. Friday's batch still includes DE-2 and NL-4."
|
||||
}
|
||||
],
|
||||
"decisions": [
|
||||
{
|
||||
"decision": "FR-7 is excluded from Friday's portal rollout batch.",
|
||||
"evidence": "the portal rollout note will say FR-7 is excluded from Friday's batch. Bea: Agreed. Excluded from Friday's batch. Chen: Yes, put that in."
|
||||
}
|
||||
],
|
||||
"todos": [
|
||||
{
|
||||
"task": "Check the FR-7 mapping table.",
|
||||
"responsible": "Bea",
|
||||
"deadline": "Thursday morning",
|
||||
"evidence": "Bea: Yes, I will check it by Thursday morning."
|
||||
}
|
||||
],
|
||||
"questions": [
|
||||
{
|
||||
"question": "Can FR-7 use the v2 mapping without a customer-side field rename?",
|
||||
"evidence": "Alex: Open question: can FR-7 use the v2 mapping without a customer-side field rename?"
|
||||
},
|
||||
{
|
||||
"question": "Can Omar confirm FR-7's preferred launch window?",
|
||||
"evidence": "Dana: Also, can Omar confirm their preferred launch window?"
|
||||
}
|
||||
],
|
||||
"positions": [
|
||||
{
|
||||
"speaker": "Dana",
|
||||
"position": "Dana wants to migrate FR-7 but recognizes the mapping table may not be clean.",
|
||||
"evidence": "Dana: I want to, but we do not know if the mapping table is clean."
|
||||
},
|
||||
{
|
||||
"speaker": "Bea",
|
||||
"position": "Bea changed her earlier position and now says not to migrate FR-7 until the mapping table is checked.",
|
||||
"evidence": "I said last week we should migrate it. I am changing that. Do not migrate until the mapping table is checked."
|
||||
},
|
||||
{
|
||||
"speaker": "Dana",
|
||||
"position": "Dana thinks excluding FR-7 makes the rollout look messy.",
|
||||
"evidence": "Dana: I personally think excluding FR-7 makes the rollout look messy."
|
||||
}
|
||||
],
|
||||
"technical": [
|
||||
{
|
||||
"subject": "French export failure",
|
||||
"statement": "The old nightly CSV job failed; the new exporter was not running on tenant FR-7.",
|
||||
"evidence": "The old export failed. The new exporter was not running on that tenant."
|
||||
},
|
||||
{
|
||||
"subject": "Export mapping tables",
|
||||
"statement": "The API export uses the v2 mapping table, while the nightly CSV job uses the legacy table.",
|
||||
"evidence": "the API export uses the v2 mapping table, the nightly CSV job uses the legacy table."
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user