admin
f7ad9ba51f
- introduce Gold Standard evaluation corpus - document decision taxonomy - define prompt-engineering methodology - add regression workflow - establish Prompt Version 2 baseline - validate decision_simple, decision_deferred and decision_none
1.4 KiB
1.4 KiB
Prompt Engineering Methodology
Prompt engineering uses the Gold Standard corpus as the reference. The corpus is not adjusted to make a prompt pass.
Rules:
- Only one prompt change is allowed per iteration.
- Only one gold test case may be optimized at a time.
- Every prompt modification must be validated immediately.
- A prompt modification is acceptable only if it improves the current target and does not degrade any previously passing gold test.
- Never modify
expected.jsonto make a prompt pass. - Prompt engineering edits prompt files only. Python code changes require a separate explicit task.
- Maintain a prompt evolution log for every iteration.
- If a prompt cannot improve a test after several small iterations, stop and analyze the root cause.
- Prompt changes must be generally applicable and must not special-case one transcript.
- If two consecutive prompt iterations fail to improve the current target, stop further prompt modifications and classify the root cause.
- If a prompt produces unexpected behavior, first verify whether the targeted gold test has an objectively unique ground truth.
Prompt Version 2 baseline:
decision_simple: passingdecision_deferred: passingdecision_none: passing
Prompt Version 2 adds explicit support for process decisions where the group agrees to defer a substantive decision until additional information is available.