Add Vanheede real-world regression case
This commit is contained in:
@@ -0,0 +1,27 @@
|
||||
# Regression expectations
|
||||
|
||||
The exact prompts and compact diarized protocol input are the evaluation evidence. Test semantic fidelity and grounded attribution; do not require a byte-identical protocol or treat editorial concision as an extraction error.
|
||||
|
||||
| ID | Expectation |
|
||||
| --- | --- |
|
||||
| R1 | The resulting protocol is clearly English. |
|
||||
| R2 | The substantive discussion remains usable despite non-native English speech and the transcription imperfections visible in the evidence. |
|
||||
| R3 | The four diarized clusters are not casually collapsed or conflated. |
|
||||
| R4 | With the generation-002 mappings, Christian Psiorz, Martin Tazl, Dries Nollet, and Jorik Muylle remain attached to their explicit clusters. |
|
||||
| R5 | An unmapped cluster may be named only where the transcript gives positive, unique conversational evidence. Generation 001 supplies strong evidence for Martin, Christian, and Dries. |
|
||||
| R6 | The participant list alone must not justify a name assignment. In generation 001, the `SPEAKER_00` → Jorik Muylle assignment is not sufficiently explained by persisted evidence. |
|
||||
| R7 | Where the evidence is not unique, retain anonymity or state uncertainty; do not manufacture a person identity. |
|
||||
| R8 | Identity resolution must not turn an idea, question, offer, or reference into a personal responsibility. In particular, conditions around lab testing and discussion with polymer partners must retain their stated scope. |
|
||||
| R9 | Preserve the material technical, regulatory, commercial, and tender context: regulatory limits on external recycled content; durability/testing constraints; technical purity/process discussion; potential certified hybrid materials; and the stated tender follow-up. |
|
||||
| R10 | A human’s later editorial choice to omit information is distinct from a factual extraction error. No persisted human-edited protocol exists in this case, so this is a review rule rather than a textual golden answer. |
|
||||
|
||||
## Attribution-sensitive checks
|
||||
|
||||
- `SPEAKER_03` says that the tender will be sent; the generation-002 mapping supports Dries Nollet as the speaker. Verify the wording does not expand this to unrelated ownership.
|
||||
- Martin offers lab-scale tests **if** Vanheede provides material; preserve the conditional offer rather than a committed unconditional task.
|
||||
- The discussion says Jorik/Vanheede will discuss requirements with polymer partners in context, but a responsible person must be recorded only where the source explicitly assigns or accepts it.
|
||||
- The protocol must distinguish current constraints from forecasts about EU standards and market changes.
|
||||
|
||||
## Reference outputs
|
||||
|
||||
`protocol_generation/002/protocol.md` is the primary mapped-speaker reference. `protocol_generation/001/protocol.md` is intentionally retained as a cautionary unmapped comparison: it is useful for content review, but its name attribution cannot be used as a target wherever the transcript does not support it.
|
||||
Reference in New Issue
Block a user