Add request-acceptance gold experiment

This commit is contained in:
2026-08-20 09:13:30 +02:00
parent 19672adab4
commit a1fe89de52
5 changed files with 729 additions and 0 deletions
+67
View File
@@ -1680,6 +1680,73 @@ does not generalize the derivation architecture to other cases or semantic
categories. No production integration, other case run, semantic graph,
protocol derivation or Progeo run occurred.
## EXP-0032 — Request / Acceptance Gold V0
Status: Experimental; promising with semantic precision gaps
Date: 2026-08-20
This isolated regression experiment tested whether the EXP-0031 mechanism
generalizes beyond H. It used ten short synthetic cases containing only
V3-style observations. Evidence Observation V3 was neither called nor changed,
and the model received no raw transcript or expected result. The fixed
recognition schema permits only a nullable concrete request and nullable later
explicit personal commitment, plus the same-requested-work judgment and
normalized action text. Responsibility, requested actor, established status,
Action Item, protocol, confidence and generic graph fields remain forbidden.
Cases:
- RA-01 explicit positive acceptance: PASS.
- RA-02 paraphrased positive acceptance: PASS.
- RA-03 acknowledgement only: PASS.
- RA-04 tentative response: PASS.
- RA-05 different responder without personal acceptance: PASS.
- RA-06 explicit commitment to different work: PARTIAL. The model returned no
acceptance instead of recognizing a commitment with `same_requested_work`
false. The requested action correctly remained unestablished.
- RA-07 request without response: PASS.
- RA-08 collective commitment: PARTIAL. The model over-recognized the
collective `wir` statement as an explicit commitment, but no request existed
and deterministic gates prevented individual responsibility.
- RA-09 impersonal necessity: PARTIAL. The model over-recognized the impersonal
necessity as a concrete request, but the observation had no addressee and
deterministic gates prevented establishment.
- RA-10 tentative personal suggestion: PASS.
Configuration: exactly ten sequential `qwen3.5:9B` calls, one per case,
temperature 0, `think=false`, `num_ctx=16384`, `num_predict=1024`, no retries,
no voting and no prompt change between cases. Summed call time was 23.754
seconds, with 4,826 prompt-evaluation tokens and 793 evaluation tokens. The
strict schema validated every response and no responsibility or establishment
field leaked into model output.
Both positive cases recognized the request, explicit commitment and same-work
relationship, including the paraphrased acceptance, and deterministically
established Clara as responsible with due date `Dienstag`. The model rendered
the normalized action in semantically equivalent English; evaluation therefore
checks the structural deterministic result exactly while treating normalized
action wording as evidence-near semantic text rather than requiring lexical
identity. Acknowledgement and tentative response were not promoted. Every
negative case remained unestablished, and no individual responsibility was
invented.
Recognition-level errors were two false positives (RA-08 commitment and RA-09
request) and one false negative (RA-06 different-work commitment). Final
established-action false positives and false negatives were both zero. The
overall result was seven PASS, three PARTIAL and zero FAIL.
Conclusion: the narrow request-plus-acceptance architecture remains promising
for established individual actions because deterministic addressee, ordering,
speaker, same-work, provenance and deadline gates contained all recognition
errors. The recognition layer is not yet precise enough to generalize: its
handling of collective commitment, impersonal necessity and commitments to
different work needs further isolated study. No production integration or
additional semantic category is justified by this result.
Artifacts are preserved under
`artifacts/experiments/request_acceptance_gold_v0/20260820_qwen35_9b_single_run/`.
## EXP-0026 — Topic-oriented Discussion Subject reconstruction V2 prototype
Date: 2026-08-11