Add request-acceptance gold experiment
This commit is contained in:
@@ -1680,6 +1680,73 @@ does not generalize the derivation architecture to other cases or semantic
|
||||
categories. No production integration, other case run, semantic graph,
|
||||
protocol derivation or Progeo run occurred.
|
||||
|
||||
## EXP-0032 — Request / Acceptance Gold V0
|
||||
|
||||
Status: Experimental; promising with semantic precision gaps
|
||||
|
||||
Date: 2026-08-20
|
||||
|
||||
This isolated regression experiment tested whether the EXP-0031 mechanism
|
||||
generalizes beyond H. It used ten short synthetic cases containing only
|
||||
V3-style observations. Evidence Observation V3 was neither called nor changed,
|
||||
and the model received no raw transcript or expected result. The fixed
|
||||
recognition schema permits only a nullable concrete request and nullable later
|
||||
explicit personal commitment, plus the same-requested-work judgment and
|
||||
normalized action text. Responsibility, requested actor, established status,
|
||||
Action Item, protocol, confidence and generic graph fields remain forbidden.
|
||||
|
||||
Cases:
|
||||
|
||||
- RA-01 explicit positive acceptance: PASS.
|
||||
- RA-02 paraphrased positive acceptance: PASS.
|
||||
- RA-03 acknowledgement only: PASS.
|
||||
- RA-04 tentative response: PASS.
|
||||
- RA-05 different responder without personal acceptance: PASS.
|
||||
- RA-06 explicit commitment to different work: PARTIAL. The model returned no
|
||||
acceptance instead of recognizing a commitment with `same_requested_work`
|
||||
false. The requested action correctly remained unestablished.
|
||||
- RA-07 request without response: PASS.
|
||||
- RA-08 collective commitment: PARTIAL. The model over-recognized the
|
||||
collective `wir` statement as an explicit commitment, but no request existed
|
||||
and deterministic gates prevented individual responsibility.
|
||||
- RA-09 impersonal necessity: PARTIAL. The model over-recognized the impersonal
|
||||
necessity as a concrete request, but the observation had no addressee and
|
||||
deterministic gates prevented establishment.
|
||||
- RA-10 tentative personal suggestion: PASS.
|
||||
|
||||
Configuration: exactly ten sequential `qwen3.5:9B` calls, one per case,
|
||||
temperature 0, `think=false`, `num_ctx=16384`, `num_predict=1024`, no retries,
|
||||
no voting and no prompt change between cases. Summed call time was 23.754
|
||||
seconds, with 4,826 prompt-evaluation tokens and 793 evaluation tokens. The
|
||||
strict schema validated every response and no responsibility or establishment
|
||||
field leaked into model output.
|
||||
|
||||
Both positive cases recognized the request, explicit commitment and same-work
|
||||
relationship, including the paraphrased acceptance, and deterministically
|
||||
established Clara as responsible with due date `Dienstag`. The model rendered
|
||||
the normalized action in semantically equivalent English; evaluation therefore
|
||||
checks the structural deterministic result exactly while treating normalized
|
||||
action wording as evidence-near semantic text rather than requiring lexical
|
||||
identity. Acknowledgement and tentative response were not promoted. Every
|
||||
negative case remained unestablished, and no individual responsibility was
|
||||
invented.
|
||||
|
||||
Recognition-level errors were two false positives (RA-08 commitment and RA-09
|
||||
request) and one false negative (RA-06 different-work commitment). Final
|
||||
established-action false positives and false negatives were both zero. The
|
||||
overall result was seven PASS, three PARTIAL and zero FAIL.
|
||||
|
||||
Conclusion: the narrow request-plus-acceptance architecture remains promising
|
||||
for established individual actions because deterministic addressee, ordering,
|
||||
speaker, same-work, provenance and deadline gates contained all recognition
|
||||
errors. The recognition layer is not yet precise enough to generalize: its
|
||||
handling of collective commitment, impersonal necessity and commitments to
|
||||
different work needs further isolated study. No production integration or
|
||||
additional semantic category is justified by this result.
|
||||
|
||||
Artifacts are preserved under
|
||||
`artifacts/experiments/request_acceptance_gold_v0/20260820_qwen35_9b_single_run/`.
|
||||
|
||||
## EXP-0026 — Topic-oriented Discussion Subject reconstruction V2 prototype
|
||||
|
||||
Date: 2026-08-11
|
||||
|
||||
Reference in New Issue
Block a user