Document failed target resolution V0 experiment
This commit is contained in:
@@ -2024,6 +2024,70 @@ ineligible CR-06 path; it is not reliable enough for rejection derivation.
|
||||
Artifacts are preserved under
|
||||
`artifacts/experiments/controlled_rejection_v1/20260820_qwen35_9b_single_run/`.
|
||||
|
||||
## EXP-0037 — Target Resolution V0
|
||||
|
||||
Status: Experimental; FAILED for target resolution
|
||||
|
||||
Date: 2026-08-20
|
||||
|
||||
Controlled Rejection V1 showed that fine-grained Negative Act Form eligibility
|
||||
contained false-positive rejection, but its target strategy failed a
|
||||
self-contained positive and unnecessarily resolved a target for an ineligible
|
||||
`none` form. This isolated experiment tested target resolution only. It
|
||||
contains no rejection derivation, status, decision, outcome, responsibility,
|
||||
topic closure, or production integration.
|
||||
|
||||
Eligibility was deterministic: only `explicit_non_pursuit` could reach the
|
||||
resolver. TR-05 personal preference, TR-06 recommendation, TR-07 temporary
|
||||
non-action, and TR-08 `none` stopped before prompt construction and recorded an
|
||||
explicit skipped-call artifact. This hard gate worked in all four cases.
|
||||
|
||||
The target schema contained exactly `candidate_observation_id`,
|
||||
`target_observation_id`, and `normalized_target_text`, with local IDs,
|
||||
same-or-earlier ordering, unique evidence provenance, null consistency, and
|
||||
recursive normative-field exclusion. TR-01 used the self-contained strategy:
|
||||
the prompt stated that linkage was deterministically fixed to the candidate and
|
||||
requested semantic normalization only. TR-02 through TR-04 used paired local
|
||||
resolution. No original transcript or new Negative Act classification call was
|
||||
used.
|
||||
|
||||
| Case | Form / eligible | Call | Target result | Verdict |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| TR-01 | `explicit_non_pursuit` / yes | yes | Returned string `"null"`; required same-observation target unresolved | FAIL |
|
||||
| TR-02 | `explicit_non_pursuit` / yes | yes | Returned string `"null"`; `obs_1` unresolved | FAIL |
|
||||
| TR-03 | `explicit_non_pursuit` / yes | yes | Returned string `"null"`; scoped `obs_1` unresolved | FAIL |
|
||||
| TR-04 | `explicit_non_pursuit` / yes | yes | Returned string `"null"`; real-plant target unresolved | FAIL |
|
||||
| TR-05 | `personal_preference` / no | no | Deterministically skipped | PASS |
|
||||
| TR-06 | `recommendation` / no | no | Deterministically skipped | PASS |
|
||||
| TR-07 | `temporary_non_action` / no | no | Deterministically skipped | PASS |
|
||||
| TR-08 | `none` / no | no | Deterministically skipped | PASS |
|
||||
|
||||
Result: four PASS, zero PARTIAL, four FAIL. Exactly four successful Ollama
|
||||
calls were made, all for eligible cases; there were zero technical call
|
||||
failures and four structural validation failures. All four raw responses used
|
||||
the JSON string `"null"` as target ID rather than a supplied observation ID or
|
||||
JSON null. Wrong-target count and unresolved-target count were therefore four.
|
||||
No qualifier-preservation claim can be made because no eligible positive target
|
||||
passed validation. TR-04 alternative isolation likewise could not be
|
||||
established. No rejection, status, decision, outcome, responsibility, or other
|
||||
normative leakage occurred, and no rejection derivation was performed.
|
||||
|
||||
Configuration: `qwen3.5:9B`, temperature 0, `think=false`,
|
||||
`num_ctx=16384`, `num_predict=1024`, no retries, voting, or prompt changes.
|
||||
Aggregate runner time was 4.034 seconds.
|
||||
|
||||
Conclusion: eligibility gating is successful and should be retained; it fully
|
||||
prevents unnecessary target calls for ineligible Negative Act forms. Target
|
||||
Resolution V0 itself failed structurally across all eligible cases. Neither the
|
||||
self-contained nor paired strategy produced a valid target, and merely
|
||||
instructing deterministic self-linkage in the semantic prompt did not make the
|
||||
linkage structurally deterministic. The repeated `"null"` string pattern
|
||||
requires diagnosis before changing the architecture or prompt. No rejection
|
||||
derivation is justified by this result.
|
||||
|
||||
Artifacts are preserved under
|
||||
`artifacts/experiments/target_resolution_v0/20260820_qwen35_9b_single_run/`.
|
||||
|
||||
## EXP-0026 — Topic-oriented Discussion Subject reconstruction V2 prototype
|
||||
|
||||
Date: 2026-08-11
|
||||
|
||||
Reference in New Issue
Block a user