Document target resolution V1 diagnostic

This commit is contained in:
2026-08-20 14:21:39 +02:00
parent 3229786b5c
commit 7fa771a7e4
5 changed files with 232 additions and 0 deletions
+50
View File
@@ -2088,6 +2088,56 @@ derivation is justified by this result.
Artifacts are preserved under
`artifacts/experiments/target_resolution_v0/20260820_qwen35_9b_single_run/`.
## EXP-0038 — Target Resolution V1 Diagnostic
Status: Experimental; linkage boundary successful, normalization incomplete
Date: 2026-08-20
Forensics on failed Target Resolution V0 found a definite prompt defect: its
illustrative value `"observation ID or null"` placed both alternatives inside
a JSON string. V0 also sent only `format: "json"`, which enforced JSON syntax
but not field types. This isolated diagnostic changed only the linkage/output
boundary. It contains no rejection derivation or normative semantics.
Ollama 0.32.6 accepted a true JSON Schema object in `format`. TR1-V1 removed
target selection from the model output entirely and deterministically linked
the self-contained candidate to itself. TR2-V1 through TR4-V1 used a closed
allowed-ID list, an enum of those IDs plus JSON null, typed positive and null
examples, recursive strict validation, and one fixed paired prompt. Linkage and
normalization were persisted separately.
| Case | Strategy / ID source | Target | Normalized target | Verdict |
| --- | --- | --- | --- | --- |
| TR1-V1 | self-contained / deterministic | `obs_1` | `Mit Dr. Schlummer arbeiten wir nicht weiter.` retained negation instead of a positive action meaning | FAIL |
| TR2-V1 | paired / LLM | `obs_1` | `externe Lösung weiterverfolgen` | PASS |
| TR3-V1 | paired / LLM | `obs_1` | `reale Anlage zur Diskussion` lost `Druckversuch` purpose and the `nutzen` action | FAIL |
| TR4-V1 | paired / LLM | `obs_1` | `Versuch in der realen Anlage durchführen`; Technikum excluded | PASS |
Result: two PASS, zero PARTIAL, two FAIL. All four responses passed their true
JSON Schemas. Every resulting target was `obs_1`; wrong-target and
unresolved-target counts were zero. The string `"null"` recurrence count was
zero, and there were zero structural validation failures. TR1 preserved the
collaboration, person, and continuation wording but failed positive-action
normalization by retaining negation. TR3 had one material scope loss. TR4
preserved real-plant scope and isolated the Technikum alternative. No
normative leakage occurred.
Configuration: exactly four `qwen3.5:9B` calls, temperature 0,
`think=false`, `num_ctx=16384`, `num_predict=1024`, no retries, voting, or
prompt tuning. Aggregate runner time was 4.557 seconds.
Conclusion: the V0 string-null failure was primarily a linkage/output-boundary
failure rather than evidence that observation-ID linkage is semantically
impossible. True typed schemas, closed ID lists, and deterministic self-linkage
eliminated every structural and target-ID failure. The experiment still fails
its complete acceptance criterion because target normalization is not reliably
positive or scope-preserving. These results justify separating linkage from
normalization, but not deriving rejection or integrating a new pipeline.
Artifacts are preserved under
`artifacts/experiments/target_resolution_v1_diagnostic/20260820_qwen35_9b_single_run/`.
## EXP-0026 — Topic-oriented Discussion Subject reconstruction V2 prototype
Date: 2026-08-11