Document target resolution V1 diagnostic
This commit is contained in:
@@ -2088,6 +2088,56 @@ derivation is justified by this result.
|
||||
Artifacts are preserved under
|
||||
`artifacts/experiments/target_resolution_v0/20260820_qwen35_9b_single_run/`.
|
||||
|
||||
## EXP-0038 — Target Resolution V1 Diagnostic
|
||||
|
||||
Status: Experimental; linkage boundary successful, normalization incomplete
|
||||
|
||||
Date: 2026-08-20
|
||||
|
||||
Forensics on failed Target Resolution V0 found a definite prompt defect: its
|
||||
illustrative value `"observation ID or null"` placed both alternatives inside
|
||||
a JSON string. V0 also sent only `format: "json"`, which enforced JSON syntax
|
||||
but not field types. This isolated diagnostic changed only the linkage/output
|
||||
boundary. It contains no rejection derivation or normative semantics.
|
||||
|
||||
Ollama 0.32.6 accepted a true JSON Schema object in `format`. TR1-V1 removed
|
||||
target selection from the model output entirely and deterministically linked
|
||||
the self-contained candidate to itself. TR2-V1 through TR4-V1 used a closed
|
||||
allowed-ID list, an enum of those IDs plus JSON null, typed positive and null
|
||||
examples, recursive strict validation, and one fixed paired prompt. Linkage and
|
||||
normalization were persisted separately.
|
||||
|
||||
| Case | Strategy / ID source | Target | Normalized target | Verdict |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| TR1-V1 | self-contained / deterministic | `obs_1` | `Mit Dr. Schlummer arbeiten wir nicht weiter.` retained negation instead of a positive action meaning | FAIL |
|
||||
| TR2-V1 | paired / LLM | `obs_1` | `externe Lösung weiterverfolgen` | PASS |
|
||||
| TR3-V1 | paired / LLM | `obs_1` | `reale Anlage zur Diskussion` lost `Druckversuch` purpose and the `nutzen` action | FAIL |
|
||||
| TR4-V1 | paired / LLM | `obs_1` | `Versuch in der realen Anlage durchführen`; Technikum excluded | PASS |
|
||||
|
||||
Result: two PASS, zero PARTIAL, two FAIL. All four responses passed their true
|
||||
JSON Schemas. Every resulting target was `obs_1`; wrong-target and
|
||||
unresolved-target counts were zero. The string `"null"` recurrence count was
|
||||
zero, and there were zero structural validation failures. TR1 preserved the
|
||||
collaboration, person, and continuation wording but failed positive-action
|
||||
normalization by retaining negation. TR3 had one material scope loss. TR4
|
||||
preserved real-plant scope and isolated the Technikum alternative. No
|
||||
normative leakage occurred.
|
||||
|
||||
Configuration: exactly four `qwen3.5:9B` calls, temperature 0,
|
||||
`think=false`, `num_ctx=16384`, `num_predict=1024`, no retries, voting, or
|
||||
prompt tuning. Aggregate runner time was 4.557 seconds.
|
||||
|
||||
Conclusion: the V0 string-null failure was primarily a linkage/output-boundary
|
||||
failure rather than evidence that observation-ID linkage is semantically
|
||||
impossible. True typed schemas, closed ID lists, and deterministic self-linkage
|
||||
eliminated every structural and target-ID failure. The experiment still fails
|
||||
its complete acceptance criterion because target normalization is not reliably
|
||||
positive or scope-preserving. These results justify separating linkage from
|
||||
normalization, but not deriving rejection or integrating a new pipeline.
|
||||
|
||||
Artifacts are preserved under
|
||||
`artifacts/experiments/target_resolution_v1_diagnostic/20260820_qwen35_9b_single_run/`.
|
||||
|
||||
## EXP-0026 — Topic-oriented Discussion Subject reconstruction V2 prototype
|
||||
|
||||
Date: 2026-08-11
|
||||
|
||||
Reference in New Issue
Block a user