Document V3 model comparison

docs/experiments.md

@@ -1568,6 +1568,65 @@ be the candidate first semantic stage. A later bounded experiment may examine
 controlled semantic interpretation, but no derivation stage, production
 integration or Progeo run was implemented here.
 
+## EXP-0030 — V3 model comparison: qwen3.5:9B vs qwen3.6:35B-A3B
+
+Date: 2026-08-19
+
+This controlled comparison reran the accepted V3 minimal semantic-preservation
+experiment unchanged with the locally installed `qwen3.6:35B-A3B`. It used the
+same implementation, A-I fixture, evidence, prompt, minimal schema, temperature
+0, `think=false`, `num_ctx=16384`, `num_predict=4096`, no retries, no voting and
+one call per case. The completed run made exactly nine calls in 96.392 seconds.
+Artifacts are preserved under
+`artifacts/experiments/evidence_observations_v3/20260819_v3_qwen36_35b_a3b_single_run/`.
+
+Structural validation passed 8/9 cases. D was semantically faithful but invalid
+because the model copied transcript speakers Antonius and Martin into
+`named_person`, although those names were not explicitly named within their
+utterances. Human semantic preservation produced 7 PASS, 1 PARTIAL and 1 FAIL:
+
+| Case | Verdict | Main result |
+| --- | --- | --- |
+| A | PASS | Preserved `can`, `perhaps`, tentative `would`, explicit `then` and no commitment, but changed the content language to English. |
+| B | PARTIAL | Preserved both approaches overall, but removed `Oder` from the 20-20 observation and locally strengthened it into collective planned conduct. |
+| C | PASS | Preserved Tim's tentative personal Textor contact and possible follow-up without established work. |
+| D | PASS | Preserved possibility, uncertainty, process and energy consequence; structural failure was confined to invalid speaker-as-named-person values. |
+| E | PASS | Preserved cost, collective rejection/non-pursuit and later determination, though the confirmation dropped explicit `Ja`. |
+| F | FAIL | Preserved quantity, timing and trial/series boundaries, but changed collective `wir` into “Martin suggests” and “Tim agrees,” inventing individual proposal/agreement meaning. |
+| G | PASS | Preserved hypothetical risk, collective recipient `wir`, Martin's personal stance, `wenn überhaupt`, conditional Technikum, impersonal `man müsste` and no decision/owner. |
+| H | PASS | Preserved request, addressee, Friday, explicit acceptance and future personal commitment without a responsibility field; content was English. |
+| I | PASS | Preserved local/pure-production and comparison boundaries, five-degree difference, upstream effort, publication purpose, unresolved permission and clarification without assignment. |
+
+Direct comparison:
+
+| Measure | `qwen3.5:9B` | `qwen3.6:35B-A3B` |
+| --- | ---: | ---: |
+| Structurally valid | 9/9 | 8/9 |
+| Human PASS | 8 | 7 |
+| Human PARTIAL | 1 | 1 |
+| Human FAIL | 0 | 1 |
+| Observations | 28 | 30 |
+| Runtime | 40.074 s | 96.392 s |
+| LLM calls | 9 | 9 |
+| Prompt-evaluation tokens | 6,268 | 6,268 |
+| Evaluation tokens | 2,590 | 2,718 |
+
+The larger model fixed the 9B weakness in G by preserving collective `wir`, and
+it split I's compound production/effort observations more cleanly. Those gains
+did not offset regressions: B was locally strengthened, F materially converted
+collective conduct into individual agreement, D violated the participant
+schema, observation count increased, and runtime was 2.4 times higher. Both
+models preserved German consistently in six of nine cases, but in different
+cases; the 35B-A3B model changed A, F and H to English, while 9B changed C, F
+and H wholly or partly to English.
+
+Result: **D — REGRESSION.** `qwen3.6:35B-A3B` does not materially improve the
+accepted minimal V3 first-stage preservation over `qwen3.5:9B`; it is worse on
+the A-I comparison because of the F ownership-adjacent strengthening and lower
+structural validity. This conclusion applies only to the minimal V3 first
+stage and does not determine model choice for any later semantic derivation.
+No production integration, derivation implementation or Progeo run occurred.
+
 ## EXP-0026 — Topic-oriented Discussion Subject reconstruction V2 prototype
 
 Date: 2026-08-11