meeting-lab/docs/regression-bugs.md

Regression Bug Tracker

Living tracker for real bugs discovered during end-to-end Meeting Lab evaluation.

Purpose:

• prevent forgotten regressions
• document known root causes
• record fixes when they happen
• verify that bugs never silently return

BUG-001

ID: BUG-001

Title: Renderer reverses the intended process direction

Pipeline stage: Working Protocol Renderer

Severity: High

Status: Open

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/final_protocol_with_context

Description:

The generated working protocol phrases the process direction as if the existing F&E process is broadly taken over for all project types. The intended meaning from the evaluation is more constrained: the existing F&E process is a framework that may be adapted or extended for broader project handling.

Expected behaviour:

The protocol should preserve the direction and uncertainty of the source discussion: F&E process/framework adapted for other project types where appropriate, without implying unconditional adoption for all project types.

Actual behaviour:

The protocol states that the existing F&E process is generally adopted for all project types.

Likely root cause:

Unknown.

Related files:

• samples/benchmarks/meeting_context_v1/final_protocol_with_context/working_protocol.md
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/comparison.md
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• prompts/working_protocol.md

Regression test available (yes/no): no

Current status:

Open. Documented from end-to-end evaluation output.

Notes:

Do not treat this entry as a prompt-change instruction. It records the observed bug only.

Current benchmark meeting_context_v1/e2e_current_20260803_151000 confirms the issue remains. The new protocol still says the existing F&E process is basically taken over for all project types.

BUG-002

ID: BUG-002

Title: Jovana todo is strengthened beyond the meeting content

Pipeline stage: Extraction / Working Protocol Renderer

Severity: High

Status: Open

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/final_protocol_with_context

Description:

The generated output strengthens a proposal or discussion about Jovana assembling criteria into an assigned todo. The evaluation identified this as overstating the meeting content.

Expected behaviour:

The pipeline should preserve the weaker source meaning unless the transcript explicitly assigns, accepts or confirms responsibility.

Actual behaviour:

The consolidated input and final protocol include an action item assigning criterion compilation to Jovana.

Likely root cause:

Unknown.

Related files:

• samples/benchmarks/meeting_context_v1/semantic_consolidator_repair_v1/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/working_protocol.md
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/comparison.md
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• prompts/todos.md
• prompts/working_protocol.md

Regression test available (yes/no): no

Current status:

Open. Documented from end-to-end evaluation output.

Notes:

This bug is subject to the responsibility attribution invariant in AGENTS.md.

Current benchmark meeting_context_v1/e2e_current_20260803_151000 confirms the issue remains. The consolidated output still contains Zusammenstellen der Kriterien durch Jovana with responsible: "Jovana", and the final protocol renders it as an action item.

BUG-003

ID: BUG-003

Title: Unexpected synthetic person names appear in generated protocols

Pipeline stage: Extraction / Working Protocol Renderer

Severity: Medium

Status: Root Cause Identified

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/final_protocol_with_context

Description:

The generated protocol contains unexpected or risky names/aliases such as Guido and Noah. These names were flagged during evaluation as synthetic or inconsistent in the generated protocol context.

Expected behaviour:

The protocol should use only names supported by the transcript, consolidated input or Meeting Context, and should preserve uncertainty when a name or alias is unclear.

Actual behaviour:

The final protocol includes names or name variants that were flagged as unexpected in evaluation.

Likely root cause:

The unexpected names already exist in the Whisper transcript before extraction. They are propagated through the pipeline and are not introduced by prompts, gold tests or the renderer.

Related files:

• samples/benchmarks/meeting_context_v1/final_protocol_with_context/working_protocol.md
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/comparison.md
• samples/benchmarks/meeting_context_v1/semantic_consolidator_repair_v1/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• samples/real_live/project_process_meeting/meeting_context.yaml

Regression test available (yes/no): no

Current status:

Root Cause Identified. Planned resolution is deferred until Meeting Context V2 / Entity Registry implementation.

Notes:

Resolution strategy:

Introduce an interactive entity verification step after Whisper transcription:

Whisper
    ↓
Entity Detection
    ↓
User Verification
    ↓
Meeting Context Builder
    ↓
Extraction

Expected effect:

Unknown or suspicious person names are detected before extraction begins and can be classified by the user as participant, mentioned person, external person, transcription error or ignore.

Implementation status:

Architecture accepted. Implementation deferred.

Regression test:

Not yet possible before Meeting Context V2 exists.

Current benchmark meeting_context_v1/e2e_current_20260803_151000 confirms the issue remains. Guido and Noah both appear in the generated working protocol.

BUG-004

ID: BUG-004

Title: Rhetorical question extracted as an open question

Pipeline stage: Extraction

Severity: Medium

Status: Open

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/final_protocol_with_context

Description:

A rhetorical or discussion-framing question is extracted and later rendered as an open question. This makes the protocol imply that the meeting left a real follow-up question unresolved.

Expected behaviour:

Only genuine unresolved questions should be extracted as open questions. Rhetorical questions or conversational framing should not become protocol open questions.

Actual behaviour:

The final protocol includes at least one open question identified during evaluation as rhetorical rather than genuinely open.

Likely root cause:

Unknown.

Related files:

• samples/benchmarks/meeting_context_v1/semantic_consolidator_repair_v1/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/working_protocol.md
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/comparison.md
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• prompts/questions.md
• prompts/working_protocol.md

Regression test available (yes/no): no

Current status:

Open. Documented from end-to-end evaluation output.

Notes:

No specific fix has been proposed.

Current benchmark meeting_context_v1/e2e_current_20260803_151000 confirms the issue remains. The final protocol still contains discussion-framing or underspecified questions as open protocol questions, including the question about exactly who is responsible for defining filter criteria.

BUG-005

ID: BUG-005

Title: Renderer treats Björn as absent although Meeting Context marks him present

Pipeline stage: Working Protocol Renderer

Severity: Medium

Status: Investigating

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/final_protocol_with_context

Description:

The generated protocol asks how missing participants such as Jovana and Björn should be included in the process, even though Meeting Context marks Björn as present. This creates a misleading participant-status implication.

Expected behaviour:

The protocol should not mark or imply Björn as absent when Meeting Context records him as present. If the source discussion concerns whether a person had reviewed materials or been included in a process, that should not be converted into meeting absence.

Actual behaviour:

The final protocol renders an open question about including missing participants, including Björn.

Likely root cause:

The extraction model receives Meeting Context but still classifies Björn with missing participants in chunk 08. Downstream artifacts preserve only minimal Meeting Context provenance, not structured participant attendance metadata, so Canonicalizer, Semantic Consolidator and Renderer cannot correct the conflict.

Related files:

• samples/real_live/project_process_meeting/meeting_context.yaml
• samples/benchmarks/meeting_context_v1/semantic_consolidator_repair_v1/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/working_protocol.md
• samples/benchmarks/meeting_context_v1/final_protocol_with_context/comparison.md
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• docs/bug-005-root-cause.md
• prompts/working_protocol.md

Regression test available (yes/no): no

Current status:

Investigating. Root-cause evidence has been documented in docs/bug-005-root-cause.md, but no fix has been implemented or verified. Current benchmark meeting_context_v1/e2e_current_20260803_151000 confirms the issue remains.

Notes:

This bug is distinct from responsibility attribution. It concerns participant presence/status handling.

Recommended resolution remains a Meeting Context constraint validator over structured participant metadata before rendering.

BUG-006

ID: BUG-006

Title: Renderer drops required consolidated items and changes item categories

Pipeline stage: Working Protocol Renderer

Severity: High

Status: Open

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/e2e_current_20260803_151000

Description:

The Working Protocol Renderer does not preserve every consolidated decision and open question even though the renderer prompt requires preserving all decisions, action items and open questions. It also renders some consolidated facts as decisions.

Expected behaviour:

Every consolidated decision and open question should be represented in the final protocol unless there is an explicit, validated reason to omit it. Background facts must not be promoted into decisions.

Actual behaviour:

The consolidated input contains decisions such as Festlegung eines Reporting-Zyklus für abgelehnte Projekte, Zusammengetragen und beantwortete Fragen führen zur Sitzung mit fünf Leuten..., and Die Diskussion wird beendet und die Änderungen werden verschickt...; these are not preserved as decisions in the final protocol. The consolidated input also contains open questions such as Wer soll die Rolle des Gatekeepers übernehmen... and Welche Prüfsteine sind relevant für die Fachabteilung?; these are omitted from the final protocol. Conversely, consolidated facts such as Im Zweifel gehen Projekte durch and Ein Projekt kann abgelehnt werden... are rendered as decisions.

Likely root cause:

Unknown.

Related files:

• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• prompts/working_protocol.md

Regression test available (yes/no): no

Current status:

Open. Documented from current end-to-end benchmark output.

Notes:

This is a renderer faithfulness problem independent from whether the upstream extracted items are themselves correct.

BUG-007

ID: BUG-007

Title: Non-committed discussion items are extracted and rendered as todos

Pipeline stage: Extraction / Working Protocol Renderer

Severity: High

Status: Open

Date discovered: 2026-08-03

Version first observed: meeting_context_v1/e2e_current_20260803_151000

Description:

The pipeline extracts and renders several action items that are not clearly assigned, accepted or confirmed commitments in the meeting evidence.

Expected behaviour:

Only explicit assignments, accepted responsibilities or confirmed follow-up actions should become todos. Discussion, examples, proposals, broad process needs or unclear alternatives should remain facts/questions or be marked unclear where supported.

Actual behaviour:

The current consolidated output and final protocol contain todos such as Feedback zu den Kriterien und Änderungen am Prozess abwarten, Einbinden der Fachbereiche (stellvertretend durch Guido) zur Definition von Prüfsteinen, and Mail an Jovana und Björn erneut senden oder Prozessanpassung vornehmen. The evidence for these items is discussion-level or ambiguous rather than a clear todo commitment.

Likely root cause:

Unknown.

Related files:

• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/semantic_consolidator/consolidated_extractions.json
• samples/benchmarks/meeting_context_v1/e2e_current_20260803_151000/working_protocol/working_protocol.md
• prompts/todos.md
• prompts/working_protocol.md

Regression test available (yes/no): no

Current status:

Open. Documented from current end-to-end benchmark output.

Notes:

This generalizes BUG-002 beyond the specific Jovana assignment case and should be evaluated against the responsibility attribution invariant.

BUG-008

ID: BUG-008

Title: Semantic Consolidator emits duplicate source fact coverage

Pipeline stage: Semantic Consolidator V0 / Constraint Repair

Severity: High

Status: Verified

Date discovered: 2026-08-04

Version first observed: progeo_meeting_20260804_083849

Description:

The Progeo end-to-end evaluation stopped during Semantic Consolidator V0 validation because the preserved model grouping output assigned the same source fact ID to more than one group.

Expected behaviour:

Every source fact ID must appear exactly once in the Semantic Consolidator V0 fact grouping output. The validator must reject duplicate or missing source coverage before a consolidated document is rendered.

Actual behaviour:

The validator correctly rejected the preserved raw consolidator response with:

Source item IDs appear in multiple groups: ['fact_0012']

fact_0012 appeared once in a merged group with fact_0002 and again as a singleton group. The generic validator report recorded this as one duplicate_id violation with both structural occurrences.

Recovery result:

The preserved raw consolidator response was repaired with the generic Constraint Repair V1 interface and deterministic structural operations:

• remove the repeated fact_0012 occurrence from the later singleton group
• remove the now-empty singleton group

No Semantic Consolidator LLM call was rerun. No prompts, Meeting Context, extraction outputs or canonicalizer output were modified. Second validation succeeded: every source fact ID appears exactly once, no IDs are missing, no empty groups remain, and non-fact categories remained unchanged.

This verifies the recovery path for this structural coverage failure. It does not fix or change the Semantic Consolidator generation behaviour itself.

Related files:

• samples/benchmarks/progeo_meeting_20260804_083849/semantic_consolidator/raw_model_response.txt
• samples/benchmarks/progeo_meeting_20260804_083849/semantic_consolidator/validator_before.json
• samples/benchmarks/progeo_meeting_20260804_083849/semantic_consolidator/repaired_model_groups.json
• samples/benchmarks/progeo_meeting_20260804_083849/semantic_consolidator/repaired_consolidated_extractions.json
• samples/benchmarks/progeo_meeting_20260804_083849/semantic_consolidator/validator_after.json
• samples/benchmarks/progeo_meeting_20260804_083849/semantic_consolidator/repair_metadata.json
• samples/benchmarks/progeo_meeting_20260804_083849/working_protocol/working_protocol.md
• docs/constraint-repair.md

Regression test available (yes/no): no

Current status:

Verified. The recovery path repaired and revalidated this benchmark artifact, then allowed the renderer to run once on the repaired consolidated output.

Notes:

The renderer output was produced, but it starts with explanatory prose instead of the required # Working Protocol heading. That is a renderer faithfulness issue, not part of the Semantic Consolidator coverage repair verified here.

BUG-009

ID: BUG-009

Title: Invalid responsible-party values survive canonicalization

Pipeline stage: Extraction / Deterministic Canonicalizer

Severity: High

Status: Verified

Date discovered: 2026-08-04

Version first observed: progeo_meeting_context_v1_20260804_110913

Description:

The context-aware Progeo benchmark produced action items whose responsible field contained dates or date fragments instead of responsible entities. The canonicalizer accepted these values as plain strings.

Expected behaviour:

When Meeting Context is available, action-item responsible values should be deterministically resolved to known participants, mentioned people or supported organization entities. Dates, locations, projects, products, technical terms, generic process words and unknown free text must not survive as responsible parties.

Actual behaviour:

The previous canonicalized Progeo artifact contained invalid responsible values such as:

• 31. August
• 15. oder 16. September
• am 31.
• 27.8.

Root cause:

The extraction schema allowed responsible to be a free-form string or null. Canonicalizer V1 parsed and preserved that string without checking it against Meeting Context or obvious non-person/non-organization patterns.

Fix:

Canonicalizer V1 now validates responsible fields when Meeting Context is available either through --meeting-context or extraction provenance. It:

• accepts exact participant and mentioned-person display names
• accepts participant and mentioned-person aliases
• normalizes accepted aliases to canonical display names
• accepts explicit organization/department names only when represented in the current Meeting Context structure
• rejects dates, relative dates, weekdays, clock times, locations, projects, products, systems, technical terms, generic process words and unknown free text
• clears rejected responsible values to null
• records structured responsibility_validation data and an aggregate responsibility_validations report

Verification:

The existing context-aware Progeo extraction artifacts were re-canonicalized without rerunning chunking, normalization or extraction:

• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/canonicalizer_bug009/canonicalized_extractions.json
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/canonicalizer_bug009/responsibility_demo.json

Action item count remained 28 and semantic content other than responsible fields remained unchanged. Invalid date-like responsible values were cleared. Martin Tazl normalized to Martin, and Marleen remained valid. Marleen Wever was rejected because the current Progeo Meeting Context does not list it as a display name or alias.

Related files:

• src/meeting_lab/consolidation/canonicalize.py
• tests/test_canonicalize.py
• samples/real_live/progeo_meeting/meeting_context.yaml
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/canonicalizer/canonicalized_extractions.json
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/canonicalizer_bug009/canonicalized_extractions.json
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/canonicalizer_bug009/responsibility_demo.json

Regression test available (yes/no): yes

Current status:

Verified. Focused tests pass and the Progeo downstream-only regression demo removes the observed invalid responsible values without changing action-item count or non-responsible semantic content.

Notes:

This does not prove that remaining accepted responsibilities are semantically supported by transcript evidence. It only prevents invalid entity values from surviving in the responsible field.

BUG-010

ID: BUG-010

Title: Semantic Consolidator JSON truncates on larger real-life meetings

Pipeline stage: Semantic Consolidator V0

Severity: High

Status: Fixed

Date discovered: 2026-08-04

Version first observed: progeo_meeting_context_v1_20260804_110913

Description:

The context-aware Progeo benchmark produced substantially more canonical fact items than the earlier run. Semantic Consolidator V0 used the fixed committed num_predict value of 4096 for its single grouping response and the preserved raw response stopped in the middle of a JSON group.

Expected behaviour:

The consolidator should allocate enough output budget for the expected fact-group JSON, preserve the raw model response, parse only valid JSON and validate exact source fact coverage before writing consolidated output.

Actual behaviour:

The raw response ended after fact_0060/start of the next group, while Ollama reported eval_count: 4096, exactly matching the old response cap. Parsing failed before source coverage validation:

Invalid model JSON: Expecting property name enclosed in double quotes

Root cause:

Output was truncated at the fixed num_predict cap. The prompt asks the model to emit one JSON group per fact unless duplicates are found, so output size scales with fact count and fact text size. The fixed cap was adequate for the previous smaller Progeo run but not for the context-aware run.

Fix:

Semantic Consolidator V0 now estimates the response budget from the actual fact payload and context-window headroom when --num-predict is not explicitly set. Explicit --num-predict values are still respected.

After valid model JSON is parsed, the consolidator also applies deterministic source-coverage repair before strict validation:

• duplicate source IDs after the first occurrence are removed
• empty groups created by removal are dropped
• missing source fact IDs are restored as singleton groups from the canonicalized input

This repair does not rewrite existing model group text, invent facts or create new semantic merges. Strict validation still runs after repair and can still reject the output.

Verification:

The Progeo context canonicalized benchmark was rerun through the fixed Semantic Consolidator into:

• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/semantic_consolidator_bug010_adaptive_repair/

The fixed run used num_predict: 14347, the model returned valid JSON with eval_count: 4568, deterministic repair restored seven omitted source fact IDs as singletons and final validation passed with 77 unique source fact IDs, no duplicates and no missing IDs.

Related files:

• src/meeting_lab/consolidation/consolidate_facts.py
• tests/test_consolidate_facts.py
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/semantic_consolidator/raw_model_response.txt
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/semantic_consolidator_bug010_adaptive_repair/consolidated_extractions.json
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/semantic_consolidator_bug010_adaptive_repair/repair_metadata.json
• samples/benchmarks/progeo_meeting_context_v1_20260804_110913/semantic_consolidator_bug010_adaptive_repair/report.md

Regression test available (yes/no): yes

Current status:

Fixed for the observed truncation failure and protected by focused consolidator tests. This does not improve semantic merge quality; it only prevents fixed-budget truncation and enforces complete source coverage deterministically.

BUG-011

ID: BUG-011

Title: Working Protocol Renderer V2 writes contract-invalid output

Pipeline stage: Working Protocol Renderer V2

Severity: High

Status: Verified

Date discovered: 2026-08-04

Version first observed: progeo_meeting_rc1_20260804_121502

Description:

The RC1 Progeo renderer completed its LLM call and wrote working_protocol.md, but the generated document did not start with # Working Protocol. It started with explanatory prose and used category-summary/report framing instead of the committed topic-oriented Working Protocol V2 structure.

Expected behaviour:

The renderer must preserve raw model output separately, deterministically clean only harmless wrapper text, validate the cleaned candidate against the committed Working Protocol V2 contract and write working_protocol.md only when the candidate is valid.

Actual behaviour:

The one-off renderer path used for the benchmark wrote the raw LLM text to working_protocol.md even though metadata recorded valid=false and readable_markdown=false.

Root cause:

The committed contract existed in prompts/working_protocol.md, but there was no reusable Working Protocol V2 renderer implementation with a strict output validator. The model ignored explicit prompt instructions, and the pipeline did not enforce the contract deterministically before preserving the final protocol file.

Fix:

src/meeting_lab/protocol/render_working_protocol.py now implements the Working Protocol V2 renderer path. It:

• preserves the raw Ollama JSON response and raw response text
• removes leading prose only when a real # Working Protocol heading exists
• normalizes harmless heading whitespace
• rejects output without the required heading
• rejects category-summary framing that lacks topic-oriented body structure
• rejects malformed Markdown such as unclosed fenced code blocks
• writes working_protocol.md only after validation succeeds

Verification:

Focused renderer tests cover valid output, deterministic preamble cleanup, leading whitespace cleanup, missing heading rejection, categorized-summary rejection, malformed Markdown rejection, raw response preservation and ensuring working_protocol.md contains only validated content.

RC1 downstream demo:

• Existing RC1 raw renderer output contained no embedded valid # Working Protocol body.
• The new renderer path was run once against the existing RC1 consolidated input with current committed settings.
• The raw response and cleaned candidate were preserved.
• Validation correctly rejected the output and did not write a final working_protocol.md.

Related files:

• src/meeting_lab/protocol/render_working_protocol.py
• tests/test_render_working_protocol.py
• docs/output-views.md
• samples/benchmarks/progeo_meeting_rc1_20260804_121502/working_protocol_bug011/

Regression test available (yes/no): yes

Current status:

Verified. The renderer contract is now enforced deterministically. This does not improve semantic quality of the generated prose; it prevents invalid renderer output from being accepted as a final Working Protocol.

Production-blocker follow-up, 2026-08-09:

Later preserved renderer failures showed that enforcement alone did not make a valid protocol reliably obtainable. Provenance-heavy consolidated JSON inputs were 199-350 KB, while failed Ollama runs consistently evaluated 16,386 prompt tokens. The leading handwritten renderer contract was therefore effectively lost or underweighted: models echoed trailing JSON or produced generic category summaries. Prompt and validator also duplicated the structure independently, and the validator did not enforce the prompt's no-empty-section rule.

The renderer now:

• preserves every renderable item in a compact semantic projection while removing source-reference and original-value bulk
• records and excludes structurally empty items instead of inventing content
• appends an authoritative contract generated from validator heading constants
• rejects empty emitted sections
• requires hidden, input-derived coverage markers for every decision, action item and open question exactly once in the matching section
• accepts optional background markers only for real projected input items
• uses adaptive output sizing instead of the truncating fixed 4,096-token cap
• keeps explicit output-budget overrides authoritative and makes no automatic renderer retry

Renderer-only verification used the validated consolidated input at /tmp/meeting-lab-bug014-regression/consolidated_extractions.json. The final run used qwen3.5:9B, temperature 0, num_ctx=32768, adaptive num_predict=8192, and stopped normally with eval_count=3709 and done_reason=stop after 53.069 seconds. Strict validation reported no violations and wrote:

• /tmp/meeting-lab-bug011-renderer-final/working_protocol.md

Coverage was complete for all renderable priority items: 10 decisions, 36 action items and 23 open questions, with no missing or duplicate markers. One upstream action item containing only null task/responsibility/deadline/evidence was recorded as structurally empty and not rendered. The protocol retained substantial technical background and did not add a new named responsibility; the only structured responsible person in the renderer input remained Marleen.

Status remains Verified for Working Protocol V2 structural validity and priority-item coverage. This does not claim full semantic or editorial protocol quality, topic quality, or resolution of the other renderer-related regression bugs.

BUG-012

ID: BUG-012

Title: Chunking sensitivity to Whisper segmentation density

Pipeline stage: Chunking / Extraction

Severity: High

Status: Verified hypothesis

Date discovered: 2026-08-05

Version first observed: progeo_whisper_turbo_20260805_120903

Description:

Experiment 1 compared two valid trimmed whisper.cpp transcriptions while keeping Meeting Context, source code, prompts, LLM, model parameters, pipeline stages, validation/repair behavior and renderer configuration identical.

The large-v3 transcription produced 3,822 input segments, 216,500 segment-text characters and 25 chunks. It completed extraction, canonicalization, Semantic Consolidator V0 and renderer invocation under the identical downstream settings.

The large-v3-turbo transcription produced 1,211 input segments, 83,473 segment-text characters and 10 chunks. Extraction failed at chunk_04. The preserved raw response repeated the same fact many times and truncated mid-string, producing invalid JSON. Because the extraction output for chunk_04 was invalid, canonicalization, semantic consolidation and rendering could not proceed.

Expected behaviour:

Chunk construction should provide stable extraction inputs across reasonable transcription segment-density differences. A valid, shorter Turbo transcript should not fail extraction solely because its segment boundaries create different chunk shapes.

Actual behaviour:

The pipeline was materially more compatible with the large-v3 transcript than with the Turbo transcript. The Turbo run failed before downstream stages even though the downstream settings were unchanged.

Important interpretation:

This does not prove that whisper.cpp large-v3 is intrinsically better than large-v3-turbo. It proves only that large-v3 was more compatible with the current chunking/extraction path in this controlled run.

Suspected root cause:

The current chunker treats Whisper JSON segments as atomic blocks and builds chunks by character budget around those source segments. Different Whisper segment density therefore changes the effective extraction input shape. The suspected failure mode is chunk/token budgeting and extraction stability, not merely transcription readability or Whisper quality.

Related files:

• samples/real_live/progeo_meeting/progeo_whispercpp_vulkan_large_v3_trimmed_converted.json
• samples/real_live/progeo_meeting/progeo_whispercpp_vulkan_turbo_trimmed_converted.json
• samples/benchmarks/progeo_whisper_large_v3_20260805_120903/
• samples/benchmarks/progeo_whisper_turbo_20260805_120903/
• samples/benchmarks/progeo_whisper_turbo_small_chunks_20260805_124324/
• samples/benchmarks/progeo_whisper_turbo_20260805_120903/extractions/chunk_04_extraction.raw.txt
• samples/benchmarks/progeo_whisper_comparison_20260805_120903_metrics.json
• src/meeting_lab/chunking/chunk_transcript.py

Regression test available (yes/no): no

Current status:

Verified hypothesis. Experiment 1B reran the same Turbo transcription through the same downstream settings while changing only the chunk budget from the default target_chars=9000, max_chars=11000, min_chars=5000, overlap_blocks=0 to target_chars=4500, max_chars=5500, min_chars=2500, overlap_blocks=0.

The reduced-size Turbo run produced 19 chunks instead of 10. All extraction chunks produced valid JSON, canonicalization completed, Semantic Consolidator V0 completed with existing deterministic source-coverage repair, and the renderer was invoked. Renderer contract validation still failed because the candidate did not start with # Working Protocol; that remains covered by BUG-011 and does not invalidate this chunking hypothesis.

This verifies the hypothesis for this transcript/configuration only. It does not claim the general chunker is fixed.

BUG-013

ID: BUG-013

Title: Semantic Consolidator repetitive output loop exhausts generation budget

Pipeline stage: Semantic Consolidator

Severity: High

Status: Verified

Date discovered: 2026-08-09

Version first observed: progeo_north_linux_20260809_124954

Description:

The Semantic Consolidator entered a pathological generation loop that emitted the same complete semantic group 192 consecutive times. The repeated group had the stable structural signature consisting of the same canonical_text, the same ordered source_item_ids (fact_0004, fact_0019) and the same merge_reason. Generation then truncated at an incomplete "canonical_text": field.

Expected behaviour:

The consolidator should detect a structural repeated-group loop deterministically. Invalid JSON should receive at most one controlled retry only when that loop signature is present. The retry must use the same model, temperature and context configuration and add only an instruction preventing duplicate group emission. Unrelated malformed JSON must not be retried.

Actual behaviour:

The first model response was invalid JSON at line 974, column 24, character 71184. Semantic Consolidator runtime was 277.910 seconds. The raw response had 71,184 characters (71,568 UTF-8 bytes), ended at "canonical_text":, and could not reach deterministic source-coverage repair because it was not parseable JSON.

Model metadata:

• Resolved num_predict: 19,532
• num_ctx: 32,768
• prompt_eval_count: 14,114
• eval_count: 18,654
• done_reason: length
• eval_count did not equal resolved num_predict; it exactly consumed the remaining evaluated context (32768 - 14114 = 18654)
• Complete groups before truncation: 194
• Consecutive identical groups: 192, starting at group index 2

Root cause:

The model produced a degenerate repeated semantic-group sequence until the remaining context budget was exhausted. BUG-010 adaptive response sizing was working as designed; increasing num_predict would not address this failure mode.

Implemented handling:

• Extract complete group objects from a response even when its outer JSON is truncated.
• Identify groups by normalized canonical_text, ordered source_item_ids and normalized merge_reason.
• Classify three or more consecutive identical complete groups as a loop; two identical occurrences remain an ordinary duplicate handled by deterministic coverage repair when the JSON is valid.
• Preserve first-attempt raw text, raw Ollama JSON and repetition metadata.
• Retry at most once only when JSON parsing fails and a loop is detected.
• Preserve separate retry artifacts and fail normally if retry parsing fails.

Related files:

• samples/benchmarks/progeo_north_linux_20260809_124954/
• samples/benchmarks/progeo_north_linux_20260809_124954/semantic_consolidator/raw_model_response.txt
• samples/benchmarks/progeo_north_linux_20260809_124954/semantic_consolidator/raw_ollama_response.json
• src/meeting_lab/consolidation/consolidate_facts.py
• scripts/run_meeting.py
• tests/test_consolidate_facts.py

Regression test available (yes/no): yes

Current status:

Verified for the repetitive-loop failure mode. A consolidator-only regression used the preserved canonicalized input and did not run extraction, canonicalization or rendering.

The first attempt reproduced the original failure signature: 71,184 response characters, eval_count=18654, done_reason=length, invalid JSON at character 71184, and a detected 192-group consecutive repetition run. This triggered the only permitted retry.

The retry used the same model and generation configuration, returned parseable JSON after 7,362 evaluated tokens with done_reason=stop, and contained no repetition loop (longest identical consecutive run: 1). Its source-coverage validation before repair was invalid: 105 groups, 154 observed source-ID occurrences, duplicate IDs, four missing IDs and one unknown ID. Deterministic repair made 63 changes: 44 duplicate-ID removals, 15 empty-group removals and four missing-ID singleton restorations. Validation after repair remained invalid only because group 89 contained unknown fact_0114.

The stage therefore failed cleanly after the single retry and preserved both attempts. No third LLM call occurred. The remaining unknown-ID failure is not a repetitive generation loop; its deterministic repair is tracked separately as BUG-014. This verification does not claim that general Semantic Consolidator output quality is solved.

BUG-014

ID: BUG-014

Title: Semantic Consolidator hallucinates unknown source item IDs

Pipeline stage: Semantic Consolidator deterministic coverage repair

Severity: High

Status: Verified

Date discovered: 2026-08-09

Version first observed: BUG-013 consolidator-only regression

Description:

The parseable BUG-013 retry response contained a semantic group at index 104 with source_item_ids equal to fact_0113 and unknown fact_0114. The canonicalized input contains 113 facts ending at fact_0113; no fact_0114 exists.

The group canonical text, "Ich habe ihn heute Morgen im Büro angetroffen.", is directly supported by valid fact_0113. The unknown ID adds no identifiable canonical source content and appears to be an additional hallucinated ID, not a safely correctable typo or a reference that can be mapped to another input fact.

Invariant:

Every source_item_id emitted by the Semantic Consolidator must refer to an existing canonical input fact. Strict validation must continue to reject any unknown ID that survives deterministic repair.

Deterministic repair rule:

• Remove every string source ID that is not present in the canonical fact-ID set. Never map it to a numerically nearby or semantically guessed ID.
• Preserve all valid source IDs in the group.
• Preserve the group when at least one valid source ID remains.
• Remove the group when unknown-ID removal leaves it empty.
• Apply existing duplicate-ID removal to surviving valid IDs.
• Restore genuinely missing canonical facts as singletons through the existing coverage repair.
• Preserve the original group text and merge reason; do not rewrite semantics.

Related files:

• /tmp/meeting-lab-bug013-regression/raw_model_response_retry.txt
• samples/benchmarks/progeo_north_linux_20260809_124954/canonicalizer/canonicalized_extractions.json
• src/meeting_lab/consolidation/consolidate_facts.py
• tests/test_consolidate_facts.py

Regression test available (yes/no): yes

Current status:

Verified with a consolidator-only regression using the preserved canonicalized input. Extraction, canonicalization, full benchmark orchestration and rendering were not run.

The first attempt reproduced BUG-013: eval_count=18654, done_reason=length, invalid JSON and a detected 192-group consecutive repetition run. This triggered one controlled retry. The retry returned 105 parseable groups with 154 source-ID occurrences, eval_count=7362, done_reason=stop, and no repetition loop.

Strict source-coverage inspection before repair found one unknown ID (fact_0114 in group 104), duplicate IDs and four missing canonical IDs. Deterministic repair made 64 changes:

• 44 remove_duplicate_source_id
• 1 remove_unknown_source_id
• 15 remove_empty_group
• 4 restore_missing_source_id_as_singleton

Strict validation after repair passed with 94 groups containing exactly 113 source-ID occurrences and 113 unique canonical source IDs. There were no missing IDs, unknown IDs or duplicate occurrences. BUG-014 is independent of BUG-013: BUG-013 detects invalid JSON caused by runaway repeated groups, whereas BUG-014 repairs unknown source IDs in parseable model grouping JSON.

BUG-015

ID: BUG-015

Title: Extraction classification precision for Decisions, Action Items, and Open Questions

Status: Open

First observed: Progeo production benchmark and BUG-011 renderer-only output

The Progeo extraction classified proposals/options as Decisions, suggestions and hypothetical work as Action Items, and uncertainty or discussion fragments as Open Questions. The production prompt combined a weak shared German rule, a standalone Decision prompt, a responsibility-only Action prompt, and no Open-Question definition in one multi-category extraction request.

Regression fixtures now record explicit positive and negative evidence for all three categories. Extraction uses one unified classification contract after the transcript, with precision-first thresholds, category boundaries, evidence requirements, and the responsibility-attribution invariant. In a seven-chunk Progeo measurement, bare uncertainty in chunk 08 stopped becoming an Open Question and the preference about real plant versus Technikum in chunk 16 stopped becoming a Decision. Supported examples including the explicit Dr. Schlummer rejection and established CET work remained detectable.

BUG-015 is not Verified. With qwen3.5:9B, temperature 0, the focused Action fixture still classified the unaccepted suggestion to contact Dirk Textor as an Action Item. Earlier prompt variants also showed category leakage by turning rejected candidates into Open Questions or Decisions. Prompt iterations were stopped under the Gold Standard methodology rather than adding phrase-specific filters. A general semantic classification architecture improvement remains necessary before this bug can be marked Verified.