Implement Semantic Consolidator V0
- add deterministic canonicalization support for extraction items - add facts-only semantic consolidation using local Ollama - preserve source evidence and validate complete fact coverage - add conservative merge rules and non-LLM tests - record the first validated real-life consolidation benchmark - document current scope, limitations and next evaluation step
This commit is contained in:
@@ -195,12 +195,14 @@ Deterministic Canonicalizer:
|
||||
Semantic Consolidator:
|
||||
|
||||
- uses the local LLM
|
||||
- merges semantically equivalent statements
|
||||
- groups content by topic
|
||||
- preserves evidence from all contributing chunks
|
||||
- marks contradictions and uncertainty
|
||||
- separates durable information from transient discussion
|
||||
- produces Canonical Meeting Knowledge
|
||||
- V0 is implemented for facts-only semantic duplicate detection
|
||||
- V0 merges semantically equivalent fact items conservatively
|
||||
- V0 preserves source references and evidence
|
||||
- V0 validates that every source fact appears exactly once
|
||||
- V0 does not process non-fact categories semantically
|
||||
- later versions should group content by topic, mark contradictions and
|
||||
uncertainty, separate durable information from transient discussion and
|
||||
prepare Canonical Meeting Knowledge
|
||||
- does not directly write a protocol
|
||||
|
||||
---
|
||||
@@ -267,6 +269,7 @@ Implemented:
|
||||
- Transcript normalization
|
||||
- Technical chunk generation
|
||||
- Experimental LLM-based information extraction
|
||||
- Canonicalizer V1 deterministic extraction canonicalization
|
||||
|
||||
The current extraction step still performs multiple tasks simultaneously.
|
||||
|
||||
|
||||
+63
-12
@@ -242,10 +242,35 @@ This object feeds the Canonical Meeting Knowledge representation.
|
||||
|
||||
---
|
||||
|
||||
# Canonical Extraction Object
|
||||
# Canonicalized Extractions
|
||||
|
||||
Planned deterministic intermediate object created from raw chunk extraction
|
||||
JSON.
|
||||
Implemented deterministic intermediate file created from raw chunk extraction
|
||||
JSON by Canonicalizer V1. This is not Canonical Meeting Knowledge.
|
||||
|
||||
Top-level structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": "1",
|
||||
"source_files": [],
|
||||
"stats": {},
|
||||
"items": []
|
||||
}
|
||||
```
|
||||
|
||||
Each item contains at least:
|
||||
|
||||
- item_id
|
||||
- category
|
||||
- text
|
||||
- evidence
|
||||
- source_file
|
||||
- source_index
|
||||
- original_value
|
||||
- source_references
|
||||
|
||||
Action items also preserve deterministic fields such as `responsible` and
|
||||
`deadline` when present.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -264,16 +289,42 @@ Example:
|
||||
}
|
||||
```
|
||||
|
||||
The Deterministic Canonicalizer should create this kind of object without an
|
||||
LLM. It validates and normalizes raw extraction objects, assigns stable IDs and
|
||||
source references, normalizes category names and basic field structure,
|
||||
performs only safe deterministic cleanup, may group exact duplicates and must
|
||||
preserve all source evidence.
|
||||
Canonicalizer V1 creates this kind of object without an LLM. It validates and
|
||||
normalizes raw extraction objects, assigns stable IDs and source references,
|
||||
normalizes category names and basic field structure, performs only safe
|
||||
deterministic cleanup, may group exact duplicates and must preserve all source
|
||||
evidence.
|
||||
|
||||
It must not perform uncertain semantic merging.
|
||||
|
||||
---
|
||||
|
||||
# Semantic Fact Group
|
||||
|
||||
Implemented by Semantic Consolidator V0.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"consolidated_id": "fact_group_0001",
|
||||
"category": "fact",
|
||||
"canonical_text": "...",
|
||||
"source_item_ids": ["fact_0001"],
|
||||
"source_references": [],
|
||||
"evidence": [],
|
||||
"merge_reason": "Singleton; no semantically equivalent fact found."
|
||||
}
|
||||
```
|
||||
|
||||
Semantic Consolidator V0 only processes fact items. It merges semantically
|
||||
equivalent facts conservatively, preserves source references and evidence, and
|
||||
validates that every source fact appears exactly once. Non-fact categories are
|
||||
copied unchanged. It is not a summarizer, topic grouper, protocol renderer or
|
||||
Canonical Meeting Knowledge generator.
|
||||
|
||||
---
|
||||
|
||||
# Consolidated Topic
|
||||
|
||||
Planned semantic object produced by the Semantic Consolidator.
|
||||
@@ -294,10 +345,10 @@ Example:
|
||||
}
|
||||
```
|
||||
|
||||
The Semantic Consolidator may use the local LLM to merge semantically
|
||||
equivalent statements, group content by topic, preserve evidence from all
|
||||
contributing chunks, mark contradictions and uncertainty and separate durable
|
||||
information from transient discussion.
|
||||
Future Semantic Consolidator versions may use the local LLM to merge
|
||||
semantically equivalent statements beyond facts, group content by topic,
|
||||
preserve evidence from all contributing chunks, mark contradictions and
|
||||
uncertainty and separate durable information from transient discussion.
|
||||
|
||||
It produces Canonical Meeting Knowledge. It does not directly write a protocol.
|
||||
|
||||
|
||||
+135
-3
@@ -954,9 +954,10 @@ marks contradictions or uncertainty.
|
||||
|
||||
Decision:
|
||||
|
||||
Deterministic canonicalization is the next implementation step after stable
|
||||
local extraction, followed by semantic consolidation. Canonical Meeting
|
||||
Knowledge and final output views are planned, not implemented.
|
||||
Deterministic canonicalization is implemented as Canonicalizer V1. The first
|
||||
semantic consolidation milestone is implemented as Semantic Consolidator V0 for
|
||||
facts-only duplicate detection. Broader semantic consolidation, Canonical
|
||||
Meeting Knowledge and final output views remain planned.
|
||||
|
||||
Lessons learned:
|
||||
|
||||
@@ -1037,3 +1038,134 @@ Evidence:
|
||||
- `PROJECT_KNOWLEDGE.md`
|
||||
- `docs/output-views.md`
|
||||
- See EXP-0017 and EXP-0019.
|
||||
|
||||
## EXP-0021 - Canonicalizer V1
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Date or period: 2026-07-31
|
||||
|
||||
Hypothesis:
|
||||
|
||||
Independent chunk extraction JSON can be converted into a stable deterministic
|
||||
intermediate format before any semantic LLM consolidation is attempted.
|
||||
|
||||
Setup:
|
||||
|
||||
Canonicalizer V1 discovers `chunk_*_extraction.json` files in stable chunk
|
||||
order, validates required categories, normalizes category names and basic field
|
||||
structure, parses existing legacy string formats where safe, trims redundant
|
||||
whitespace, assigns deterministic IDs, preserves original values and source
|
||||
references, and merges only exact duplicates when all semantic fields are
|
||||
identical.
|
||||
|
||||
Inputs:
|
||||
|
||||
- Synthetic unit-test fixtures.
|
||||
- Existing nine extraction JSON files under
|
||||
`samples/whisper/meeting_speech_cleaned_chunks/`.
|
||||
|
||||
Model / configuration:
|
||||
|
||||
- No LLM.
|
||||
- CLI module: `meeting_lab.consolidation.canonicalize`.
|
||||
|
||||
Result:
|
||||
|
||||
Canonicalizer V1 produces `schema_version`, `source_files`, `stats` and
|
||||
`items`. It is deterministic preparation for the future Semantic Consolidator
|
||||
and is not Canonical Meeting Knowledge.
|
||||
|
||||
Decision:
|
||||
|
||||
Canonicalizer V1 is the current implemented deterministic canonicalization
|
||||
stage. Semantic Consolidator V0 now uses this representation for facts-only
|
||||
semantic duplicate detection; broader semantic consolidation and Canonical
|
||||
Meeting Knowledge remain planned.
|
||||
|
||||
Lessons learned:
|
||||
|
||||
Exact duplicate handling, source-reference preservation and legacy string
|
||||
parsing can be tested without model calls. Any uncertain semantic merge remains
|
||||
out of scope for this stage.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/meeting_lab/consolidation/canonicalize.py`
|
||||
- `tests/test_canonicalize.py`
|
||||
- `docs/data-models.md`
|
||||
- `docs/pipeline.md`
|
||||
|
||||
## EXP-0022 - Semantic Consolidator V0 facts-only merge
|
||||
|
||||
Status: Accepted
|
||||
|
||||
Date or period: 2026-07-31
|
||||
|
||||
Hypothesis:
|
||||
|
||||
The Canonicalizer V1 output contains enough stable structure for a local LLM to
|
||||
identify semantically equivalent fact items without losing source coverage or
|
||||
changing non-fact categories.
|
||||
|
||||
Setup:
|
||||
|
||||
Semantic Consolidator V0 consumed the existing Canonicalizer V1 benchmark JSON,
|
||||
selected only items with `category: "fact"`, and sent one bounded consolidation
|
||||
request to local Ollama. The merge rules required semantic equivalence, not
|
||||
topic similarity, and validation required every source fact ID to appear
|
||||
exactly once.
|
||||
|
||||
Inputs:
|
||||
|
||||
- `samples/benchmarks/canonicalizer_v1/canonicalized_extractions.json`
|
||||
- 33 fact items.
|
||||
|
||||
Model / configuration:
|
||||
|
||||
- `qwen3.5:9B`
|
||||
- Ollama endpoint: `http://127.0.0.1:11434/api/generate`
|
||||
- Thinking disabled.
|
||||
- One LLM call.
|
||||
- `num_ctx=32768`
|
||||
- `num_predict=4096`
|
||||
|
||||
Result:
|
||||
|
||||
- Runtime: 390.119 seconds on the current machine.
|
||||
- Merged fact groups: 1.
|
||||
- Source facts involved in merges: 2.
|
||||
- Singleton fact groups: 31.
|
||||
- Validation: passed.
|
||||
- No source fact was lost or duplicated.
|
||||
- Non-fact categories remained unchanged.
|
||||
|
||||
Accepted merge:
|
||||
|
||||
- `fact_0025` + `fact_0031`
|
||||
- Canonical statement: "Der Leiter F&E führt die Projektliste auf dem
|
||||
zweiwöchentlichen Schnittstellen-Stand-Up."
|
||||
|
||||
Decision:
|
||||
|
||||
Semantic Consolidator V0 is complete for its current narrow scope:
|
||||
conservative facts-only semantic duplicate detection with source evidence
|
||||
preserved. The selected `report.md` and `consolidated_extractions.json`
|
||||
benchmark artifacts should be versioned for later comparison. The raw model
|
||||
response remains a local diagnostic artifact and is not versioned.
|
||||
|
||||
Lessons learned:
|
||||
|
||||
Semantic duplicate consolidation is technically viable and conservative enough
|
||||
for continued evaluation, but broader semantic synthesis remains a separate
|
||||
future stage. The measured runtime is useful for this machine and run, but
|
||||
should not be generalized into a universal benchmark.
|
||||
|
||||
Evidence:
|
||||
|
||||
- `src/meeting_lab/consolidation/consolidate_facts.py`
|
||||
- `prompts/consolidate_facts.md`
|
||||
- `tests/test_consolidate_facts.py`
|
||||
- `samples/benchmarks/semantic_consolidator_v0/report.md`
|
||||
- `samples/benchmarks/semantic_consolidator_v0/consolidated_extractions.json`
|
||||
- Local diagnostic only: `samples/benchmarks/semantic_consolidator_v0/raw_model_response.txt`
|
||||
|
||||
+10
-9
@@ -51,15 +51,16 @@ technical validation.
|
||||
|
||||
The next planned architecture stage before this representation is explicit:
|
||||
|
||||
- The Deterministic Canonicalizer validates and normalizes extraction objects,
|
||||
assigns stable source references and IDs, performs only safe deterministic
|
||||
cleanup and preserves all source evidence. It uses no LLM and must not make
|
||||
uncertain semantic merges.
|
||||
- The Semantic Consolidator uses the local LLM to merge semantically equivalent
|
||||
statements, group content by topic, preserve evidence from all contributing
|
||||
chunks, mark contradictions and uncertainty, separate durable information from
|
||||
transient discussion and produce Canonical Meeting Knowledge. It does not
|
||||
directly write a protocol.
|
||||
- Canonicalizer V1 validates and normalizes extraction objects, assigns stable
|
||||
source references and IDs, performs only safe deterministic cleanup and
|
||||
preserves all source evidence. It uses no LLM and must not make uncertain
|
||||
semantic merges.
|
||||
- Semantic Consolidator V0 uses the local LLM only for facts-only semantic
|
||||
duplicate detection. It preserves source evidence and does not directly write
|
||||
a protocol or produce Canonical Meeting Knowledge.
|
||||
- Future Semantic Consolidator versions should group content by topic, mark
|
||||
contradictions and uncertainty, separate durable information from transient
|
||||
discussion and prepare Canonical Meeting Knowledge.
|
||||
|
||||
## Renderers
|
||||
|
||||
|
||||
+34
-15
@@ -135,7 +135,15 @@ Deterministic
|
||||
|
||||
## Current Status
|
||||
|
||||
Planned
|
||||
Implemented as Canonicalizer V1.
|
||||
|
||||
CLI:
|
||||
|
||||
```text
|
||||
PYTHONPATH=src .venv/bin/python -m meeting_lab.consolidation.canonicalize \
|
||||
samples/whisper/meeting_speech_cleaned_chunks \
|
||||
-o /tmp/canonicalized_extractions.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -231,7 +239,7 @@ LLM
|
||||
|
||||
## Current Status
|
||||
|
||||
Planned
|
||||
Implemented as Canonicalizer V1.
|
||||
|
||||
---
|
||||
|
||||
@@ -325,16 +333,23 @@ Canonicalized extraction objects.
|
||||
|
||||
## Output
|
||||
|
||||
Canonical Meeting Knowledge.
|
||||
Semantic Consolidator V0 output is `consolidated_extractions.json` with fact
|
||||
groups and unchanged non-fact items.
|
||||
|
||||
Future broader semantic consolidation should produce Canonical Meeting
|
||||
Knowledge.
|
||||
|
||||
## Responsibilities
|
||||
|
||||
- Merge semantically equivalent statements
|
||||
- Group content by topic
|
||||
- V0: merge semantically equivalent fact items only
|
||||
- V0: preserve all non-fact categories unchanged
|
||||
- V0: validate that every source fact ID appears exactly once
|
||||
- Future: merge semantically equivalent statements across categories
|
||||
- Future: group content by topic
|
||||
- Preserve evidence from all contributing chunks
|
||||
- Mark contradictions and uncertainty
|
||||
- Separate durable information from transient discussion
|
||||
- Reconcile category shifts where supported by evidence
|
||||
- Future: mark contradictions and uncertainty
|
||||
- Future: separate durable information from transient discussion
|
||||
- Future: reconcile category shifts where supported by evidence
|
||||
|
||||
## Must Not
|
||||
|
||||
@@ -348,7 +363,9 @@ Local LLM, with deterministic pre/post-processing where useful.
|
||||
|
||||
## Current Status
|
||||
|
||||
Planned
|
||||
Semantic Consolidator V0 is implemented and experimentally validated for
|
||||
facts-only conservative duplicate detection. Broader semantic consolidation and
|
||||
Canonical Meeting Knowledge generation remain planned.
|
||||
|
||||
---
|
||||
|
||||
@@ -507,16 +524,18 @@ A processing stage may be replaced by another implementation as long as it prese
|
||||
|
||||
⬜ Specialized Extraction
|
||||
|
||||
⬜ Deterministic Canonicalization
|
||||
✔ Deterministic Canonicalization
|
||||
|
||||
⬜ Semantic Consolidation
|
||||
✅ Semantic Consolidation V0 - facts-only duplicate detection
|
||||
|
||||
⬜ Canonical Meeting Knowledge
|
||||
|
||||
⬜ Output View Rendering
|
||||
```
|
||||
|
||||
The immediate architecture focus is the Deterministic Canonicalizer followed by
|
||||
the Semantic Consolidator. These stages preserve source evidence, recover
|
||||
global context from independent chunk extractions and prepare Canonical Meeting
|
||||
Knowledge for parallel Output View rendering.
|
||||
The immediate evaluation focus is using the Semantic Consolidator V0 output as
|
||||
input for the unchanged Working Protocol renderer. Canonicalizer V1 now
|
||||
preserves source evidence, and Semantic Consolidator V0 conservatively merges
|
||||
semantically equivalent fact items. Broader semantic consolidation should later
|
||||
recover global context from independent chunk extractions and prepare Canonical
|
||||
Meeting Knowledge for parallel Output View rendering.
|
||||
|
||||
Reference in New Issue
Block a user