Implement Semantic Consolidator V0

- add deterministic canonicalization support for extraction items
- add facts-only semantic consolidation using local Ollama
- preserve source evidence and validate complete fact coverage
- add conservative merge rules and non-LLM tests
- record the first validated real-life consolidation benchmark
- document current scope, limitations and next evaluation step
This commit is contained in:
2026-07-31 11:25:46 +02:00
parent 6e34334506
commit 90aa34d5d0
20 changed files with 5517 additions and 93 deletions
+63 -12
View File
@@ -242,10 +242,35 @@ This object feeds the Canonical Meeting Knowledge representation.
---
# Canonical Extraction Object
# Canonicalized Extractions
Planned deterministic intermediate object created from raw chunk extraction
JSON.
Implemented deterministic intermediate file created from raw chunk extraction
JSON by Canonicalizer V1. This is not Canonical Meeting Knowledge.
Top-level structure:
```json
{
"schema_version": "1",
"source_files": [],
"stats": {},
"items": []
}
```
Each item contains at least:
- item_id
- category
- text
- evidence
- source_file
- source_index
- original_value
- source_references
Action items also preserve deterministic fields such as `responsible` and
`deadline` when present.
Example:
@@ -264,16 +289,42 @@ Example:
}
```
The Deterministic Canonicalizer should create this kind of object without an
LLM. It validates and normalizes raw extraction objects, assigns stable IDs and
source references, normalizes category names and basic field structure,
performs only safe deterministic cleanup, may group exact duplicates and must
preserve all source evidence.
Canonicalizer V1 creates this kind of object without an LLM. It validates and
normalizes raw extraction objects, assigns stable IDs and source references,
normalizes category names and basic field structure, performs only safe
deterministic cleanup, may group exact duplicates and must preserve all source
evidence.
It must not perform uncertain semantic merging.
---
# Semantic Fact Group
Implemented by Semantic Consolidator V0.
Example:
```json
{
"consolidated_id": "fact_group_0001",
"category": "fact",
"canonical_text": "...",
"source_item_ids": ["fact_0001"],
"source_references": [],
"evidence": [],
"merge_reason": "Singleton; no semantically equivalent fact found."
}
```
Semantic Consolidator V0 only processes fact items. It merges semantically
equivalent facts conservatively, preserves source references and evidence, and
validates that every source fact appears exactly once. Non-fact categories are
copied unchanged. It is not a summarizer, topic grouper, protocol renderer or
Canonical Meeting Knowledge generator.
---
# Consolidated Topic
Planned semantic object produced by the Semantic Consolidator.
@@ -294,10 +345,10 @@ Example:
}
```
The Semantic Consolidator may use the local LLM to merge semantically
equivalent statements, group content by topic, preserve evidence from all
contributing chunks, mark contradictions and uncertainty and separate durable
information from transient discussion.
Future Semantic Consolidator versions may use the local LLM to merge
semantically equivalent statements beyond facts, group content by topic,
preserve evidence from all contributing chunks, mark contradictions and
uncertainty and separate durable information from transient discussion.
It produces Canonical Meeting Knowledge. It does not directly write a protocol.