Implement Semantic Consolidator V0
- add deterministic canonicalization support for extraction items - add facts-only semantic consolidation using local Ollama - preserve source evidence and validate complete fact coverage - add conservative merge rules and non-LLM tests - record the first validated real-life consolidation benchmark - document current scope, limitations and next evaluation step
This commit is contained in:
+63
-12
@@ -242,10 +242,35 @@ This object feeds the Canonical Meeting Knowledge representation.
|
||||
|
||||
---
|
||||
|
||||
# Canonical Extraction Object
|
||||
# Canonicalized Extractions
|
||||
|
||||
Planned deterministic intermediate object created from raw chunk extraction
|
||||
JSON.
|
||||
Implemented deterministic intermediate file created from raw chunk extraction
|
||||
JSON by Canonicalizer V1. This is not Canonical Meeting Knowledge.
|
||||
|
||||
Top-level structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": "1",
|
||||
"source_files": [],
|
||||
"stats": {},
|
||||
"items": []
|
||||
}
|
||||
```
|
||||
|
||||
Each item contains at least:
|
||||
|
||||
- item_id
|
||||
- category
|
||||
- text
|
||||
- evidence
|
||||
- source_file
|
||||
- source_index
|
||||
- original_value
|
||||
- source_references
|
||||
|
||||
Action items also preserve deterministic fields such as `responsible` and
|
||||
`deadline` when present.
|
||||
|
||||
Example:
|
||||
|
||||
@@ -264,16 +289,42 @@ Example:
|
||||
}
|
||||
```
|
||||
|
||||
The Deterministic Canonicalizer should create this kind of object without an
|
||||
LLM. It validates and normalizes raw extraction objects, assigns stable IDs and
|
||||
source references, normalizes category names and basic field structure,
|
||||
performs only safe deterministic cleanup, may group exact duplicates and must
|
||||
preserve all source evidence.
|
||||
Canonicalizer V1 creates this kind of object without an LLM. It validates and
|
||||
normalizes raw extraction objects, assigns stable IDs and source references,
|
||||
normalizes category names and basic field structure, performs only safe
|
||||
deterministic cleanup, may group exact duplicates and must preserve all source
|
||||
evidence.
|
||||
|
||||
It must not perform uncertain semantic merging.
|
||||
|
||||
---
|
||||
|
||||
# Semantic Fact Group
|
||||
|
||||
Implemented by Semantic Consolidator V0.
|
||||
|
||||
Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"consolidated_id": "fact_group_0001",
|
||||
"category": "fact",
|
||||
"canonical_text": "...",
|
||||
"source_item_ids": ["fact_0001"],
|
||||
"source_references": [],
|
||||
"evidence": [],
|
||||
"merge_reason": "Singleton; no semantically equivalent fact found."
|
||||
}
|
||||
```
|
||||
|
||||
Semantic Consolidator V0 only processes fact items. It merges semantically
|
||||
equivalent facts conservatively, preserves source references and evidence, and
|
||||
validates that every source fact appears exactly once. Non-fact categories are
|
||||
copied unchanged. It is not a summarizer, topic grouper, protocol renderer or
|
||||
Canonical Meeting Knowledge generator.
|
||||
|
||||
---
|
||||
|
||||
# Consolidated Topic
|
||||
|
||||
Planned semantic object produced by the Semantic Consolidator.
|
||||
@@ -294,10 +345,10 @@ Example:
|
||||
}
|
||||
```
|
||||
|
||||
The Semantic Consolidator may use the local LLM to merge semantically
|
||||
equivalent statements, group content by topic, preserve evidence from all
|
||||
contributing chunks, mark contradictions and uncertainty and separate durable
|
||||
information from transient discussion.
|
||||
Future Semantic Consolidator versions may use the local LLM to merge
|
||||
semantically equivalent statements beyond facts, group content by topic,
|
||||
preserve evidence from all contributing chunks, mark contradictions and
|
||||
uncertainty and separate durable information from transient discussion.
|
||||
|
||||
It produces Canonical Meeting Knowledge. It does not directly write a protocol.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user