Add controlled Whisper and LLM benchmark results
Document and preserve the controlled Progeo benchmark series. Whisper comparison: - Compare whisper.cpp large-v3 and large-v3-turbo - Identify and verify BUG-012: extraction stability depends on chunk size - Repeat both transcription variants with identical reduced chunk budgets - Select large-v3-turbo as the current production transcription model LLM comparison: - Compare qwen3.5:9b with qwen3.5:35B-A3B - Preserve identical transcript, Meeting Context, prompts and chunking - Retain qwen3.5:9b as the production recommendation - Record runtime, repair burden and semantic-quality findings Current production benchmark configuration: - whisper.cpp large-v3-turbo - target_chars=4500 - max_chars=5500 - min_chars=2500 - overlap_blocks=0 - qwen3.5:9b - num_ctx=32768 - think=false - temperature=0 BUG-012 remains verified but not yet fixed in the production chunker.
This commit is contained in:
@@ -0,0 +1,158 @@
|
||||
{
|
||||
"source_file": "progeo_whispercpp_vulkan_turbo_trimmed_converted.json",
|
||||
"chunk_count": 19,
|
||||
"chunks": [
|
||||
{
|
||||
"number": 1,
|
||||
"filename": "chunk_01.txt",
|
||||
"chars": 4547,
|
||||
"blocks": 53,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 2,
|
||||
"filename": "chunk_02.txt",
|
||||
"chars": 4529,
|
||||
"blocks": 72,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 3,
|
||||
"filename": "chunk_03.txt",
|
||||
"chars": 4590,
|
||||
"blocks": 64,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 4,
|
||||
"filename": "chunk_04.txt",
|
||||
"chars": 4665,
|
||||
"blocks": 41,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 5,
|
||||
"filename": "chunk_05.txt",
|
||||
"chars": 4556,
|
||||
"blocks": 52,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 6,
|
||||
"filename": "chunk_06.txt",
|
||||
"chars": 4509,
|
||||
"blocks": 55,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 7,
|
||||
"filename": "chunk_07.txt",
|
||||
"chars": 4550,
|
||||
"blocks": 64,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 8,
|
||||
"filename": "chunk_08.txt",
|
||||
"chars": 4513,
|
||||
"blocks": 63,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 9,
|
||||
"filename": "chunk_09.txt",
|
||||
"chars": 4540,
|
||||
"blocks": 56,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 10,
|
||||
"filename": "chunk_10.txt",
|
||||
"chars": 4549,
|
||||
"blocks": 64,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 11,
|
||||
"filename": "chunk_11.txt",
|
||||
"chars": 4529,
|
||||
"blocks": 52,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 12,
|
||||
"filename": "chunk_12.txt",
|
||||
"chars": 4503,
|
||||
"blocks": 52,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 13,
|
||||
"filename": "chunk_13.txt",
|
||||
"chars": 4542,
|
||||
"blocks": 61,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 14,
|
||||
"filename": "chunk_14.txt",
|
||||
"chars": 4528,
|
||||
"blocks": 66,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 15,
|
||||
"filename": "chunk_15.txt",
|
||||
"chars": 4583,
|
||||
"blocks": 97,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 16,
|
||||
"filename": "chunk_16.txt",
|
||||
"chars": 4543,
|
||||
"blocks": 48,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 17,
|
||||
"filename": "chunk_17.txt",
|
||||
"chars": 4585,
|
||||
"blocks": 86,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 18,
|
||||
"filename": "chunk_18.txt",
|
||||
"chars": 4504,
|
||||
"blocks": 89,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
},
|
||||
{
|
||||
"number": 19,
|
||||
"filename": "chunk_19.txt",
|
||||
"chars": 4011,
|
||||
"blocks": 76,
|
||||
"first_timestamp": null,
|
||||
"last_timestamp": null
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in New Issue
Block a user