Add controlled Whisper and LLM benchmark results

Browse Source

Document and preserve the controlled Progeo benchmark series.

Whisper comparison:
- Compare whisper.cpp large-v3 and large-v3-turbo
- Identify and verify BUG-012: extraction stability depends on chunk size
- Repeat both transcription variants with identical reduced chunk budgets
- Select large-v3-turbo as the current production transcription model

LLM comparison:
- Compare qwen3.5:9b with qwen3.5:35B-A3B
- Preserve identical transcript, Meeting Context, prompts and chunking
- Retain qwen3.5:9b as the production recommendation
- Record runtime, repair burden and semantic-quality findings

Current production benchmark configuration:
- whisper.cpp large-v3-turbo
- target_chars=4500
- max_chars=5500
- min_chars=2500
- overlap_blocks=0
- qwen3.5:9b
- num_ctx=32768
- think=false
- temperature=0

BUG-012 remains verified but not yet fixed in the production chunker.

...

This commit is contained in:

admin

2026-08-05 14:19:40 +02:00

parent dcc3a5c734

commit 03bc6b1d90

544 changed files with 464774 additions and 0 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

samples/benchmarks/progeo_llm_comparison_20260805_133942_metrics.json

+2376

File diff suppressed because one or more lines are too long