11 changed files with 453 additions and 14 deletions
+21
View File
@@ -1,5 +1,18 @@
# Project Knowledge # Project Knowledge
## Meeting language in direct protocols
Direct protocol prompts derive their explicit output language from persisted
Meeting Context `meeting.language`, including regeneration and diarized fallback.
Missing context/language explicitly defaults to `de`; omitted language is accepted
without modifying context data. Protocol runtime `output_language` is derived
provenance, not a separate setting. No transcript/context translation is performed.
Prompt evolution (2026-09-11): replaced unconditional German output with the
meeting-language instruction in the shared prompt builder. Names remain verbatim.
Validation uses mocked generation for German/English, plain/diarized inputs,
regeneration and legacy contexts; no live model or extraction gold run is involved.
This is a compact operational summary of the current Meeting Lab state. This is a compact operational summary of the current Meeting Lab state.
## Objective ## Objective
@@ -39,6 +52,14 @@ Implemented:
configurable safe input budget, falls back to complete plain transcript text configurable safe input budget, falls back to complete plain transcript text
when necessary, and fails before any Ollama request if even that input is too when necessary, and fails before any Ollama request if even that input is too
large. Silent head/tail truncation is prohibited. large. Silent head/tail truncation is prohibited.
- The `qwen3.8:27b` direct-protocol stage explicitly requests `num_ctx=32768`
and `think=false`; the practical prompt target is approximately 29,000 tokens.
A 31,038-token synthetic prompt passed, but larger prompts are not assumed safe
from the model's advertised 262,144-token native context alone.
- `regenerate_mvp_protocol` updates the run's validated Meeting Context and
regenerates protocol artifacts from the existing diarized transcript when
available. It never reruns audio preparation, Whisper or Pyannote, and it
preserves anonymous speaker labels in the source transcript.
- Non-LLM unit tests for chunking, extraction helpers, protocol rendering and - Non-LLM unit tests for chunking, extraction helpers, protocol rendering and
gold-test runner validation. gold-test runner validation.
- Meeting Context V1 scaffold and documentation for manually maintained - Meeting Context V1 scaffold and documentation for manually maintained
+7
View File
@@ -1,5 +1,12 @@
# Meeting Lab # Meeting Lab
The direct protocol uses saved Meeting Context `meeting.language`: `de` requests
German output and `en` requests English output, for plain and diarized transcripts
and protocol-only regeneration. Meeting Assistant supplies the same selection to
Whisper. Missing context/language retains German output for older runs. Effective
output language is recorded as `output_language` in protocol runtime metadata.
Transcript text, authored context, names and speaker mappings are not translated.
Experimentierumgebung zur Entwicklung eines lokalen Diskussionsanalyzers für Meetingtranskripte. Experimentierumgebung zur Entwicklung eines lokalen Diskussionsanalyzers für Meetingtranskripte.
## Ziel ## Ziel
+3 -3
View File
@@ -48,9 +48,9 @@ derived representation sent to prompt construction.
Before contacting Ollama, Meeting Lab conservatively estimates prompt tokens Before contacting Ollama, Meeting Lab conservatively estimates prompt tokens
from UTF-8 byte count without adding a model tokenizer dependency. The default safe from UTF-8 byte count without adding a model tokenizer dependency. The default safe
budget is 16,200 estimated tokens, below the observed 16,386-token effective budget is 29,000 estimated tokens within the explicitly configured 32,768-token
boundary even when a larger `num_ctx` was requested. The estimate is calibrated Ollama context. The estimate is calibrated against the currently validated
against the currently validated German BPD input and is configurable through German BPD input and is configurable through
`MvpMeetingConfig.protocol_safe_input_token_budget` or `MvpMeetingConfig.protocol_safe_input_token_budget` or
`--protocol-safe-input-token-budget`. `--protocol-safe-input-token-budget`.
@@ -108,6 +108,8 @@ def validate_meeting_context(data: dict[str, Any]) -> None:
meeting = _require_mapping(data, "meeting") meeting = _require_mapping(data, "meeting")
_require_non_empty_string(meeting, "meeting.meeting_id") _require_non_empty_string(meeting, "meeting.meeting_id")
_require_non_empty_string(meeting, "meeting.title") _require_non_empty_string(meeting, "meeting.title")
# Legacy contexts may omit language; protocol generation defaults to German.
if "language" in meeting:
_require_non_empty_string(meeting, "meeting.language") _require_non_empty_string(meeting, "meeting.language")
organization = _optional_mapping(data.get("organization"), "organization") organization = _optional_mapping(data.get("organization"), "organization")
+63
View File
@@ -30,6 +30,7 @@ from src.meeting_lab.models.meeting_context import (
from src.meeting_lab.progress import ProgressEvent, ProgressSink, ProgressStatus from src.meeting_lab.progress import ProgressEvent, ProgressSink, ProgressStatus
from src.meeting_lab.protocol.generate_direct_protocol import ( from src.meeting_lab.protocol.generate_direct_protocol import (
DEFAULT_MODEL, DEFAULT_MODEL,
DEFAULT_NUM_CTX,
DEFAULT_SAFE_INPUT_TOKEN_BUDGET, DEFAULT_SAFE_INPUT_TOKEN_BUDGET,
DirectProtocolResult, DirectProtocolResult,
generate_direct_protocol, generate_direct_protocol,
@@ -55,6 +56,7 @@ class MvpMeetingConfig:
threads: str | int = "auto" threads: str | int = "auto"
model: str = DEFAULT_MODEL model: str = DEFAULT_MODEL
ollama_endpoint: str = DEFAULT_ENDPOINT ollama_endpoint: str = DEFAULT_ENDPOINT
protocol_num_ctx: int = DEFAULT_NUM_CTX
protocol_safe_input_token_budget: int = DEFAULT_SAFE_INPUT_TOKEN_BUDGET protocol_safe_input_token_budget: int = DEFAULT_SAFE_INPUT_TOKEN_BUDGET
diarization: str = "off" diarization: str = "off"
diarization_runtime: str = "native" diarization_runtime: str = "native"
@@ -69,6 +71,64 @@ class MvpRunResult:
protocol_path: Path | None protocol_path: Path | None
def regenerate_mvp_protocol(
run_dir: Path,
*,
meeting_context: ContextInput,
model: str = DEFAULT_MODEL,
ollama_endpoint: str = DEFAULT_ENDPOINT,
protocol_num_ctx: int = DEFAULT_NUM_CTX,
protocol_safe_input_token_budget: int = DEFAULT_SAFE_INPUT_TOKEN_BUDGET,
progress_sink: ProgressSink | None = None,
) -> MvpRunResult:
"""Regenerate only protocol artifacts from an existing completed run."""
started = time.perf_counter()
run_dir = Path(run_dir)
context = _effective_context(meeting_context)
if context is None:
raise ValueError("Meeting Context is required for protocol regeneration.")
if protocol_num_ctx <= 0:
raise ValueError("Protocol Ollama context size must be positive.")
if protocol_safe_input_token_budget <= 0:
raise ValueError("Protocol safe input token budget must be positive.")
diarized_transcript = run_dir / "diarization" / "transcript_diarized.json"
plain_transcript = run_dir / "transcript" / "transcript.json"
transcript_path = (
diarized_transcript if diarized_transcript.is_file() else plain_transcript
)
if not transcript_path.is_file():
raise FileNotFoundError(
f"Existing run has no protocol transcript artifact: {run_dir}"
)
context_path = run_dir / "context" / "meeting_context.yaml"
write_meeting_context(context, context_path)
_emit(progress_sink, "protocol_generation", "started", started)
try:
result = generate_direct_protocol(
transcript_path,
context_path,
model=model,
endpoint=ollama_endpoint,
num_ctx=protocol_num_ctx,
safe_input_token_budget=protocol_safe_input_token_budget,
)
protocol_path = _persist_protocol(run_dir, result)
except Exception as exc:
_emit(
progress_sink,
"failed",
"failed",
started,
message=f"protocol_generation: {type(exc).__name__}: {exc}",
)
raise
_emit(progress_sink, "protocol_generation", "completed", started)
_emit(progress_sink, "completed", "completed", started)
return MvpRunResult(0, run_dir, protocol_path)
def create_unique_run_dir( def create_unique_run_dir(
output_root: Path, output_root: Path,
meeting_name: str, meeting_name: str,
@@ -131,6 +191,8 @@ def _validate_inputs(
raise ValueError("A diarization container image is required.") raise ValueError("A diarization container image is required.")
if config.protocol_safe_input_token_budget <= 0: if config.protocol_safe_input_token_budget <= 0:
raise ValueError("Protocol safe input token budget must be positive.") raise ValueError("Protocol safe input token budget must be positive.")
if config.protocol_num_ctx <= 0:
raise ValueError("Protocol Ollama context size must be positive.")
def _emit( def _emit(
@@ -363,6 +425,7 @@ def run_mvp_meeting(
preserved_context, preserved_context,
model=config.model, model=config.model,
endpoint=config.ollama_endpoint, endpoint=config.ollama_endpoint,
num_ctx=config.protocol_num_ctx,
safe_input_token_budget=config.protocol_safe_input_token_budget, safe_input_token_budget=config.protocol_safe_input_token_budget,
) )
stage_runtimes["protocol"] = round(time.perf_counter() - stage_started, 3) stage_runtimes["protocol"] = round(time.perf_counter() - stage_started, 3)
@@ -3,28 +3,34 @@
from __future__ import annotations from __future__ import annotations
DIRECT_PROTOCOL_INSTRUCTION = """Erstelle aus dem vollständigen Transkript und dem Meeting-Kontext ein vollständiges, strukturiertes und professionelles internes Besprechungsprotokoll in deutscher Sprache. DIRECT_PROTOCOL_INSTRUCTION = """Erstelle aus dem vollständigen Transkript und dem Meeting-Kontext ein vollständiges, strukturiertes und professionelles internes Besprechungsprotokoll.
Das Protokoll muss themenorientiert sein, nicht chronologisch und nicht nach technischen Kategorien gegliedert. Beginne mit # Meeting Protocol. Verwende für jedes kohärente Thema eine Überschrift ## <Thema> und darunter eine strukturierte Synthese der Diskussion. Bewahre relevante Diskussionsverläufe, unterschiedliche Positionen, offene Punkte und Entscheidungsgrundlagen. Dokumentiere die wesentlichen Inhalte nachvollziehbar und fasse Themenblöcke so zusammen, dass auch Personen, die nicht am Meeting teilgenommen haben, den Kontext und die Entwicklung der Diskussion verstehen können. Nenne Entscheidungen oder abgestimmte Positionen nur, wenn sie tatsächlich belegt sind. Führe Maßnahmen nur auf, wenn eine konkrete zukünftige Handlung gestützt ist; nenne verantwortliche Personen und Fristen ausschließlich bei expliziter Zuweisung, Annahme oder Bestätigung im Transkript. Vorschläge, Einwände, Möglichkeiten und vorläufige Ideen sind keine Entscheidungen oder Verpflichtungen. Bewahre relevante Einschränkungen und ungelöste Meinungsverschiedenheiten. Nenne offene Punkte nur, wenn sie wirklich offen bleiben. Nicht jedes Thema benötigt Entscheidungen, Maßnahmen oder offene Punkte. Das Protokoll muss themenorientiert sein, nicht chronologisch und nicht nach technischen Kategorien gegliedert. Beginne mit # Meeting Protocol. Verwende für jedes kohärente Thema eine Überschrift ## <Thema> und darunter eine strukturierte Synthese der Diskussion. Bewahre relevante Diskussionsverläufe, unterschiedliche Positionen, offene Punkte und Entscheidungsgrundlagen. Dokumentiere die wesentlichen Inhalte nachvollziehbar und fasse Themenblöcke so zusammen, dass auch Personen, die nicht am Meeting teilgenommen haben, den Kontext und die Entwicklung der Diskussion verstehen können. Nenne Entscheidungen oder abgestimmte Positionen nur, wenn sie tatsächlich belegt sind. Führe Maßnahmen nur auf, wenn eine konkrete zukünftige Handlung gestützt ist; nenne verantwortliche Personen und Fristen ausschließlich bei expliziter Zuweisung, Annahme oder Bestätigung im Transkript. Vorschläge, Einwände, Möglichkeiten und vorläufige Ideen sind keine Entscheidungen oder Verpflichtungen. Bewahre relevante Einschränkungen und ungelöste Meinungsverschiedenheiten. Nenne offene Punkte nur, wenn sie wirklich offen bleiben. Nicht jedes Thema benötigt Entscheidungen, Maßnahmen oder offene Punkte.
Erzeuge keine reine Wiedergabe des Transkripts und verlängere das Protokoll nicht unnötig durch Wiederholungen. Synthetisiere zusammengehörige Aussagen, entferne Füllwörter und Gesprächsrauschen und erfinde keine Fakten, Entscheidungen, Zustimmungen, Verantwortlichen oder Fristen. Gib kein JSON, keine internen Labels und keine Analyse oder Denkprotokolle aus. Das Ergebnis soll als Markdown-Protokoll nach geringfügiger menschlicher Redaktion intern versendbar sein. Eine kompakte themenübergreifende Maßnahmenliste am Ende ist optional, wenn sie nützlich und vollständig belegt ist.""" Erzeuge keine reine Wiedergabe des Transkripts und verlängere das Protokoll nicht unnötig durch Wiederholungen. Synthetisiere zusammengehörige Aussagen, entferne Füllwörter und Gesprächsrauschen und erfinde keine Fakten, Entscheidungen, Zustimmungen, Verantwortlichen oder Fristen. Gib kein JSON, keine internen Labels und keine Analyse oder Denkprotokolle aus. Das Ergebnis soll als Markdown-Protokoll nach geringfügiger menschlicher Redaktion intern versendbar sein. Eine kompakte themenübergreifende Maßnahmenliste am Ende ist optional, wenn sie nützlich und vollständig belegt ist."""
COMPACT_DIARIZED_PROTOCOL_INSTRUCTION = """Erstelle aus dem vollständigen Transkript und Meeting-Kontext ein vollständiges, professionelles internes Besprechungsprotokoll auf Deutsch. Das Transkript ist in aufeinanderfolgende anonyme Sprecherblöcke gegliedert. COMPACT_DIARIZED_PROTOCOL_INSTRUCTION = """Erstelle aus dem vollständigen Transkript und Meeting-Kontext ein vollständiges, professionelles internes Besprechungsprotokoll. Das Transkript ist in aufeinanderfolgende anonyme Sprecherblöcke gegliedert.
Beginne mit # Meeting Protocol. Gliedere themenorientiert mit ## <Thema> und synthetisiere je Thema den relevanten Diskussionsverlauf, Kontext, unterschiedliche Positionen, Entscheidungsgrundlagen, Einschränkungen und ungelöste Meinungsverschiedenheiten so, dass Dritte ihn nachvollziehen können. Nenne Entscheidungen nur bei Beleg. Nenne Maßnahmen, Verantwortliche und Fristen nur bei expliziter Zuweisung, Annahme oder Bestätigung; Vorschläge sind keine Verpflichtungen. Beginne mit # Meeting Protocol. Gliedere themenorientiert mit ## <Thema> und synthetisiere je Thema den relevanten Diskussionsverlauf, Kontext, unterschiedliche Positionen, Entscheidungsgrundlagen, Einschränkungen und ungelöste Meinungsverschiedenheiten so, dass Dritte ihn nachvollziehen können. Nenne Entscheidungen nur bei Beleg. Nenne Maßnahmen, Verantwortliche und Fristen nur bei expliziter Zuweisung, Annahme oder Bestätigung; Vorschläge sind keine Verpflichtungen.
Entferne nur Wiederholungen, Füllwörter und Gesprächsrauschen. Erfinde keine Fakten oder Identitäten. Gib kein JSON, keine Sprecherlabels und kein Denkprotokoll aus. Eine belegte themenübergreifende Maßnahmenliste am Ende ist optional.""" Entferne nur Wiederholungen, Füllwörter und Gesprächsrauschen. Erfinde keine Fakten oder Identitäten. Gib kein JSON, keine Sprecherlabels und kein Denkprotokoll aus. Eine belegte themenübergreifende Maßnahmenliste am Ende ist optional."""
MAPPED_SPEAKER_ATTRIBUTION_INSTRUCTION = """Nutze die autoritativen SPEAKER_XX-zu-Teilnehmer-Zuordnungen im Meeting-Kontext, um ausdrücklich belegte Aussagen, Positionen, Entscheidungen, Zuweisungen und angenommene persönliche Verpflichtungen namentlich zuzuordnen. Eine ausdrückliche Ich-Zusage eines zugeordneten Sprechers belegt persönliche Verantwortung. Unterscheide stets den Sprecher einer Aussage von darin nur erwähnten Personen. Leite für nicht zugeordnete Sprecher keine Identität ab und erfinde keine persönliche Verantwortung. Gib die technischen SPEAKER_XX-Bezeichnungen nicht im nutzerseitigen Protokoll aus."""
def build_direct_protocol_prompt( def build_direct_protocol_prompt(
transcript: str, transcript: str,
meeting_context: str | None = None, meeting_context: str | None = None,
*, *,
instruction: str = DIRECT_PROTOCOL_INSTRUCTION, instruction: str = DIRECT_PROTOCOL_INSTRUCTION,
meeting_language: str = "de",
) -> str: ) -> str:
context = meeting_context.strip() if meeting_context else "Kein Meeting-Kontext bereitgestellt." context = meeting_context.strip() if meeting_context else "Kein Meeting-Kontext bereitgestellt."
language = {"de": "German", "en": "English"}.get(meeting_language, meeting_language)
return ( return (
f"{instruction}\n\n" f"{instruction}\n\n"
f"Write the meeting protocol in {language}. "
"Preserve speaker and person names exactly as supplied.\n\n"
f"MEETING-KONTEXT:\n{context}\n\n" f"MEETING-KONTEXT:\n{context}\n\n"
f"VOLLSTAENDIGES TRANSKRIPT:\n{transcript.strip()}\n" f"VOLLSTAENDIGES TRANSKRIPT:\n{transcript.strip()}\n"
) )
@@ -20,6 +20,7 @@ from src.meeting_lab.models.meeting_context import (
) )
from src.meeting_lab.protocol.direct_protocol_prompt import ( from src.meeting_lab.protocol.direct_protocol_prompt import (
COMPACT_DIARIZED_PROTOCOL_INSTRUCTION, COMPACT_DIARIZED_PROTOCOL_INSTRUCTION,
MAPPED_SPEAKER_ATTRIBUTION_INSTRUCTION,
build_direct_protocol_prompt, build_direct_protocol_prompt,
) )
from src.meeting_lab.protocol.transcript_input import ( from src.meeting_lab.protocol.transcript_input import (
@@ -33,7 +34,7 @@ DEFAULT_MODEL = "qwen3.6:35B-A3B"
DEFAULT_NUM_CTX = 32768 DEFAULT_NUM_CTX = 32768
DEFAULT_NUM_PREDICT = 8192 DEFAULT_NUM_PREDICT = 8192
DEFAULT_TIMEOUT = 1800 DEFAULT_TIMEOUT = 1800
DEFAULT_SAFE_INPUT_TOKEN_BUDGET = 16_200 DEFAULT_SAFE_INPUT_TOKEN_BUDGET = 29_000
ESTIMATED_UTF8_BYTES_PER_TOKEN = 4.4 ESTIMATED_UTF8_BYTES_PER_TOKEN = 4.4
@@ -95,6 +96,7 @@ def select_transcript_input(
rendered_context: str | None, rendered_context: str | None,
*, *,
safe_input_token_budget: int = DEFAULT_SAFE_INPUT_TOKEN_BUDGET, safe_input_token_budget: int = DEFAULT_SAFE_INPUT_TOKEN_BUDGET,
meeting_language: str = "de",
) -> SelectedTranscriptInput: ) -> SelectedTranscriptInput:
"""Select complete prompt input without allowing silent tail truncation.""" """Select complete prompt input without allowing silent tail truncation."""
if safe_input_token_budget <= 0: if safe_input_token_budget <= 0:
@@ -106,10 +108,17 @@ def select_transcript_input(
plain_text = plain_segment_transcript(transcript.get("segments")) plain_text = plain_segment_transcript(transcript.get("segments"))
except TranscriptInputError as exc: except TranscriptInputError as exc:
raise DirectProtocolError(str(exc)) from exc raise DirectProtocolError(str(exc)) from exc
instruction = COMPACT_DIARIZED_PROTOCOL_INSTRUCTION
if (
rendered_context
and "Confirmed diarization speaker mappings" in rendered_context
):
instruction = f"{instruction}\n\n{MAPPED_SPEAKER_ATTRIBUTION_INSTRUCTION}"
compact_prompt = build_direct_protocol_prompt( compact_prompt = build_direct_protocol_prompt(
compact.text, compact.text,
rendered_context, rendered_context,
instruction=COMPACT_DIARIZED_PROTOCOL_INSTRUCTION, instruction=instruction,
meeting_language=meeting_language,
) )
compact_estimate = estimate_input_tokens(compact_prompt) compact_estimate = estimate_input_tokens(compact_prompt)
if compact_estimate <= safe_input_token_budget: if compact_estimate <= safe_input_token_budget:
@@ -131,7 +140,9 @@ def select_transcript_input(
representation = "plain_transcript" representation = "plain_transcript"
fallback_used = False fallback_used = False
plain_prompt = build_direct_protocol_prompt(plain_text, rendered_context) plain_prompt = build_direct_protocol_prompt(
plain_text, rendered_context, meeting_language=meeting_language
)
plain_estimate = estimate_input_tokens(plain_prompt) plain_estimate = estimate_input_tokens(plain_prompt)
if plain_estimate > safe_input_token_budget: if plain_estimate > safe_input_token_budget:
raise DirectProtocolError( raise DirectProtocolError(
@@ -168,10 +179,15 @@ def generate_direct_protocol(
load_meeting_context(context_path) if context_path is not None else None load_meeting_context(context_path) if context_path is not None else None
) )
rendered_context = render_meeting_context_for_prompt(context) if context else None rendered_context = render_meeting_context_for_prompt(context) if context else None
# Older runs without a meeting language retain the historical German output.
meeting_language = (
str(context.data["meeting"].get("language") or "de") if context else "de"
)
selected = select_transcript_input( selected = select_transcript_input(
transcript, transcript,
rendered_context, rendered_context,
safe_input_token_budget=safe_input_token_budget, safe_input_token_budget=safe_input_token_budget,
meeting_language=meeting_language,
) )
model_metadata = model_check(endpoint, model, 10) model_metadata = model_check(endpoint, model, 10)
@@ -185,6 +201,7 @@ def generate_direct_protocol(
) )
data = generation.raw_response data = generation.raw_response
runtime_metadata = { runtime_metadata = {
"output_language": meeting_language,
"model": model, "model": model,
"prompt_token_count": data.get("prompt_eval_count"), "prompt_token_count": data.get("prompt_eval_count"),
"output_token_count": data.get("eval_count"), "output_token_count": data.get("eval_count"),
@@ -205,6 +222,19 @@ def generate_direct_protocol(
"input_token_estimation_method": "utf8_bytes_divided_by_4.4", "input_token_estimation_method": "utf8_bytes_divided_by_4.4",
"fallback_used": selected.fallback_used, "fallback_used": selected.fallback_used,
"diarization_enabled": selected.diarization_enabled, "diarization_enabled": selected.diarization_enabled,
"speaker_attribution_available": (
True
if selected.representation == "diarized_compact"
else False
if selected.representation == "plain_transcript_fallback"
else None
),
"speaker_attribution_loss_reason": (
"plain_transcript_fallback"
if selected.representation == "plain_transcript_fallback"
else None
),
"speaker_mapping_count": len(context.speaker_mappings) if context else 0,
} }
return DirectProtocolResult( return DirectProtocolResult(
protocol_text=generation.text, protocol_text=generation.text,
+4 -2
View File
@@ -116,6 +116,7 @@ class GeneratorTests(unittest.TestCase):
self.assertEqual(check.call_count, 1) self.assertEqual(check.call_count, 1)
self.assertEqual(call.call_count, 1) self.assertEqual(call.call_count, 1)
self.assertEqual(call.call_args.args[1], "qwen3.6:35B-A3B") self.assertEqual(call.call_args.args[1], "qwen3.6:35B-A3B")
self.assertEqual(call.call_args.kwargs["num_ctx"], 32768)
self.assertEqual(result.runtime_metadata["request_count"], 1) self.assertEqual(result.runtime_metadata["request_count"], 1)
self.assertEqual(result.runtime_metadata["prompt_token_count"], 123) self.assertEqual(result.runtime_metadata["prompt_token_count"], 123)
self.assertFalse(result.runtime_metadata["think"]) self.assertFalse(result.runtime_metadata["think"])
@@ -181,7 +182,7 @@ class OllamaTests(unittest.TestCase):
with patch.object(ollama.requests, "post", return_value=response) as post: with patch.object(ollama.requests, "post", return_value=response) as post:
result = ollama.generate_once( result = ollama.generate_once(
"http://localhost:11434", "http://localhost:11434",
"chosen:model", "qwen3.8:27b",
"prompt", "prompt",
timeout=30, timeout=30,
num_ctx=32768, num_ctx=32768,
@@ -190,8 +191,9 @@ class OllamaTests(unittest.TestCase):
self.assertEqual(post.call_count, 1) self.assertEqual(post.call_count, 1)
payload = post.call_args.kwargs["json"] payload = post.call_args.kwargs["json"]
self.assertEqual(payload["model"], "chosen:model") self.assertEqual(payload["model"], "qwen3.8:27b")
self.assertEqual(payload["options"]["temperature"], 0.0) self.assertEqual(payload["options"]["temperature"], 0.0)
self.assertEqual(payload["options"]["num_ctx"], 32768)
self.assertFalse(payload["think"]) self.assertFalse(payload["think"])
self.assertFalse(payload["stream"]) self.assertFalse(payload["stream"])
self.assertEqual(result.raw_response, raw) self.assertEqual(result.raw_response, raw)
+164
View File
@@ -0,0 +1,164 @@
"""Meeting-language regression tests without media or model execution."""
import json
import tempfile
import unittest
from functools import partial
from pathlib import Path
from unittest.mock import Mock, patch
from src.meeting_lab.llm.ollama import OllamaGeneration
from src.meeting_lab.models.meeting_context import load_meeting_context
from src.meeting_lab.orchestration.mvp import regenerate_mvp_protocol
from src.meeting_lab.protocol.direct_protocol_prompt import build_direct_protocol_prompt
from src.meeting_lab.protocol.generate_direct_protocol import (
estimate_input_tokens,
generate_direct_protocol,
select_transcript_input,
)
class MeetingLanguageTests(unittest.TestCase):
def test_diarized_plain_fallback_retains_english_instruction(self) -> None:
segments = [
{
"start": index,
"end": index + 1,
"text": "Word.",
"speaker_id": f"SPEAKER_{index % 2:02d}",
}
for index in range(200)
]
plain = " ".join(segment["text"] for segment in segments)
budget = estimate_input_tokens(
build_direct_protocol_prompt(plain, meeting_language="en")
)
selected = select_transcript_input(
{"text": plain, "segments": segments, "speaker_labels_anonymous": True},
None,
meeting_language="en",
safe_input_token_budget=budget,
)
self.assertEqual(selected.representation, "plain_transcript_fallback")
self.assertIn("Write the meeting protocol in English.", selected.prompt)
self.assertEqual(selected.text, plain)
def test_generation_and_regeneration_preserve_language_and_inputs(self) -> None:
for language, expected in (
("de", "German"),
("en", "English"),
(None, "German"),
):
for diarized in (False, True):
with (
self.subTest(language=language, diarized=diarized),
tempfile.TemporaryDirectory() as directory,
):
run = Path(directory)
context_path = run / "context" / "meeting_context.yaml"
context_path.parent.mkdir()
context_text = (
'schema_version: "1"\nmeeting:\n'
" meeting_id: test\n title: Mixed terminology\n"
+ (f" language: {language}\n" if language else "")
+ ' notes: "Freigabe für Product X"\n'
"participants:\n - participant_id: person\n"
' display_name: "Jörg Müller"\n'
"speaker_mappings:\n SPEAKER_00: person\n"
)
context_path.write_text(context_text, encoding="utf-8")
transcript = run / ("diarization" if diarized else "transcript")
transcript.mkdir()
transcript /= (
"transcript_diarized.json" if diarized else "transcript.json"
)
transcript.write_text(
json.dumps(
{
"text": "I will check Product X.",
"speaker_labels_anonymous": diarized,
"segments": [
{
"id": 0,
"start": 0,
"end": 1,
"speaker_id": "SPEAKER_00",
"text": "I will check Product X.",
}
],
}
),
encoding="utf-8",
)
original = transcript.read_bytes()
call = Mock(
return_value=OllamaGeneration(
text="# Meeting Protocol",
raw_response={},
client_wall_time_seconds=0.1,
)
)
generate = partial(
generate_direct_protocol,
model_check=Mock(return_value={}),
generation_call=call,
)
result = generate(transcript, context_path)
self.assertIn(
f"Write the meeting protocol in {expected}.",
result.exact_prompt,
)
self.assertNotIn("in deutscher Sprache", result.exact_prompt)
self.assertNotIn("auf Deutsch", result.exact_prompt)
self.assertIn("Jörg Müller", result.exact_prompt)
self.assertIn("Mixed terminology", result.exact_prompt)
self.assertIn("I will check Product X.", result.transcript_input)
self.assertEqual(
result.runtime_metadata["output_language"], language or "de"
)
self.assertEqual(
context_path.read_text(encoding="utf-8"), context_text
)
context = load_meeting_context(context_path)
with (
patch(
"src.meeting_lab.orchestration.mvp.generate_direct_protocol",
side_effect=generate,
),
patch(
"src.meeting_lab.orchestration.mvp.transcribe_audio",
side_effect=AssertionError("Retranscription is forbidden"),
),
):
regenerate_mvp_protocol(run, meeting_context=context)
metadata = json.loads(
(run / "protocol" / "runtime_metadata.json").read_text()
)
self.assertEqual(metadata["output_language"], language or "de")
self.assertIn(
f"Write the meeting protocol in {expected}.",
call.call_args.args[2],
)
self.assertEqual(transcript.read_bytes(), original)
self.assertEqual(
load_meeting_context(context_path).data, context.data
)
self.assertEqual(context.speaker_mappings, {"SPEAKER_00": "person"})
def test_no_context_defaults_to_german(self) -> None:
with tempfile.TemporaryDirectory() as directory:
transcript = Path(directory) / "transcript.json"
transcript.write_text('{"text": "English source text."}')
result = generate_direct_protocol(
transcript,
model_check=Mock(return_value={}),
generation_call=Mock(
return_value=OllamaGeneration(
text="Protocol",
raw_response={},
client_wall_time_seconds=0.1,
)
),
)
self.assertIn("Write the meeting protocol in German.", result.exact_prompt)
self.assertEqual(result.runtime_metadata["output_language"], "de")
+62 -2
View File
@@ -115,7 +115,7 @@ class MvpApiTests(unittest.TestCase):
patch.object(mvp_api, "prepare_audio", side_effect=fake_prepare), patch.object(mvp_api, "prepare_audio", side_effect=fake_prepare),
patch.object( patch.object(
mvp_api, "generate_direct_protocol", side_effect=fake_protocol mvp_api, "generate_direct_protocol", side_effect=fake_protocol
), ) as protocol_generator,
patch.object(subprocess, "run") as subprocess_run, patch.object(subprocess, "run") as subprocess_run,
): ):
result = mvp_api.run_mvp_meeting( result = mvp_api.run_mvp_meeting(
@@ -142,6 +142,65 @@ class MvpApiTests(unittest.TestCase):
], ],
) )
self.assertTrue(all(event.progress is None for event in events)) self.assertTrue(all(event.progress is None for event in events))
self.assertEqual(protocol_generator.call_args.kwargs["num_ctx"], 32_768)
self.assertEqual(
protocol_generator.call_args.kwargs["safe_input_token_budget"],
29_000,
)
def test_protocol_only_regeneration_reuses_diarized_artifacts(self):
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
run_dir = root / "existing-run"
diarization_dir = run_dir / "diarization"
diarization_dir.mkdir(parents=True)
source = diarization_dir / "transcript_diarized.json"
source.write_text(
json.dumps(
{
"text": "SPEAKER_00: Existing statement.\n",
"segments": [
{
"start": 0.0,
"end": 1.0,
"speaker_id": "SPEAKER_00",
"text": "Existing statement.",
}
],
"speaker_labels_anonymous": True,
}
),
encoding="utf-8",
)
source_before = source.read_bytes()
mapped_context = context_data()
mapped_context["speaker_mappings"] = {"SPEAKER_00": "person-1"}
with (
patch.object(mvp_api, "prepare_audio") as preparation,
patch.object(mvp_api, "transcribe_audio") as transcription,
patch.object(mvp_api, "diarize_audio") as diarization,
patch.object(
mvp_api, "generate_direct_protocol", side_effect=fake_protocol
) as protocol,
):
result = mvp_api.regenerate_mvp_protocol(
run_dir,
meeting_context=mapped_context,
model="qwen3.8:27b",
protocol_num_ctx=32_768,
protocol_safe_input_token_budget=29_000,
)
self.assertEqual(result.exit_code, 0)
self.assertEqual(result.protocol_path, run_dir / "protocol.md")
preparation.assert_not_called()
transcription.assert_not_called()
diarization.assert_not_called()
self.assertEqual(protocol.call_args.args[0], source)
self.assertEqual(protocol.call_args.kwargs["num_ctx"], 32_768)
self.assertEqual(source.read_bytes(), source_before)
persisted = load_meeting_context(run_dir / "context/meeting_context.yaml")
self.assertEqual(persisted.speaker_mappings, {"SPEAKER_00": "person-1"})
def test_failure_emits_terminal_failure_event(self): def test_failure_emits_terminal_failure_event(self):
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
@@ -180,7 +239,8 @@ class MvpApiTests(unittest.TestCase):
self.assertEqual(delegated.whisper_executable, "whisper-cli") self.assertEqual(delegated.whisper_executable, "whisper-cli")
self.assertEqual(delegated.ffmpeg_executable, "ffmpeg") self.assertEqual(delegated.ffmpeg_executable, "ffmpeg")
self.assertTrue(delegated.audio_normalization) self.assertTrue(delegated.audio_normalization)
self.assertEqual(delegated.protocol_safe_input_token_budget, 16_200) self.assertEqual(delegated.protocol_num_ctx, 32_768)
self.assertEqual(delegated.protocol_safe_input_token_budget, 29_000)
self.assertEqual(api.call_args.kwargs["meeting_context"], context_data()) self.assertEqual(api.call_args.kwargs["meeting_context"], context_data())
def test_cli_explicit_audio_normalization_values_are_propagated(self): def test_cli_explicit_audio_normalization_values_are_propagated(self):
+85 -1
View File
@@ -2,9 +2,10 @@ import json
import tempfile import tempfile
import unittest import unittest
from pathlib import Path from pathlib import Path
from unittest.mock import Mock from unittest.mock import Mock, patch
from src.meeting_lab.diarization.alignment import diarized_transcript_text from src.meeting_lab.diarization.alignment import diarized_transcript_text
from src.meeting_lab.llm import ollama
from src.meeting_lab.llm.ollama import OllamaGeneration from src.meeting_lab.llm.ollama import OllamaGeneration
from src.meeting_lab.protocol.direct_protocol_prompt import ( from src.meeting_lab.protocol.direct_protocol_prompt import (
COMPACT_DIARIZED_PROTOCOL_INSTRUCTION, COMPACT_DIARIZED_PROTOCOL_INSTRUCTION,
@@ -136,6 +137,81 @@ class ProtocolInputBudgetTests(unittest.TestCase):
self.assertEqual(result.transcript_input, compact) self.assertEqual(result.transcript_input, compact)
self.assertEqual(call.call_count, 1) self.assertEqual(call.call_count, 1)
def test_mapped_speakers_and_statements_reach_final_ollama_payload(self) -> None:
diarized = {
"text": "",
"segments": [
{
"start": 0.0,
"end": 1.0,
"speaker_id": "SPEAKER_00",
"text": "We will run the trial on Wednesday.",
},
{
"start": 1.0,
"end": 2.0,
"speaker_id": "SPEAKER_01",
"text": "I will prepare the raw materials before then.",
},
{
"start": 2.0,
"end": 3.0,
"speaker_id": "SPEAKER_00",
"text": "Good. Anna owns the material preparation.",
},
],
"speaker_labels_anonymous": True,
"alignment_source": "exclusive_diarization",
}
context = {
"schema_version": "1",
"meeting": {
"meeting_id": "speaker-test",
"title": "Speaker test",
"language": "en",
},
"participants": [
{"participant_id": "martin", "display_name": "Martin"},
{"participant_id": "anna", "display_name": "Anna"},
],
"speaker_mappings": {"SPEAKER_00": "martin", "SPEAKER_01": "anna"},
"mentioned_people": [],
"organization": {"departments": []},
"known_entities": {},
}
response = Mock()
response.raise_for_status.return_value = None
response.json.return_value = {"response": "# Meeting Protocol\n", "done": True}
with tempfile.TemporaryDirectory() as directory:
root = Path(directory)
transcript = self._write(root, diarized)
context_path = root / "context.yaml"
context_path.write_text(json.dumps(context), encoding="utf-8")
with patch.object(ollama.requests, "post", return_value=response) as post:
result = generate_direct_protocol(
transcript,
context_path,
model="qwen3.8:27b",
model_check=Mock(return_value={}),
)
prompt = post.call_args.kwargs["json"]["prompt"]
self.assertEqual(result.exact_prompt, prompt)
self.assertIn("- SPEAKER_00: Martin (participant_id: martin)", prompt)
self.assertIn("- SPEAKER_01: Anna (participant_id: anna)", prompt)
self.assertIn("SPEAKER_00: We will run the trial on Wednesday.", prompt)
self.assertIn(
"SPEAKER_01: I will prepare the raw materials before then.", prompt
)
self.assertIn("SPEAKER_00: Good. Anna owns the material preparation.", prompt)
self.assertNotIn("Martin: We will run the trial on Wednesday.", prompt)
self.assertNotIn("Anna: I will prepare the raw materials before then.", prompt)
self.assertIn("autoritativen SPEAKER_XX-zu-Teilnehmer-Zuordnungen", prompt)
self.assertIn("Ich-Zusage", prompt)
self.assertIn("nur erwähnten Personen", prompt)
self.assertIn("keine persönliche Verantwortung", prompt)
def test_plain_fallback_selected_when_diarized_compact_exceeds_budget(self) -> None: def test_plain_fallback_selected_when_diarized_compact_exceeds_budget(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
root = Path(directory) root = Path(directory)
@@ -180,6 +256,13 @@ class ProtocolInputBudgetTests(unittest.TestCase):
) )
self.assertTrue(result.runtime_metadata["fallback_used"]) self.assertTrue(result.runtime_metadata["fallback_used"])
self.assertEqual(result.transcript_input, plain) self.assertEqual(result.transcript_input, plain)
self.assertNotIn("SPEAKER_00", result.exact_prompt)
self.assertNotIn("SPEAKER_01", result.exact_prompt)
self.assertFalse(result.runtime_metadata["speaker_attribution_available"])
self.assertEqual(
result.runtime_metadata["speaker_attribution_loss_reason"],
"plain_transcript_fallback",
)
def test_oversized_plain_transcript_fails_before_any_network_call(self) -> None: def test_oversized_plain_transcript_fails_before_any_network_call(self) -> None:
with tempfile.TemporaryDirectory() as directory: with tempfile.TemporaryDirectory() as directory:
@@ -225,6 +308,7 @@ class ProtocolInputBudgetTests(unittest.TestCase):
) )
self.assertFalse(result.runtime_metadata["fallback_used"]) self.assertFalse(result.runtime_metadata["fallback_used"])
self.assertFalse(result.runtime_metadata["diarization_enabled"]) self.assertFalse(result.runtime_metadata["diarization_enabled"])
self.assertIsNone(result.runtime_metadata["speaker_attribution_available"])
if __name__ == "__main__": if __name__ == "__main__":