Do not mix recording and interpretation
The browser’s first responsibility is to capture complete, usable audio. The recording should stop only through a clear user action or an intentionally configured silence rule. Show recording state, elapsed time, input level, and a review or continue step so the user knows whether the full conversation was captured.
Upload the original audio as one source of truth where size and privacy allow. Chunking can support long sessions, but boundaries must not lose words or reorder content. Store recording metadata such as format, sample rate, language hint, duration, and client timestamp.
Use a dedicated transcription stage
Speech-to-text should produce a faithful transcript before any attempt to fill fields. Preserve mixed-language phrases, numbers, units, negations, and uncertainty. Do not silently “correct” clinical or technical terms into a more familiar word without retaining the original evidence.
If speaker separation is important, capture it explicitly or label the limitation. The transcription response should include status and, where available, timing or confidence metadata. A failed or partial transcript should stop the downstream workflow rather than produce confident-looking structured data from incomplete text.
Extract against a strict schema
The second stage should receive the transcript and a defined schema. Each field needs a name, type, allowed values, description, and missing-data behavior. Require valid JSON or another structured output rather than asking the model to write a narrative and then parsing it with fragile regular expressions.
For each extracted value, preserve the supporting text span or a concise evidence note. This helps the reviewer see why a value was chosen. Do not invent absent values. Use null, unknown, or requires_review according to the schema instead of filling every field.
Separate rules from model judgement
Deterministic rules are appropriate for format checks, numerical ranges, required fields, and known thresholds. The language model is useful for mapping free speech to concepts and resolving ordinary phrasing. Keep those responsibilities visible so a safety rule is not buried inside an unpredictable prompt.
Run validation after extraction. Check units, numeric ranges, mutually exclusive selections, date formats, and mandatory relationships. A valid JSON object can still contain an impossible value. Validation errors should return the workflow to review rather than being silently accepted.
Handle mixed language deliberately
A bilingual workflow should allow the user to switch or mix languages without restarting. Keep a language-neutral schema and map display labels separately. Preserve proper nouns and clinically significant words as spoken. Test accents, code-switching, background noise, and numbers in both languages.
Prompts should include examples of acceptable mixed-language evidence, but not encourage translation that changes meaning. Where a translated display is needed, retain the original transcript beside it. Reviewers should be able to return to the source wording.
Use confidence as a routing signal
Confidence should influence workflow, not be shown as decorative precision. Low audio quality, uncertain transcript spans, conflicting values, or missing required fields should route to manual completion. Confidence may be field-specific because a name can be clear while a dosage or measurement is uncertain.
Define fallback behavior before launch: request clarification, leave the field blank, show the transcript segment, or ask the clinician to select from allowed values. Never let a low-confidence extraction automatically trigger a high-impact action.
Keep a human confirmation and audit trail
The final screen should show transcript, extracted fields, warnings, and evidence. The authorized user confirms or overrides values and records a reason for material changes. Save model version, prompt version, transcript ID, extracted result, validation issues, final decision, and timestamps.
Monitor the workflow with reviewed examples. Categorize failures into audio capture, transcription, extraction, validation, and interface errors. Improving the correct stage is more effective than repeatedly changing one large prompt.