Work · Speech
meeting-notes.
A recording goes in. A transcript, a summary and a list of action items come out — each item with an owner and a due date when one was said.
The problem
Who said they would do what, and by when.
Imagine a weekly standup recorded in a mix of English and Chinese. Afterwards someone listens again to write down who said they would do what, and by when. A missed “Friday” or a task given to the wrong person is easy to let through.
The useful result is not a transcript alone. It is a list of action items — each with an owner and a due date when one was said — that a person can check against the words.
What we built
A transcript, then a list of action items you can check.
The pipeline is five steps. A recording is transcribed in English, Chinese, or both in the same sentence. Speaker labels are assigned locally. A model writes a short summary and a list of action items; each item has an owner and a due date when one was mentioned, and must quote a transcript line. You get Markdown, JSON and subtitles (SRT). The backend can be a hosted API or open models on your own machine; speaker labels always run locally.
A transcript you still have to read is only half the job. A list of action items, each tied to a speaker and a spoken due date, is something a person can check and then use — and because every item has to quote a transcript line, the notes do not invent tasks. They can still leave things out, and a due date can be missed, as the table below shows.
Results
10 of 10 action items found, 9 of 10 due dates right.
-
10 / 10
Action items found
-
10 / 10
Owners right
-
9 / 10
Due dates right
On three meetings — one English, one Chinese, and one that mixes both in the same sentences — the API backend with faster-whisper large-v3-turbo and Qwen2.5-7B-Instruct (Q4_K_M) found 10 of 10 action items, with 10/10 owners right and 9/10 due dates right. Precision was 10/10 (no extra items). Transcript error was CER 2.0% (Chinese), MER 1.1% (mixed) and WER 0.6% (English). Speakers were 2/2 on every sample, with 99%, 97% and 96% of speech time given the right label.
The one due-date miss is on weekly-sync-en.wav: the item “Ask the design team for the new pictures for the app store” came back with no due date, against the scripted “today.” On the Chinese meeting, one of two scripted decisions was listed (1/2); the other two meetings listed every scripted decision (1/1, 1/1).
| Sample | Lang | Transcript | Speakers | Items | Owner | Due | Precision | Time (s) | Notes |
|---|---|---|---|---|---|---|---|---|---|
design-review-zh.wav | zh | CER 2.0% | 2/2 (99%) | 3/3 | 3/3 | 3/3 | 3/3 | 93 | decisions 1/2 |
sprint-planning-mixed.wav | mixed | MER 1.1% | 2/2 (97%) | 3/3 | 3/3 | 3/3 | 3/3 | 101 | — |
weekly-sync-en.wav | en | WER 0.6% | 2/2 (96%) | 4/4 | 4/4 | 3/4 | 4/4 | 191 | due None vs “today” |
| All | 10/10 | 10/10 | 9/10 | 10/10 | 93–191 |
Methodology. Three meetings, ten action items in total. An item counts as found when the notes list a matching task whose quote aligns to a transcript line; owner and due are scored only on items that were found. Precision is found items over items listed. API backend, faster-whisper large-v3-turbo, Qwen2.5-7B-Instruct (Q4_K_M) behind a local llama.cpp server, sherpa-onnx speaker labels, 12 November 2026. Per-meeting times were 93, 101 and 191 seconds. The raw table is in docs/eval.md. The script is scripts/evaluate.py; the samples are in samples/.
The Hugging Face backend with Whisper small and Qwen2.5-1.5B-Instruct found 9 of 10 action items (6/9 owners, 6/9 dues, 9/14 precision). Transcript error was higher on the Chinese and mixed meetings (CER 6.3%, MER 6.3%; English WER stayed 0.6%). Times were 31–36 seconds.
Run it yourself
Three commands, then the demo.
git clone https://github.com/ModelShifts/meeting-notes
cd meeting-notes && python -m venv .venv && source .venv/bin/activate
pip install -e ".[hf,demo]" && meeting-notes serveOn a CPU-only machine, install the CPU build of PyTorch first. For a smaller footprint, set MS_WHISPER_MODEL=small and MS_HF_MODEL=Qwen/Qwen2.5-1.5B-Instruct. Python 3.11 or newer, and ffmpeg on the PATH.