---
license: mit
pipeline_tag: automatic-speech-recognition
language: [en]
base_model: openai/whisper-tiny.en
tags: [whisper, whisper-cpp, ggml, quantized, on-device, research]
---
# Tock ASR 0.2.0 — Whisper tiny.en Q5_1

Research candidate for local English transcription. Based on OpenAI Whisper tiny.en, GGML conversion by whisper.cpp, quantized locally with whisper.cpp v1.7.6 Q5_1. No new pretraining or fine-tuning is claimed. Upstream weights and runtime keep their MIT licenses and attribution. The adapter code is original Embermote work; the quantization algorithm is upstream work.

## Comparative evaluation

| Same-run protocol | Original FP16 | Tock Q5_1 |
| --- | ---: | ---: |
| LibriSpeech test-clean WER | 5.5670% | 6.2044% |
| Word edits / reference words | 2952 / 53027 | 3290 / 53027 |
| Artifact bytes | 77704715 | 32166155 |
| Evaluation wall seconds | 272.981 | 265.761 |

The compressed artifact is 58.60% smaller, with 0.6374 percentage points higher WER on this test. No accuracy improvement or statistically established speedup is claimed.

Both variants ran all 2,620 recordings (5.4035 hours) from LibriSpeech test-clean, without filtering failures. Same M4 Pro Mac, CPU + Apple Accelerate, 4 threads, greedy decoding, English, temperature fallback disabled, max text context 0. whisper.cpp commit `a8d002cfd879315632a579e73f0148d06959de36`. WER is total word edit distance divided by total normalized reference words, not the average of per-clip error rates.

Whisper EnglishTextNormalizer at `31243bad24cc746f07d4c8bfdd2d974872cb1803` is applied identically to hypotheses and references. Model hashes, data hash, test-ID hash, complete decoder config, and aggregate per-speaker results are in metrics.json. All raw audio, text references, and per-recording predictions stay private.

Wall times are a single full pass per model, including batch startup/loading/output writes, and are not a repeated isolated latency benchmark. They do not establish iPhone, Neural Engine, Android, peak RAM, or long-form performance. These are our own measurements, not Open ASR Leaderboard results. Only test-clean has been run; the other leaderboard datasets have not.

## Reproduce

With the pinned private source checkouts and model/dataset archives described in ml/BENCHMARKS.md:

```sh
.venv/bin/python -m ml.embermote_ml.benchmark_asr --output ml/runs/tock-asr-full
```

Input is 16 kHz mono PCM WAV. The native CLI can produce text and segment timestamps. Timestamp accuracy is unmeasured. This artifact is custom GGML, not a Transformers AutoModel checkpoint.

```sh
whisper-cli -m tock-tiny.en-q5_1.bin -f recording.wav -l en -t 4 -bs 1 -bo 1 -nf -mc 0 -otxt
```

Workspace adapters: `@embermote/tock-asr` for native Node.js and `dev.embermote:tock-asr:0.2.0` for desktop/server Kotlin/JVM. Both require a separately installed native executable. They are not yet published to package registries and do not run in a browser or Android app as provided.

## Browser preview

The model page includes an on-device WebAssembly demo built from whisper.cpp v1.7.6 with Emscripten 3.1.74. It uses this same Q5_1 artifact in a dedicated single-thread worker with SIMD, greedy decoding, no previous text context and no temperature fallback. Audio is decoded and resampled locally to 16 kHz mono. The demo accepts English clips of 0.1–30 seconds and files up to 15 MB. Only public model weights are cached; audio and transcripts are not uploaded or persisted.

Browser runtime size and memory are additional to the model weights. The native benchmark above does not measure browser speed or establish browser WER. The displayed demo timer measures inference for the selected clip, excluding download, model loading and audio preparation. Desktop browsers are recommended; mobile device performance is not benchmarked.

## Training provenance and limitations

This candidate inherits Whisper's upstream pretraining. OpenAI documents 680,000 hours across the Whisper family, including an English subset; that is upstream work, not data trained here. We did not fine-tune on or train with LibriSpeech test-clean. Quantization used model weights only.

The benchmark is clean read English speech. It does not establish robustness on meetings, telephone speech, background noise, varied microphones, code-switching, or long-form files. Whisper can hallucinate words, particularly on silence or noise. Review transcripts before consequential use.

## Sources

- [OpenAI Whisper](https://github.com/openai/whisper), model family and original MIT license.
- [GGML model conversion](https://huggingface.co/ggerganov/whisper.cpp), downloaded tiny.en artifact SHA-256 `921e4cf8686fdd993dcd081a5da5b6c365bfde1162e72b08d75ac75289920b1f`.
- [whisper.cpp](https://github.com/ggml-org/whisper.cpp/tree/v1.7.6), native runtime and Q5_1 quantizer.
- [LibriSpeech](https://www.openslr.org/12), CC BY 4.0 evaluation corpus by Panayotov et al.; archive SHA-256 `39fde525e59672dc6d1551919b1478f724438a95aa55f874b576be21967e6c23`.

Candidate SHA-256: `20997113f1fa483058c65fa5959a1027887a823db39ac5de6c78c272e2dd5aac`.