whisper.cpp is a high-performance C/C++ port of OpenAI’s Whisper ASR: MIT-licensed, CPU- and GPU-friendly, with offline demos on phones and desktops. Its public whisper-stream example samples the mic every half second and re-runs transcription in a loop; whisper-server exposes an HTTP inference path for WAV uploads. There is no cloud list price in dollars per audio minute — you bring the machine, the model file, and the ops.
The LecSync Realtime Speech API is a managed WebSocket for live meeting and lecture captions: preview and final text, translation between any two of 60 languages, a speaker and audio-aligned timestamps on every final segment, and running minutes. It costs $0.006 per audio minute ($0.36 per hour) with those features included.
Figures below were checked on October 29, 2026 against the whisper.cpp README, stream example, models list, server example, the MIT LICENSE, OpenAI Whisper’s tokenizer language map, and the LecSync developers page. This post is about self-host streaming ASR versus a managed caption API — not hosted Whisper SKUs such as Groq, Cloudflare Workers AI, or Together AI.
What is Whisper.cpp self-host streaming vs a realtime caption API?
Whisper.cpp is a local inference library: you download a ggml model, run whisper-cli, whisper-stream, or whisper-server on your own CPU/GPU/NPU, and own latency, memory, and deployment. A realtime caption API such as LecSync keeps one managed socket open for live bilingual captions, speakers and minutes while people are still talking — you pay a flat minute bill instead of operating the stack.
The short answer
If you need offline or air-gapped ASR, MIT license freedom, or full control of where audio is decoded, Whisper.cpp is the stronger fit. The stream demo is explicitly a “naive” mic loop; you tune step/length, VAD, model size and hardware yourself. There is no per-minute cloud invoice from the project.
If you need live bilingual captions + speakers + minutes on one flat minute bill, LecSync is the closer product. Whisper.cpp does not ship built-in any-to-any live text translation, managed speaker labels on a caption socket, or a live digest the way LecSync does. Whisper’s own translate task (upstream / --translate on the server) targets English, not arbitrary caption pairs.
How each path works
LecSync. Your server calls POST https://api.lecsync.com/v1/realtime/connect with the audio format, a language pair and the features you want, then streams 16 kHz mono PCM. Preview text arrives while someone is still talking; a final segment settles once the sentence confirms. Translations share the segment ID. With speakers: true, each final carries a speaker and startMs/endMs on the recording clock. With digest: true, minutes arrive section by section. The first word shows up about 0.28 seconds after speech starts; translation adds roughly 88 ms. Seven gateway regions (dal, jp3, kc, nj, ru2, ru3, uk) keep audio near the capture point; backlog is capped at 2 seconds.
Whisper.cpp (self-host). Clone ggml-org/whisper.cpp, download a ggml model (tiny through large-v3-turbo; multilingual unless the name ends in .en), and build with CMake. For mic-style streaming, the README’s real-time example runs:
./build/bin/whisper-stream -m ./models/ggml-base.en.bin -t 8 --step 500 --length 5000
That samples about every 500 ms over a 5 s window (naive continuous re-transcription). Sliding-window VAD mode (--step 0) waits for speech activity before emitting timestamped blocks. Optional Silero-VAD flags, Metal/CUDA/Vulkan/ROCm backends, and quantized q5_0 weights cut memory. Experimental tinydiarize (-tdrz with a *-tdrz model) appends [SPEAKER_TURN] markers locally — not the same as managed per-segment speaker IDs on a live caption API. whisper-server accepts WAV over HTTP (/inference); it is an OAI-like file server, not a managed preview→final caption WebSocket with MT and digest.
Pricing and ops narrative (no invented Whisper.cpp $/min)
| Path | Published commercial rate | What you actually pay | Notes |
|---|---|---|---|
| LecSync Realtime Speech API | $0.006 / min all-in | Cloud minute bill | STT + translation + speakers + digest included |
| Whisper.cpp | None (MIT software) | Hardware + electricity + your engineering time | License free; ops cost is yours |
LecSync math (Calculated from the public minute price): $0.006 × 60 = $0.36 per hour of audio; 1,000 hours = $360. Disabling features does not lower the price; enabling all of them does not raise it.
Whisper.cpp has no official cloud $/min. Claiming a GPU-rental “Whisper.cpp price” would be invented list pricing. Qualitatively: small models (tiny/base) fit modest CPU RAM (README memory table: tiny ~273 MB mem, base ~388 MB); large needs ~3.9 GB mem on disk/weights alone before concurrent streams. You choose CPU vs GPU, concurrency, model size, and whether captions stay on-prem. Latency is whatever your box delivers — community benches live in whisper.cpp issue #89 via whisper-bench; the project does not publish a single “first word on screen” SLA comparable to LecSync’s ~0.28 s figure.
For stitching your own STT + MT after local ASR, see built-in translation vs a DIY pipeline. For flat-vs-add-on SaaS math, see what 1,000 hours cost once translation, speakers and summaries are added.
Side-by-side facts (checked October 29, 2026)
| LecSync Realtime Speech API | Whisper.cpp (self-host) | |
|---|---|---|
| Primary job | Live meeting/lecture captions + MT | Local Whisper ASR inference |
| Price model | $0.006/min flat, all features | MIT; no cloud $/min |
| Delivery | Managed WebSocket, preview→final | CLI / stream mic loop / HTTP WAV server |
| Live text translation | Any two of 60, same socket | Upstream translate → English; not any-to-any live captions |
| Speakers | Speaker on every final (speakers: true) | Experimental tinydiarize [SPEAKER_TURN]; stereo diarize flag on server |
| Live minutes / digest | digest: true | Not built-in |
| Languages | 60, auto-detected | Multilingual Whisper models; ~100 codes in OpenAI LANGUAGES map |
| Latency story | ~0.28 s first word; +~88 ms MT | Hardware-dependent; stream demo ~500 ms step / naive loop |
| Regions / privacy | 7 gateway regions; optional 30-day store | Fully local / your VPC if you host it |
| License / lock-in | Managed SaaS | MIT; you own the binary and models you download |
What 1,000 hours of product work looks like
| Workload | LecSync | Whisper.cpp self-host |
|---|---|---|
| Same-language live captions | $360 cloud | $0 license + your GPU/CPU capacity |
| Translated live captions (arbitrary pair) | $360 | DIY: local STT + separate MT service or model |
| Speakers with timestamps during the session | Included | Build on tinydiarize / your own diarization |
| Live minutes | Included | Build your own summarizer |
| Air-gapped / on-device privacy | Audio hits LecSync regions (store off by default) | Audio can stay on the device or private host |
| Predictable SaaS invoice | Yes | No — capacity planning replaces the minute bill |
Whisper.cpp wins when privacy, license freedom, or zero per-minute cloud STT matter more than shipping a caption product tomorrow. LecSync wins when the product is a live bilingual caption feed with speakers and minutes and you would rather not run ASR GPUs, MT, diarization and backlog control yourself.
When Whisper.cpp is the better pick
- Offline / on-device / regulated environments where audio must not leave the machine (iOS/Android/WASM demos in the repo).
- MIT license and full stack control — quantize, swap backends (Metal, CUDA, Vulkan, ROCm, OpenVINO, NPUs), pin model hashes.
- Batch or file transcription on your own fleet via
whisper-cliorwhisper-serverHTTP WAV inference. - Mic demos and R&D with
whisper-stream’s 500 ms step loop or VAD sliding window. - No per-minute ASR invoice — you already have idle CPU/GPU and an ops team.
When LecSync fits better
- Translated live captions on one socket — any pair of 60 languages, each translation tied to its segment.
- Speakers and recording-aligned timestamps during the live stream — meeting client with speakers and timestamps.
- Live minutes while the session runs (
digest: true) — live digest vs post-call summarization. - Predictable minute billing at $0.36 an hour with no GPU capacity planning for caption peaks.
- Classroom / mixed-language lecture feeds where captions must keep up as the speaker switches languages, without you wiring STT + MT + digest.
Category winners
| Category | Winner | Why |
|---|---|---|
| Offline / local privacy | Whisper.cpp | MIT; runs fully on-device |
| No per-minute cloud STT bill | Whisper.cpp | Software only; you own hardware |
| Live any-to-any captions | LecSync | 60 languages, MT on the same socket |
| Flat all-in $/min for captions | LecSync | $0.006 includes MT, speakers, digest |
| Managed first-word latency story | LecSync | Public ~0.28 s / +~88 ms MT |
| Experimental local speaker turns | Whisper.cpp (tinydiarize) | [SPEAKER_TURN] markers; you operate it |
| Live digest / minutes | LecSync | digest: true |
FAQ
Does Whisper.cpp publish a cloud price per audio minute?
No. whisper.cpp is MIT-licensed inference software. Public pages list model disk/memory sizes and how to build stream/server binaries — not a SaaS $/min. Any GPU rental figure you see elsewhere is an Estimated ops cost from a cloud vendor, not a Whisper.cpp list price.
Is whisper-stream the same as a managed realtime caption API?
No. The stream README calls it a naive real-time mic example: sample every ~500 ms (--step 500) over a window (--length 5000), or VAD mode with --step 0. You still own model choice, thread count, hardware, and anything beyond raw transcript text (MT, speakers, digest).
Can Whisper.cpp do live translation into an arbitrary target language?
Whisper’s multitask translate path (and whisper-server’s -tr / --translate) is designed to produce English from non-English speech. That is not the same as LecSync’s any-to-any live caption pairs on one socket. For DIY STT+MT trade-offs, see built-in translation vs a DIY pipeline.
How many languages do multilingual Whisper.cpp models cover?
Models without .en in the name are multilingual Whisper weights. OpenAI’s Whisper LANGUAGES map in tokenizer.py lists about 100 language codes (checked October 29, 2026). Coverage and accuracy still vary by language; whisper.cpp inherits that model behavior.
When should a live-caption product still self-host Whisper.cpp?
When audio residency, air-gap, or license control dominate — for example an on-prem lecture lab that only needs same-language transcripts. If the product needs bilingual captions, speakers and minutes with a predictable minute bill, start from the LecSync Realtime Speech API.
Final verdict
Whisper.cpp is the right tool when you want MIT-licensed, self-hosted Whisper ASR and you accept owning latency, capacity and the caption product glue. LecSync is the right tool when you want a managed realtime speech API at $0.006/min all-in with live translation, speakers and digest on one connection.
Connect with POST https://api.lecsync.com/v1/realtime/connect, then stream PCM — details on the developers page.