Voice Stack — Whisper, Piper, Coqui, Kokoro

The local voice-AI stack — speech-to-text (STT) plus text-to-speech (TTS) — is mature in 2026. Whisper family for STT; Piper, Coqui XTTS, Kokoro for TTS; integrations with Home Assistant for end-to-end local voice control.

For the bigger picture see overview, hardware tiers; for general transcription see transcription; for productivity dictation see voice dictation.

Speech-to-text (STT)

★ ★ Whisper / WhisperX / faster-whisper

• ★ ★ OpenAI Whisper — the foundational open-weight STT model. MIT.
• ★ ★ whisper.cpp (github.com/ggml-org/whisper.cpp) — C++ port; runs everywhere, GGUF-style quants.
• ★ ★ faster-whisper (github.com/SYSTRAN/faster-whisper) — CTranslate2-based; 4–5× faster than reference Whisper, same quality.
• ★ ★ WhisperX (github.com/m-bain/whisperX) — adds word-level timestamps, speaker diarization (pyannote-audio), VAD pre-segmentation. The production transcription pipeline.
• ★ Distil-Whisper — distilled smaller variants; ~2× faster, slight quality drop.
• Whisper Large-v3 — current best-quality model.
• Whisper Turbo — 2024 release, fast inference, slightly lower quality than Large-v3.

Other STT

• Vosk — older, offline-friendly, smaller models.
• NVIDIA Parakeet / Canary — strong English; CC-BY-NC; Apache for some.
• Moonshine (Useful Sensors) — small, real-time, CPU-friendly.
• Sherpa-ONNX — multi-engine framework.

Text-to-speech (TTS)

★ ★ Piper

• ★ ★ Piper (github.com/rhasspy/piper) — fast, high-quality, runs on a Raspberry Pi.
• MIT license.
• Many voices in many languages, freely downloadable.
• The default TTS for Home Assistant Voice Assist — see smarthome-voice.
• Sub-real-time on most CPUs; acceptable quality for voice-assistant use.

★ Kokoro TTS

• ★ Kokoro 82M (StyleTTS2-based) released late 2024 — extraordinary quality for the size.
• Apache 2.0; runs efficiently on CPU.
• Multiple voices; good prosody.
• Newer competitor to Piper for quality-conscious users.

★ Coqui XTTS / Coqui TTS

• ★ XTTS v2 — voice cloning from short samples; Apache-2.0 weights but with some non-commercial use restrictions on the voice cloning feature.
• High quality; bigger model, GPU helps.
• Coqui (the company) shut down in 2024; community continues maintaining.

★ F5-TTS

• 2024 zero-shot voice cloning model; CC-BY-NC.
• High quality; relatively small.

★ ★ ElevenLabs (paid, mentioned for comparison)

• Closed; paid; the quality gold standard. Honest comparison: significantly better naturalness than open weights for now, but the gap is closing fast (Kokoro / F5-TTS narrow it meaningfully).

Other notable

• OpenVoice (MyShell) — voice cloning; MIT.
• MeloTTS — multilingual; MIT.
• StyleTTS 2 — research model that birthed Kokoro and others.

End-to-end voice assistants

Home Assistant Voice Assist

• ★ ★ HA Voice (rebranded from Year of the Voice 2023–24 push) — fully local voice via Whisper + Piper + an open-weight LLM (Llama / Qwen) via Ollama.
• Hardware: HA Voice Preview Edition (~$60) or DIY with ESP32-S3-Box / M5Stack Atom Echo / Raspberry Pi.
• See smarthome-voice and home-assistant-llm-integrations.

Open Voice OS / Mycroft fork community

• Mycroft AI shut down 2023; OpenVoiceOS continues; smaller community.

LlamaVoice / Voxta

• Newer entrants for personal voice assistants; varied license terms.

Real-time conversational voice

• OpenAI Realtime API (paid) — closed; gold standard for low-latency natural conversation.
• Local equivalents — combinations of Whisper + LLM + TTS streamed; achievable but requires careful pipelining for sub-500ms round-trip; Tier 2+ hardware needed.

What works well in May 2026

• ★ ★ Whisper Large-v3 + faster-whisper for STT — <1× real-time on a 3060+ GPU; quality near-human-transcriptionist.
• ★ ★ Piper for low-latency / always-on smart-home TTS.
• ★ Kokoro for higher-quality TTS at modest CPU cost.
• ★ ★ WhisperX for "transcribe a meeting / podcast" with speakers labeled.

Honest limits

• End-to-end latency. Sub-500ms round-trip (audio in → LLM → audio out) is achievable but not casual; 1–3 seconds is what most local voice assistants feel like in 2026.
• Voice cloning ethics. Don't clone voices without consent.
• Multilingual TTS. Piper handles many languages; quality varies a lot per voice.
• Background noise. WhisperX with VAD pre-filtering helps; perfect ASR in noisy environments is still hard.

Pick this if…

• STT for transcription / dictation: ★ ★ faster-whisper or WhisperX.
• STT in real-time on a CPU: Whisper-base via whisper.cpp or Moonshine.
• TTS for Home Assistant / always-on: ★ ★ Piper.
• TTS for higher-quality narration: Kokoro or Coqui XTTS.
• Voice cloning from a short sample: Coqui XTTS or F5-TTS — with consent.
• End-to-end local voice assistant: Home Assistant Voice Assist + Piper + Whisper + Ollama.