Whisper-Large-v3 vs Deepgram Nova-2 vs Azure Speech: STT Benchmarks for Nigerian Pidgin and Yoruba Voice Agents

Building interactive voice agents for Nigerian users is one of the toughest challenges in applied machine learning today. It is not just about converting sound waves to text; it is about surviving terrible acoustic environments. When a merchant in Computer Village calls your AI customer support line, the audio does not come in as pristine 44.1kHz FLAC audio over fiber optic internet. It arrives as an 8kHz AMR-NB mono stream recorded on a budget Android phone, transmitted over a congested 3G network, with a 5kVA generator humming loudly in the background.

To make matters more complex, Nigerian spoken communication rarely sticks to pure Queen's English. Users mix English with Nigerian Pidgin, Yoruba, Igbo, or Hausa within a single sentence—a phenomenon known as code-switching. Standard Speech-to-Text (STT) models trained on Western conversational datasets fall apart when hit with "I don send the money since morning but my account never credit."

At Neobot Tech, we benchmarked three top-tier STT engines—Deepgram Nova-2, self-hosted Whisper-Large-v3, and Azure AI Speech—on 500 real-world Nigerian voice recordings. We evaluated them across four critical vectors: Word Error Rate (WER) on Pidgin and Yoruba code-switching, inference latency (Real-Time Factor), bandwidth cost, and resilience against audio degradation.

The Contenders: Cloud API vs. Self-Hosted Infrastructure

Before evaluating performance numbers, let us clarify what each engine brings to the table for engineering teams operating out of Lagos, Abuja, or Ibadan.

1. Deepgram Nova-2

Deepgram's flagship STT model is built specifically for low-latency streaming applications. It uses an end-to-end deep learning architecture designed to process raw audio without relying on traditional heavy autoregressive decoder passes. Deepgram provides direct cloud API access with WebSocket streaming support out of the box.

2. Self-Hosted Faster-Whisper (Whisper-Large-v3)

OpenAI's Whisper-Large-v3 is widely regarded as the gold standard for open-source multilingual transcription. However, running the official PyTorch implementation in production is too slow for real-time applications. Instead, we host faster-whisper—a reimplementation using CTranslate2, a fast inference engine for Transformer models—deployed on an NVIDIA RTX 4090 GPU server housed locally in a Lagos colocation facility to minimize network round-trips.

3. Azure AI Speech

Microsoft's flagship speech platform has long been favoured by enterprise financial institutions due to its robust custom language model features, multi-region failover capabilities, and native support for African accented English options.

---

Benchmark Comparison Matrix

The table below summarizes our benchmark results from running 500 audio samples collected from real customer calls across Lagos fintech and logistics workflows. Audio was downsampled to 8kHz to simulate actual telephony environments.

| Criteria | Deepgram Nova-2 | Self-Hosted Faster-Whisper (v3) | Azure AI Speech | | :--- | :--- | :--- | :--- | : | Pidgin Word Error Rate (WER) | 14.2% | 18.6% (11.1% with prompt) | 22.8% | | Yoruba Code-Switch WER | 21.4% | 16.8% | 25.1% | | Real-Time Factor (RTF) | 0.08 | 0.18 (FP16 on GPU) | 0.22 | | First-Byte Latency (TTFT) | 180 ms | 420 ms | 310 ms | | Cost per 1,000 Min | $4.30 | $1.10 (at 60% GPU util) | $16.00 | | Bandwidth Requirement | High (Cloud API Stream) | Zero (Local/LAN execution) | High (Cloud API Stream) | | Custom Vocabulary Support | Excellent (Keywords/Boost) | Superior (Initial Prompting) | Good (Phrase Lists) | | Offline/Air-Gapped Capable | No | Yes | No |

---

Accuracy Deep Dive: Code-Switching, Dialects, and Noisy Audio

Transcription accuracy in West Africa cannot be measured using standard English speech datasets like LibriSpeech. Accuracy lives or dies on how well the model handles regional phrasing, local slang, and acoustic noise.

Handling Nigerian Pidgin and Slang

Deepgram Nova-2 surprised us with its out-of-the-box comprehension of Nigerian Pidgin terms. Words like "abeg", "kobo", "moniepoint", "pos", and "upneppa" were correctly recognized without additional fine-tuning. Deepgram achieves this by combining acoustic token generalization with contextual vocabulary boosting, allowing developers to pass key terms in the API request header.

Standard Whisper-Large-v3 often struggles with Pidgin when left unguided, frequently attempting to translate Pidgin words into formal English (e.g., transcribing "I wan transfer" as "I want to transfer"). However, when we passed a initial contextual prompt—`