YouVerify vs Prembly vs Smile ID: Building a Multi-Provider Identity Verification Engine in Go

The Case Study: Onboarding Stalls During CBN's KYC Mandate

In December 2023, the Central Bank of Nigeria (CBN) issued a circular mandating that all funded bank accounts and digital wallets must be linked to a verified Bank Verification Number (BVN) and National Identification Number (NIN). For a savings and micro-investment client we were advising—handling roughly 450,000 active wallets—this meant retrofitting their entire sign-up flow and clearing a backlog of unverified user profiles within weeks.

Their existing setup relied on a single identity vendor (Prembly) to validate customer BVNs and NINs against NIBSS and NIMC databases. Within 72 hours of the mandate going live, API response times spiked from 800ms to over 12 seconds. Mornings were brutal: between 8:00 AM and 11:00 AM West Africa Time (WAT), HTTP 504 Gateway Timeouts surged to 38%.

When a user typed their BVN into the mobile app and hit 'Submit', the frontend hung until the gateway timed out. The user dropped off, tried again five minutes later, and created a self-inflicted stampede that further overwhelmed the upstream provider. Onboarding conversion fell from 84% to 43% in a single week.

We were called in to redesign the identity pipeline so that upstream database outages at NIMC or vendor API degradations would not paralyze customer onboarding.

Constraints and Initial Failures

Fixing identity verification in Nigeria comes with severe technical and regulatory constraints:

1. Upstream Fragility: Vendors like YouVerify, Prembly, and Smile Identity do not own the underlying data. They act as wrappers around NIMC (for NIN) and NIBSS (for BVN). If NIMC's core database experiences network degradation, every provider depending on NIMC degrades simultaneously.
2. NDPR Strictness: The Nigeria Data Protection Act (NDPA) prohibits storing raw BVN/NIN values in plain text or using them as unencrypted lookup keys in cache stores like Redis.
3. Name String Discrepancies: A user named Oluwaseun Ibrahim Adebayo might be registered as Adebayo Oluwaseun in NIBSS, but Oluwaseun I. Adebayo in NIMC. Naive exact-string matching causes legitimate users to fail KYC checks.
4. Cost Multipliers: Calling three providers simultaneously for every request triples unit economics instantly, burning thousands of dollars in redundant API credits.

What Failed First: Naive Sequential Fallbacks

The client's internal team initially attempted a naive synchronous retry queue in Python:

Try Prembly -> (if error or timeout after 5s) -> Try YouVerify -> (if error after 5s) -> Try Smile ID

This architectural pattern failed catastrophically in production. When NIBSS went slow, a user request would wait 5 seconds on Prembly, fail, wait 5 seconds on YouVerify, fail, and then wait 5 seconds on Smile ID before throwing an error to the mobile client.

The cumulative 15-second latency exceeded the HTTP client timeouts on both iOS and Android. Mobile apps severed the TCP connection, while the backend continued executing the background calls, consuming thread pools and database connections. Similar to what we observed when PostgreSQL Transaction Pooling Breaks Prepared Statements: PgBouncer vs AWS RDS Proxy Under 20,000 Payday Concurrency, holding connections open while waiting on sluggish external downstream dependencies rapidly exhausted backend resources.

The Architecture: Hedged Concurrent Verification with Adaptive Circuit Breakers

We abandoned sequential fallback logic entirely. Instead, we re-architected the verification engine in Go using a Hedged Request with Dynamic Fallback pattern, governed by stateful circuit breakers and real-time vendor latency scoring.

                    +--------------------------------+
                    |   Incoming Verification Request |
                    +---------------+----------------+
                                    |
                                    v
                    +--------------------------------+
                    |   Go Verification Engine       |
                    |   - HMAC Lookup Cache          |
                    |   - Redis Health Score Evaluator|
                    +---------------+----------------+
                                    |
          +-------------------------+-------------------------+
          | (Primary: Score 90+)    | (Hedged t+400ms)        | (Circuit OPEN)
          v                         v                         v
  +---------------+         +---------------+         +---------------+
  |  YouVerify    |         |   Prembly     |         |   Smile ID    |
  |  Circuit CLOSED         |  Circuit CLOSED         |  Circuit OPEN |
  +-------+-------+         +-------+-------+         +---------------+
          |                         |                         |
          +-------------------------+-------------------------+
                                    |
                                    v
                    +--------------------------------+
                    | First Valid Response / Cancel  |
                    | Jaro-Winkler Name Matching     |
                    +---------------+----------------+
                                    |
                                    v
                    +--------------------------------+
                    |   AES-256 PII Vault Storage    |
                    +--------------------------------+

1. Redis Health Scoring & Circuit Breaking

Every provider client is wrapped using the sony/gobreaker library. We keep track of provider latency and failure rates over sliding 60-second windows in Redis. Each provider is assigned a dynamic score from 0 to 100 based on three metrics:

• Successful response rate (50% weight)
• Latency under 1.5 seconds (30% weight)
• Exact match confidence score (20% weight)

When a provider's error rate exceeds 25% or P95 latency breaches 3,500ms, its circuit breaker flips to OPEN. The engine stops sending requests to that vendor for 30 seconds, routing traffic to healthy alternatives.

2. Hedged Request Execution

Instead of making calls sequentially or broadcasting to all providers at once, we execute hedged requests:

1. The engine fetches the current highest-scoring provider (e.g., YouVerify).
2. It dispatches an asynchronous HTTP request to YouVerify.
3. A timer (time.NewTimer(400 * time.Millisecond)) starts concurrently.
4. If YouVerify responds within 400ms with a definitive MATCH or NO_MATCH, the context is canceled, and we return the result immediately.
5. If YouVerify has not responded after 400ms, the engine fires a second request to the runner-up provider (e.g., Prembly) without canceling the first.
6. Whichever provider responds first with a non-5xx payload wins. The context for the other request is canceled immediately.

This keeps median latency locked at under 500ms while capping P99 latency at ~1.2 seconds, even when a vendor drops requests.

3. PII Vaulting and HMAC Indexing

To comply with NDPA regulations without sacrificing cache hits, we drew from the secure indexing patterns built during our work on Deterring Virtual Account Triangulation Fraud: Building a Deterministic Risk Engine and NDPR PII Vault in Go.

Raw BVNs and NINs are never stored in log files, trace contexts, or plaintext database columns. Incoming BVN/NIN numbers are hashed using HMAC-SHA256 with a pepper stored in AWS KMS. This hash acts as an exact-match lookup key in Redis. The encrypted payload containing name, DOB, and phone number is encrypted via AES-256-GCM before writing to storage.

Go Implementation: Hedged Provider Runner

Below is a simplified version of the core Go worker handling the hedged requests with context cancellation:

package main

import (
	"context"
	"errors"
	"sync"
	"time"
)

type KYCResponse struct {
	Provider  string
	FirstName string
	LastName  string
	DOB       string
	Matched   bool
	Error     error
}

type ProviderFunc func(ctx context.Context, idNumber string) (*KYCResponse, error)

func ExecuteHedgedVerification(ctx context.Context, idNumber string, primary ProviderFunc, secondary ProviderFunc) (*KYCResponse, error) {
	ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
	defer cancel()

	ch := make(chan *KYCResponse, 2)
	var wg sync.WaitGroup

	// Launch Primary Provider
	wg.Add(1)
	go func() {
		defer wg.Done()
		res, err := primary(ctx, idNumber)
		if err == nil && res != nil && res.Matched {
			select {
			case ch <- res:
			case <-ctx.Done():
			}
		}
	()()

	// Wait timer for hedging trigger
	timer := time.NewTimer(400 * time.Millisecond)
	defer timer.Stop()

	select {
	case res := <-ch:
		return res, nil
	case <-timer.C:
		// Primary took longer than 400ms; trigger secondary provider
		wg.Add(1)
		go func() {
			defer wg.Done()
			res, err := secondary(ctx, idNumber)
			if err == nil && res != nil && res.Matched {
				select {
				case ch <- res:
				case <-ctx.Done():
				}
			}
		}()
	case <-ctx.Done():
		return nil, errors.New("kyc verification timed out across all providers")
	}

	// Second chance select if timer fired but secondary returned first
	select {
	case res := <-ch:
		return res, nil
	case <-ctx.Done():
		return nil, errors.New("kyc verification failed: timeout or invalid match")
	}
}

Handling Name Discrepancies with Jaro-Winkler Distance

Upstream responses rarely return names formatted cleanly. If the user enters Oluwaseun Adebayo on the frontend, but the BVN query returns Adebayo Oluwaseun Ibrahim, an exact string comparison (==) will fail, blocking a valid customer.

We implemented a two-stage field matching pipeline:

1. Normalization: Convert all strings to uppercase, remove special characters, strip common prefixes/titles (MR, MRS, DR), and split the name into tokenized sets.
2. Jaro-Winkler Distance Calculation: Compare permutation pairs between user input tokens and BVN/NIN payload tokens.

If the Jaro-Winkler metric between full token sets is >= 0.88, we mark the identity as verified. If it falls between 0.70 and 0.87, we trigger a secondary non-blocking verification step (requesting a selfie for liveness/facial matching via Smile ID) instead of rejecting the account outright.

Concrete Outcomes and Performance Metrics

Deploying the Go verification engine completely reshaped the client's onboarding stability within 14 days of rollout:

• P99 Verification Latency Dropped by 88%: Down from 8,400ms during peak morning load to 620ms.
• Onboarding Conversion Recovered: User onboarding completion rate rose from 43% to 94.2%, exceeding pre-mandate benchmarks.
• Zero Timeout Cascades: During a 2-hour NIMC database outage in mid-January 2024, the engine's circuit breaker automatically isolated failed NIN endpoints, alerted the operations dashboard, and fell back to cached verified HMAC sets and liveness checks without dropping customer HTTP connections.
• Cost Reduction: By leveraging Redis health scores to prioritize lower-cost, high-reliability providers first and reserving expensive biometric fallbacks strictly for low name-match confidence scores (0.70 - 0.87), monthly provider billings fell by 28%.

Playbook: Engineering Resilient KYC Integrations in West Africa

If you are scaling a fintech application across Nigeria or the broader West African region, do not treat identity providers as simple HTTP endpoints. Treat them as unstable networks.

Provider API Landscape Matrix

| Vendor | Primary Strengths | Standard Response Latency (P50) | Handling of Upstream Downtime | Cost Profile | Name Match Capability | | :--- | :--- | :--- | :--- | :--- | :--- | | YouVerify | Excellent BVN/NIN data depth; fast webhooks. | 450ms - 900ms | Returns explicit status error codes (422, 503). | Mid-tier; volume discounts available. | Strict; requires client-side normalization. | | Prembly (Identitypass) | Broad regional footprint (Nigeria, Ghana, Kenya). | 600ms - 1,400ms | Retries internally; can lead to prolonged response tail latency. | Highly competitive per-lookup pricing. | Offers basic string-similarity flags in payload. | | Smile ID | Industry standard for biometric liveness & document verification. | 800ms - 1,800ms | Graceful degradation; fallback to async webhooks. | Premium pricing; best reserved for fallback/high-risk flows. | Built-in facial/biometric match confidence score. |

Step-by-Step Implementation Strategy

1. Never rely on single-provider SDKs. Wrap all HTTP calls in native language interface abstractions (like Go interfaces or TypeScript types). You must be able to swap or disable a provider via a feature flag without redeploying backend binaries.
2. Decouple Provider Execution from HTTP Response Loops. Use background queues and context timeouts liberally. If an upstream identity call takes longer than 2.5 seconds, yield control back to the user app with a status of VERIFICATION_PENDING and process completion asynchronously via webhooks or background polling.
3. Decentralize Your Match Thresholds. Do not rely on provider boolean is_match flags. Ask vendors for raw payload strings, strip formatting, run localized tokenization algorithms (such as Jaro-Winkler or Levenshtein Distance), and log score distributions to fine-tune your risk parameters over time.
4. Audit Your Storage Layer for PII. Ensure BVNs, NINs, and phone numbers are never stored in plain text. HMAC-SHA256 indexing combined with AES-256-GCM envelope encryption satisfies NDPA requirements while allowing high-speed lookup cache operations.

When building fintech infrastructure under shifting regulatory conditions, your system architecture must assume that core banking rails and state identity registries will drop connections without warning. Building multi-provider redundancy from day one is not over-engineering—it is the baseline cost of running a reliable financial service in West Africa.