AI & Technology
Gemini Models Tested: Which One Is Right for Call Center Transcription?
We ran the same real call center recording through 7 Gemini model variants — spanning the 2.5 and 3.x generations — and compared the transcripts side-by-side on latency, cost, speaker diarization, PII redaction, and name consistency. The results were stark.
Why This Comparison Exists
Call Coach IQ uses Gemini via Google's Genkit pipeline to transcribe, diarize, and analyze every call that flows through the platform. Choosing the right model is not an abstract API decision — it directly determines whether a transcript separates the agent from the customer correctly, whether PII is redacted before it hits storage, and how quickly a scored coaching note lands in a manager's dashboard.
We took a real call center recording — a Pioneer Connect customer service call with account data, PIN entry, and IVR interaction — and ran it through all seven current Gemini model variants across both the 2.5 and 3.x generations. Same audio, seven transcripts, compared directly.
Four findings were significant enough to highlight before the full breakdown: both Flash-Lite variants share the same diarization inversion bug, PII redaction is unreliable across most models, name consistency reverts in 6 of 7 models, and the Pro-tier cost premium is not justified by the accuracy results.
Results at a Glance
| Model | Latency | Cost/min | Utterances | Diarization | Redaction | Name Consistency |
|---|---|---|---|---|---|---|
| Gemini 3.1 Pro (Preview)Not recommended | 75.6s | $0.0096 | 51 | ✓ Correct | ✗ Missed | ✓ Correct |
| Gemini 3 Flash (Preview)★ Recommended | 31.9s | $0.0023 | 45 | ✓ Correct | ✓ Redacted | ⚠ Reverts |
| Gemini 3.1 Flash-LiteNot recommended | 10.0s | $0.0011 | 46 | ✗ Inverted | ✗ Missed | ⚠ Reverts |
| Gemini 3.5 Flash | 46.5s | $0.0076 | 56 | ✓ Correct | ✓ Redacted | ⚠ Reverts |
| Gemini 2.5 Flash★ Recommended | 27.9s | $0.0013 | 57 | ✓ Correct | ✓ Redacted | ⚠ Reverts (garbled) |
| Gemini 2.5 Flash-LiteNot recommended | 8.7s | $0.0004 | 38 | ✗ Inverted | ✗ Missed | ⚠ Reverts |
| Gemini 2.5 Pro | 56.3s | $0.0070 | 67 | ✓ Correct | ✗ Missed | ⚠ Reverts |
7 independent transcriptions of the same Pioneer Connect support call. See methodology section for full details.
Four Findings That Define This Comparison
Flash-Lite Diarization Inversion Persists Across Both Generations
Both Gemini 2.5 Flash-Lite and Gemini 3.1 Flash-Lite exhibit the same speaker-label inversion: after the opening turn, labels flip and persist for the rest of the call. This is not a generational fix — the same failure mode appears in both the 2.5 and 3.x Flash-Lite variants. Avoid Flash-Lite in any pipeline where speaker attribution drives downstream outputs.
PII Redaction Is Unreliable Across Most Models
Five of the 7 models miss at least one sensitive field. Account number redaction is missed by Gemini 3.1 Pro, 3.1 Flash-Lite, 2.5 Flash-Lite, and 2.5 Pro. Card number redaction is missed by all models except Gemini 3.5 Flash. Only Gemini 3 Flash (Preview) and Gemini 3.5 Flash reliably redact account-level PII without a secondary pass. Do not assume model-level redaction is complete without verifying against your specific PII field set.
Name Consistency Reverts in 6 of 7 Models
Six of the 7 models revert agent names after initial correct capture — outputting a generic role label (e.g., "Agent") in place of the actual name for subsequent turns. Only Gemini 3.1 Pro (Preview) maintains correct name consistency throughout. For coaching workflows that rely on entity-resolved speaker names, a post-processing normalization step is required for all other models.
Pro-Tier Cost Premium Not Justified by Accuracy
Gemini 3.1 Pro (Preview) costs $0.0096/min and Gemini 2.5 Pro costs $0.0070/min. Neither outperforms Gemini 3 Flash (Preview) at $0.0023/min on the criteria that matter most for call center transcription — diarization and PII redaction. Gemini 3.1 Pro misses account number redaction despite costing four times as much. The Pro tier offers higher utterance counts and name consistency, but those gains do not justify the price for most production workloads.
Per-Model Breakdown
Gemini 3 Flash (Preview)
★ RecommendedVerdict: Recommended — best cost/accuracy balance
Gemini 3 Flash (Preview) delivers the best overall balance of speed, cost, and accuracy across the 7 models tested. At $0.0023/min and 31.9 seconds of latency, it correctly diarizes all speakers and redacts the account number without a secondary pass.
The one gap is name consistency: after initially capturing the agent name correctly, the model reverts to a generic label for subsequent turns. This is a consistent pattern across most models in this evaluation and is addressable with a lightweight post-processing step.
For most call center pipelines — regulated or otherwise — Gemini 3 Flash (Preview) is the primary recommendation. It avoids the diarization inversion that plagues both Flash-Lite variants, provides redaction without the cost premium of Pro-tier models, and processes a typical call in under 35 seconds.
Gemini 2.5 Flash
★ RecommendedVerdict: Recommended — best GA alternative
Gemini 2.5 Flash is the recommended generally-available fallback for pipelines that cannot use preview models. At $0.0013/min, it is the lowest-cost model with correct diarization. It captured the most utterances of the non-Pro models — 57 — and correctly redacted the account number.
Name consistency reverts with some transcription noise on reversion, which is a mild quality concern for coaching workflows that depend on clean speaker attribution strings. The diarization labels themselves are correct throughout, so the underlying speaker separation is sound.
If your procurement policy requires GA-only models, Gemini 2.5 Flash is the clear choice over 2.5 Pro (higher cost, misses redaction) and 2.5 Flash-Lite (diarization inversion, content loss).
Gemini 3.5 Flash
Verdict: Recommended for best redaction — higher cost
Gemini 3.5 Flash is the only model that correctly redacted both the account number and the card number in this evaluation. If maximum PII coverage is the primary requirement, it is the right pick.
The cost is $0.0076/min — significantly higher than Gemini 3 Flash (Preview) at $0.0023/min — and latency is 46.5 seconds. Diarization is correct throughout. Name consistency reverts in the same pattern as other models.
For regulated industries where missing a card number redaction carries compliance or legal exposure, the cost premium is likely justified. For general call center transcription, Gemini 3 Flash (Preview) covers the account number case at a fraction of the price.
Gemini 2.5 Pro
Verdict: Not recommended — high cost, misses redaction
Gemini 2.5 Pro produced the highest utterance count in the evaluation at 67 — indicating it captures the most granular turn-level detail. Diarization is correct throughout. However, it misses account number redaction despite apparently capturing the correct numeric value in the transcript, which suggests a possible hallucination or redaction-logic gap rather than a transcription failure.
At $0.0070/min, it is the second most expensive model tested and costs more than five times Gemini 3 Flash (Preview), which outperforms it on redaction. The cost premium is not justified by accuracy gains in this evaluation.
If complete transcript granularity matters more than redaction coverage, 2.5 Pro is the strongest GA-only option. Add a secondary redaction layer for any PII-sensitive content.
Gemini 3.1 Pro (Preview)
Not recommendedVerdict: Not recommended — cost not justified
Gemini 3.1 Pro (Preview) is the only model in this evaluation with correct name consistency throughout — agent names are correctly attributed from first mention to last. It also achieves correct diarization. However, it misses account number redaction, which is a material gap for payment-related call center use cases.
At $0.0096/min, it is the most expensive model tested — more than four times the cost of Gemini 3 Flash (Preview). Given that it fails on redaction while costing more, it does not justify the price premium for typical call center transcription workloads.
The correct name consistency behavior is a genuine differentiator if your pipeline does downstream entity resolution against agent names. Outside that specific requirement, the cost-to-accuracy ratio is unfavorable.
Gemini 3.1 Flash-Lite
Not recommendedVerdict: Not recommended — diarization inversion
Gemini 3.1 Flash-Lite exhibits the same systemic diarization inversion as Gemini 2.5 Flash-Lite: after the opening speaker turn, labels flip and persist for the remainder of the call. This is not a one-off mislabel — it is a consistent failure that affects every subsequent coaching note, compliance flag, and scored output derived from the transcript.
It also misses account number redaction. At $0.0011/min, it is the second cheapest model, but the diarization failure alone disqualifies it from any production deployment where speaker attribution matters.
The speed advantage — 10.0 seconds latency — is real, but it does not compensate for the reliability gap. Use Gemini 3 Flash (Preview) instead at twice the cost but with correct diarization.
Gemini 2.5 Flash-Lite
Not recommendedVerdict: Not recommended — diarization inversion and content loss
Gemini 2.5 Flash-Lite is the fastest and cheapest model in this evaluation at 8.7 seconds and $0.0004/min, but it carries the most failures. Diarization is inverted — speaker labels flip after the first turn and persist throughout. It misses account number redaction and partially leaks the card number. It also produced the fewest utterances (38), indicating content is being dropped.
The combination of diarization inversion, redaction failure, and content loss makes 2.5 Flash-Lite unsuitable for any call center transcription pipeline where transcript quality, speaker attribution, or PII handling matter.
This is the model to avoid regardless of cost constraints. Even for non-regulated, low-PII-sensitivity workloads, the content-drop problem means you are working with an incomplete picture of the call.
Which Model Should You Use?
The right model depends on your call type, compliance requirements, GA policy, and tolerance for preview APIs. This matrix maps the most common call center scenarios to a recommendation based on what we observed.
| Your Scenario | Recommended | Fallback |
|---|---|---|
| General call center transcription (regulated or standard) | Gemini 3 Flash (Preview) Best cost/accuracy balance. Correct diarization, account number redaction, and 31.9s latency at $0.0023/min. GA fallback is 2.5 Flash. | Gemini 2.5 Flash |
| Maximum PII redaction coverage (card + account numbers) | Gemini 3.5 Flash Only model that redacts both account and card numbers natively. Higher cost at $0.0076/min but no secondary pass required. | Gemini 3 Flash (Preview) + secondary redaction |
| GA-only procurement policy | Gemini 2.5 Flash Lowest-cost GA model with correct diarization and account number redaction. 2.5 Pro captures more utterances but misses redaction and costs more. | Gemini 2.5 Pro |
| Name-entity accuracy required (agent name resolution) | Gemini 3.1 Pro (Preview) Only model with consistent name accuracy throughout. Add a secondary redaction layer for account numbers. High cost — evaluate against the value of clean entity strings. | Any model + post-processing normalization |
| Avoid Flash-Lite in any configuration | Gemini 3 Flash (Preview) Both Flash-Lite variants (2.5 and 3.1) exhibit systemic diarization inversion. The cost savings do not offset the transcript reliability gap. | Gemini 2.5 Flash |
How Call Coach IQ Uses These Models
Call Coach IQ currently ships Gemini 3 Flash (Preview) as the default transcription model, with Gemini 2.5 Flash as the GA fallback. The full pipeline runs transcription, speaker diarization, PII redaction, and AI scoring in a single Genkit flow — so diarization and redaction failures at the model layer compound into every downstream output: coaching notes, compliance flags, and performance scores.
The findings in this evaluation directly informed that model selection. The diarization inversion in both Flash-Lite variants would produce coaching notes scored against the wrong person. The redaction gaps in Pro-tier models would route unredacted payment data into storage and analytics pipelines at higher cost. Neither failure mode is acceptable in production.
We will update this comparison as the 3.x generation moves from preview to GA, as pricing changes, and as new models become available. Open questions we are tracking: does Gemini 3 Flash (Preview) pricing hold at GA? Does Gemini 3.1 Pro name consistency generalize beyond this call type? If you are evaluating models for your own pipeline, we recommend running the same real call through all candidates rather than relying on synthetic benchmarks alone.
Methodology
- Test methodology: a single real call center recording (Pioneer Connect support call) was transcribed independently by all 7 model variants. Results were compared side-by-side.
- Call type: customer service / collections — two-party call (agent + customer) followed by IVR system interaction.
- Call content: included account number disclosure, PIN entry, credit/debit card processing, and payment verification code — a realistic PII-dense scenario.
- Speaker configurations: 2-party (human-to-human), IVR system, and hold queue sections were all present.
- PII ground truth: manually identified fields evaluated for redaction — account number, card number, and payment verification code.
- Latency: measured as end-to-end transcription time for the same audio file per model.
- Cost/min: based on published pricing at time of evaluation (July 2026); preview model pricing may change at GA.
- Utterances: total speaker turn segments produced by each model for the same recording.
- Diarization: assessed by matching labeled turns against the known ground-truth speaker for each line.
- PII redaction: assessed against manually identified sensitive fields; "Missed" means at least one field appeared in plaintext.
- Name consistency: assessed by whether the model maintained the correct agent name after initial capture or reverted to a generic role label.
- This is a qualitative evaluation on a representative call, not a large-scale quantitative benchmark. Results reflect performance on this call type and may vary across call configurations.
- Open questions: Does 3 Flash (Preview) pricing hold at GA? Does 3.1 Pro name consistency generalize across call types? We will update this article as these questions resolve.
See How Call Coach IQ Scores Calls with Gemini
Call Coach IQ uses Gemini 3 Flash (Preview) and 2.5 Flash to transcribe, diarize, redact, and score every call automatically — with results available within minutes of a call ending. See it live on your call type.
Request a DemoRelated: Automatic Call Transcription for QA →
Call Recording & Transcription Software → · AI Implementation Guide →

