Is there any way to use a custom lexicon or vocabulary with Gemini to improve recognition accuracy? If not directly supported, what are practical workarounds people use — e.g. preprocessing prompts, fine-tuning, or combining Gemini with another ASR that supports phrase boosting?
It's not perfect, but it's taken it from being an issue that made all our transcripts look terrible, to an issue I no longer think about.
I imagine just using the second spellchecking pass with Gemini would be almost as effective.
Happy to share more details if helpful.
Feed it that list and the transcript along with a simple prompt along the lines of "Attached is a transcript of a conversation created from an audio file. The model doing the transcription has trouble with company names/industry terms/acronyms/whatever else and will have made errors with those. I have also attached a list of company names/etc. that may have been spoken in the transcribed audio. Please review the transcription, and output a corrected version, along with a list of all corrections that you made. The list of corrections should include the original version of the word that you fixed, what you updated it to, and where it is in the document." If it's getting things wrong, you can also ask it to give an explanation of why it made each change that it did and use that to iterate on your prompt and the context you're giving it with your list of words.
Gemini might have similar capabilities for custom vocabulary, though I'm not certain about their specific implementation. The two-pass ASR+LLM approach could work with Gemini's output as well.
"Transcribe this audio. Be careful to spell the following names and acronyms right: list-goes-here"
Are there constraints where you have to use Gemini ?
Return company name only from dictionary
#dictionary 1:Apple 2:..
And than Vercel AI sdk + Zod Schema + Gemini 2.5 pro and it pretty accurate
# User-Defined Dictionary
Always use the following exact terms if they sound similar in the audio:
```json
{{jsonDictionary}}
```