Extremely high?

I wrote a stand-alone commercial DSP app which seems to still be sold ten years later. There might be security issues, there probably are since it hasn't been recompiled in years as far as I know, but I'm certain they aren't in the DSP parts.

DSP code involves large blocks of numbers which you translate into other numbers. If you're writing a format or facility within ffmpeg, you aren't directly reading or writing to files, or connecting to the Internet. It's probably that the whole API you write to is just blocks of numbers and some sort of data description.

ffmpeg as a whole might have security violations in it but adding new plug-ins or format won't necessarily increase those chances.