For NIST we cannot say anything about backdoors. We don't use those curves because we don't trust NIST. Not because we have any prove they are bad.
So to avoid that, there is a parameter selection process that supposedly leaves no room for backdoors, though at some CCC congress DJB described how you could use a similar process to add backdoors.
So basically, EC is based on magic. We cannot prove it is bad, we just have to hope there is no hidden magic.
Note you say 'only marginal room'. Soon the whole world will use exactly one curve. With 'only marginal room for hidden backdoors'.
I feel way more comfortable to know that with RSA what you see is what you get.