Who says the hardware doesn't have a separate IC overriding the ostensibly clean firmware? So you need not only verified hardware schematics, but also verification that the hardware you're running is actually based on that verified design. For which there is currently no way of doing that, as far as I know. You need to either trust the vendor at some level, or treat every device as hostile - while still getting its intended use out of it.