They do need a driver to set up which DMA channel to use, etc. While these devices can go nuts once things are up and running, just plugging it in won't do anything until the computer actually acknowledges it.
So, for example, if one builds a custom Linux kernel without support for Firewire, then plugging in a firewire controller would do nothing? It couldn't access anything?
It can still read and write to memory. If it needs to "initialize" anything, it will have to commandeer the CPU by creatively rewriting memory that is in use.
