The level of none compliance was actually pretty surprising to me. You're right speed is not really that important, realistically you're not going to have a large amount of TOML in any given project, nor are you going to read and write it constantly.
One issue I've run into is programs that have configuration language that looks a lot like TOML or INI format... yet it's neither. So a TOML or INI parser might be able to read and write the configuration files 90% or the time, but fail in strange ways in some edge cases.