AsNoTracking makes the materialization indeed faster, but EF always has a cost associated with it, both for initial use (set up of the context etc, which is noticeable) as well as every other query after that (expression tree parsing etc, not as noticeable especially for recurring queries).
I hope it's one of the pain points they will address and solve during their rewrite to EF7 at one point.
That being said, my opinion is that unless you have really strict performance and/or latency requirements, EF should be good enough. At least for simpler queries and in 'longer-running' applications :)