preventing people with this belief from sprinkling ORM code all throughout the codebase is reason enough to ban the use of an ORM.
The specifics of how you retrieve data is an implementation detail. If someone wants to use an ORM have at it, but don't sprinkle it throughout the codebase, place it behind well defined functionality (behind a system dedicated to pulling data).
The fact that ORM's have their own query language should be all you need to know.
One could easily argue that you should be able to manually interact with db readers all throughout the codebase, after all, it's an abstraction.
But no one would ever actually argue that. Like such forward-only db readers, ORM's are a way to pull data, they are not the abstraction layer for pulling data for your application.