poll(), select() et al don't define "readable" as meaning "a byte of data is available". They define it as "read() will not block".
When you read to the end of a file, read() returns "". This is the same as when you try to read from a closed socket. Conceptually, they are linking the concepts of "EOF" and "closed", not the concepts of "EOF" and "waiting for data". And indeed, if you call poll() on a socket that has been closed on the remote end, you will find it is "readable".
Ultimately, regular files just were never intended to be used as pipes. The abstractions just weren't chosen to work in that way.