So the parent commenter's generalization "at most 2000 books in a lifetime" is hardly accurate - and has been way bypassed by any bookworm I know.
Heck, one can read 2000 books before the time they have kids too. I had already read about that number by 20-25 starting from about 10-11 or so with sci-fi stuff like Verne and Asimov, and moving to literature and non-fiction soon.
I know because I need to pack them every time I move - it's the most labour intensive part of a move to take the library with us, in 100s of cardboard boxes, which then stay unopened for 3-6 months after moving to a new house.
This happens both when I'm listening to audiobooks and doing something else, or when just sitting and doing nothing when listening. I don't have the same problem with reading, I can do that with high focus for hours without any problem.
I think the key here is to listen while doing something undistracting and boring -- so not like driving (which requires your attention often), but rather e.g. on the treadmill or commuting in some longer distance, where you can passively zone one (eg. long train commute, not 3 subway stops).
