Books feed you minimum data ... a string of characters ... completely devoid of any of the senses that you actually imagine you are experiencing: an active colorful audible 3D world.
Your own brain does almost all the work of manufacturing the reality, the author only provides the barest of guides. And your mind also adds layers of meaning onto whatever meaning the author makes explicit.
So I would say the act of reading a book is far from a passive or trivial activity.
In fact, I am hard pressed to imagine another activity so dependent on our own brain's ability for continuous creative production. D&D, scientific research, etc. all happen at a much slower pace.
