This has only ever made sense when Django is not being used as a web framework. A person may want to use just the templating engine in a script, or the ORM in another framework. I could see how installing all of Django in those scenarios to use some small portion of it could be considered bloat.
It is imaginable that Django the web framework could be made up of sub-packages for the admin, templates, ORM, etc, but it was created at a time where Python package managers didn't really exist. Being a single package was (and probably still is) easier.