Apps are getting larger due to more and more code accumulating as time goes on and apple's toolkit / compilers causing binaries to become larger.
Swift itself creates incredibly bloated binaries compared to objective-c. More people are using more 3rd party libraries than 5 years ago, where you just used apple's built in libraries for the most part. If you just used the shared system libraries, then that didn't count against your binary size.