And those lightweight VM base images are possible because Docker applied a downward pressure on OS base image sizes! Alpine Linux doesn't get enough credit for this; in addition to being a great base image, it was also the first distro to prioritise fast and small image creation (Gentoo and Arch were small, but not fast to create).
There's also the part where you have a much easier time building alpine packages inside a container rather than a VM. There is no docker run equivalent that lets you quickly run a shell script inside the deployment Linux distribution to build the package.
It's not as easy; a block device has to be bootable and so usually bundles a kernel (large). And because the filesystem inside is opaque, you can't do layering like Docker does easily via overlayfs and friends. libguestfs does a heroic job of making VM images easier to manipulate from code, but it's an uphill battle...
