except with docker youre managing two userlands, the underlying host and the container, instead of one.
I've used packer a bunch of times, generally for things that dont change frequently. For instance, for creating the image the docker containers run on.
Or for one-off utility servers that are just running third party code that doesnt support docker. For the thing that id work on and deploy every day, id want something else on top whether thats docker/puppet/codedeploy/etc - probably docker today.
Some of the most business critical code ive worked with was built with packer, because the extra complication of docker added no value, we wanted as little software on the server as possible since each added thing could fail, and we wanted fast bootup times and minimal launch scripts for autoscaling.