Consider:
FROM nodejs
COPY /mycode /app
RUN npm install /app
Now suppose I change my app code. In a Dockerfile situation, the change to the `COPY` invalidates the `RUN npm install /app` layer, even if I didn't change anything that NPM would care about.
An NPM buildpack can signal that there's nothing to change, allowing the overall lifecycle to skip re-building that layer.
There's also the problem of efficient composition. Suppose I have this:
RUN wget https://example.com/popular-shell-script.sh && \
go get https://git.example.com/something-else@abc123 && \
./popular-shell-script.sh && \
rm ./popular-shell-script
And this:
RUN go get https://git.example.com/something-else@abc123
Both of the resulting images will contain the same `something-else` binary and in an ideal world of file-level manifests I could save on rebuilds and bandwidth consumption (NixOS has this, approximately).
But I don't get to do that, because the layers have different overall contents and different digests. Buildpacks don't get you all the way to a file-centric approach, but because they follow a repeatable, controlled pattern of selecting the contents and order of layers, they greatly improve layer reuse between many images.