This is just what I do. I don't use Alembic, but I've got a really simple tool that handles migrations. The tool, along with its sequence of migrations is deployed along with the app. The systemd service for Postgres runs the tool after the database is up.
Because the service definition has my migration tool as a dependency, it gets rebuilt when the tool is updated. That causes Postgres to get restarted and the tool to get run IFF there are new migrations. The nix configuration is parameterized, so that in dev or staging environments, the database is completely rebuilt and loaded with test data on each deploy, but in production, we just apply migrations.
With Nixops, this works even on a multi-system deployment. The full deployment typically takes several minutes, but most of that time is in installing the updated packages, which doesn't interrupt the running system. Once everything is installed, the cut-over only takes a second or two. Eventually, I'll have enough traffic to warrant more careful orchestration of the cut-over, but for now it's fine.