What I've experienced however, is that this isn't as easy as I expected. The "middle ground" here is between "inventing form whole cloth a bespoke library to solve AuthZ" and "do absolutely nothing whatsoever new". Using CASL in my infra's middleware to solve AuthN seems to be working for me. I can write the rules in a way CASL understands, and I was able to quickly implement the middleware without having to do anything crazy.
As for "JWT structure being inconsistent" I mean to say that often I'm asked to "connect" many different providers, e.g. Google social sign-in, Okta, Auth0, etc., all of whom have their own way of structuring their JWT payloads (beyond the standard that is), and using Cognito as an intermediary has been helpful to avoid all of that, as again, I'm trying to do as little as possible myself.