NextJS does more than create-react-app, but you're not required to use all those extra features. You can use it to just escape the tedium of configuring JS tooling.
I spent 4 hours trying to figure out how to build a static nextjs website. I ended up giving up and switching to astro. You basically need to understand SSR to do static sites.
