From what I've seen, invokers without extra code are designed for client side interactivity (e.g. showing a modal), which is orthogonal to the (correct, IMO) use case of HTMX. You shouldn't use HTMX to turn every custom user interaction into a network round trip, you should use it instead of fetching JSON. If a user interaction wouldn't trigger a network request if you built your app with react/vue/svelte/{insert_framework_of_choice}, then it shouldn't trigger one if you build it with HTMX either.
You can add custom commands with invokes and custom JS, but at that point you're basically re-implementing HTMX yourself, and I'm not sure what the advantage of that would be over hx-trigger="command" (especially when you also consider using fixi if you don't need all the features of HTMX). There's also the triptych proposals [2] from one of the HTMX maintainers which would add some of HTMX's behaviors to HTML, but those are a long way from being baseline, and may get there.