Yes. But keep in mind that the copyright of any contributions you might get will be held by the author of that contribution, not by you. The contributor might license that contribution under the AGPL only, and thereby put a stop to your little scheme. You would have to either get a license from the contributor to your dual licensing scheme, or simply require all contributors to sign the copyright over to you, which is most commonly done in a so-called Contributor Licensing Agreement (CLA). However, CLAs are rightly disparaged for being unfair, in that it lets you, and only you, profit and monopolize what might become a community effort. A requirement of a CLA will most certainly discourage many, if not most, potential contributors from contributing.
I knew about this, that's why I said "sole copyright owner". The goal is to put the code out there and restrict other from modifying it without publishing their changes to their users.
But I want to keep the option of licensing it differently for specific (b2b) clients. It's a not a true open source / free software in spirit as I don't plan to accept outside contributions.
You seem to be assuming that you are the only person who will ever contribute meaningfully to the software. But if, as you probably also hope, the software is widespead and successful, there will most probably arise at least one popular third-party extension/modification. And you will not be able to incorporate this addition into the product which you distribute to your b2b customers; this will make your version of the software inferior to the free version.
Don't let the other commenter dissuade you. There are many successful open source AGPL projects that have CLAs which are not widely disparaged. There's no shame in owning your creation, nor is it immoral to ask contributors to assign their copyright to you. To each their own. Good luck!