The ordinary experience for someone viewing a magic act is that something impossible has happened. The fact that we know, logically, that it cannot be the case doesn't change the fact that what we experience is impossible. It's unexplainable by the majority of viewers.

If the argument is "magic doesn't exist, therefore what you experienced is entirely normal and unremarkable" then frankly the interlocutor just don't get the point of why people watch magic.