It's not silly. If you disconnect from the internet and you can no longer produce the same level of art that's all the proof that's required. Clearly you are sourcing it from somewhere, not your own brain.
I can prove to you that my writing and my code are mine because you can stand behind me and look over my shoulder to see that I am creating it, one bit (or at most 8) at the time. Visual evidence of it being a copy is not required: what's required is a track of creation aka provenance. This is a very well defined area in copyright. So the proof would be trivial: you recreate your work again, without access to Stable Diffusion or equivalent, while being monitored and if you can not then that would count as a bust in my book. If you're a good enough artist that you can do that and you merely use SD as a way to get some inspiration then that's another matter. But if the output of SD is in the workflow in such a way that its output is directly your input then that would be troublesome (to me). If it is copied then that still would be troublesome (to me). If you feel ok with that it is entirely up to you, but I have my views on that, which I'm perfectly at liberty to share.
And of course once you've made it work likely it is trivial to make it again so probably the above test conditions would need some sharpening but you get the principle.
Without provenance you are still creating art, but you are not creating original art. And if you were a good enough artist before SD then you should still be after, even if you're not using it and everybody else is, at least your work will be and will continue to be original.
I'm trying to imagine a world in which Rembrandt van Rijn admitted to using SD to create the Nightwatch, projecting it on a big canvas and then erasing the prompt. I figure it wouldn't give me quite the same feeling that it does today. Of course artists will use the tools that are at their disposal, but this tool essentially is a pocket sized library of all the other art that could be vacuumed up into the model and that just doesn't sit right with me. Which parts are yours and which parts are SD? If you claim an outsider can't un-entangle them to the point where cause and effect are separated do you really feel that that is the bar that should be met? If so what if I trained a model on your art exclusively and then used that to produce prompt driven 'art' to compete with you, would you think that's fair? And if not, why would the number of inputs be a factor? It's the principles that matter.
FWIW my dad made a living as a painter for quite a few years and even though I can't paint worth anything I know enough about brush technique to get oil to stick to canvas. Given this tool I could produce 'art' that I probably could pass off as original and it would likely not be detected by your standard.
But without the tool I would be absolutely hopeless so that's a pretty good argument. The more of an artist someone is the more brittle that argument becomes, there may well be a border beyond which what someone creates using such a tool does qualify as original work but again, to me, such a work would feel tainted.
Do you disclose the use of SD to the customers that buy these works?