To comply with the GPL you only need to publish the one specific snapshot of source code you've combined and redistributed with GPL code, but you don't need to permanently relicense your project if it doesn't contain any code you don't have rights to use. The "tainted" version will be granted as GPLed forever, but other earlier or later versions that don't use any GPL code don't have to.

Or you can go the copyright way, and claim it wasn't a copyright violation (because it was a fair use, or non-copyrightable code) or settle the matter in whatever way the law lets you get away with.