You're assuming that Microsoft needs to shut down Copilot to comply with the licenses of the software they misused. That is not the case. To make Copilot legitimate, all Microsoft has to do is restrict Copilot's inputs to non-proprietary code, release the Copilot dataset under a compatible license, and clarify that the code generated by Copilot is also covered under that license. Attribution can be done by inserting a comment with link to a paginated list of all of the contributors whose code was used in Copilot.
Microsoft can even continue to sell Copilot as a service while keeping it license-compliant, since most developers are not going to self-host the entire dataset. Microsoft can also choose to exclude copyleft-licensed code from Copilot or create multiple flavors of Copilot, each licensed differently. You can get your "productivity enhancement" without needing Microsoft to violate software licenses.
The damage is not in the monetary payment denied to free and open source software contributors, payment these contributors never demanded. The damage is in Microsoft violating other people's software licenses to create a proprietary product derived from copyleft-licensed and attribution-required code, and in Microsoft encouraging other developers to violate these licenses. Microsoft needs to rectify these violations with specific performance.
