Copyleft grants you the right to make changes and redistribute the resulting work under a compatible license. If you have a problem with CLAs, don't sign them. Many people do not care.
Even without a CLA the author could simply remove contributions and still relicense. This isn't novel, and has been done before.
Obviously it may be difficult to remove contributions.