At first glance this might seem a little silly. Amazon has the key. (You don't even get to see the key yourself.) So Amazon can read all your data. And every time you read from the bucket it's automatically decrypted, so the encryption won't protect you from anyone who has somehow achieved permission to read your data.
But that's not the point. The point is to protect against attacks like "I found this pile of dusty drives stacked in the maintenance closet at Amazon," or "I went digging in the local dump near an Amazon data center and unearthed this hard drive, and look what I found backed up on it."
Dear Amazon S3 Customer,
Amazon S3 now supports server side encryption with customer-provided keys (SSE-C), a new encryption option for Amazon S3. When using SSE-C, Amazon S3 encrypts your objects with the custom encryption keys that you provide. Since Amazon S3 performs the encryption for you, you get the benefits of using your encryption keys without the cost of writing or executing your own encryption code.
And do they keep your key?
How long?
Now, one good idea might be to redundantly back-up Amazon's backups at some other host, using GPG to encrypt those. This ensures against Amazon encryption errors, billing errors, mistyped legal injunctions, Jeff Bezos declaring you his personal enemy, et cetera.
This may be obvious, but rolling my own at-rest encryption is not going to significantly protect my data from an attacker who works inside Amazon, nor from an attacker who roots my instance. If the keys are in the cloud, the keys are in the cloud.
UPDATE: oh, yeah, I forgot the use case where you are writing to s3 from outside Amazon's datacenter over HTTPS. Okay, that is a much stronger case for GPG in advance. It wouldn't matter if we assumed that TLS always worked. But this is TLS. Does your upload client check the certificate chain? So many of them do not. I sure hope Amazon's CLI client does.....