You can tell codespaces to include the AWS commandline tooling automatically via the devcontainer "features" attributes. And you can tell it to run a script once the codespace has initially been created using the postCreateCommand (which imo is a lot easier to debug than beforeCreate...
For us the s3 credentials live in the github repo as codespace secrets (although I think you could set up a much better auth approach via the vscode aws plugins possibly).