Show HN: Bash-My-AWS – CLI Commands for AWS (opens in new tab)

Like it or not (I do...) terraform is the de-facto industry standard, and pretty much the only mature cloud resources management tool I'm aware of.

It is unwise IMHO to use CloudFormation currently unless you're provisioning resources so obscure they didn't yet make it to tf aws provider.

BTW your Dockerfile pretty much boils down to:

    FROM alpine:3.10

    RUN apk add --no-cache \
        python3

    RUN pip3 install awscli

    COPY config /root/.aws/
    COPY credentials /root/.aws/

I would strongly recommend to use cloud formation through a typed proxy like troposphere. Also would not recommend to use terraform at all since you will run into warts and fundamental issues quickly. I have done projects with both and my current blessed workflow is a custom python driver which uses CF via troposphere and minimal boto3 as glue. Also I work at AWS.

Gruntwork has a lot of Open Source tooling around AWS and their new guides are pretty great for some of what you're mentioning

https://gruntwork.io/guides/

I am in no way affiliated with them other than being a customer

If I’m understanding you correctly, I think you want CloudFormation?

At work we are using Terraform to manage everything that is related to AWS resources, including accounts, IAM policies and groups. We also used Serverless framework and CloudFormation, but Terraform is what works for us and I can recommend it as a main IaaS tool

We use Pulumi to manage both our GCP and AWS resources, and we really like it.

You might consider using Terraform directly if you want something more mature.

I have sometimes questioned whether I should be spending my personal time developing an open source tool so tied to a single companies services.

My reasons for continuing include:

- I prefer to use command line over ClickOps

- Using Bash-My-AWS makes me more effective at work

- The emergent UX is equally applicable to other services (e.g. bash-my-github, bash-my-spotify)

- The intrinsic satisfaction from creating

- Helping improve the experience for others

More charitably, AWS provides a CLI for most resources and commands and (in large measure) either provides backwards compatibility or a long depreciation window. This allows hackers to build syntactic sugar on top of their infrastructure.

I love that a community has an option to build components and share the same. It has made my work much more productive.

But we certainly agree on that last point: it’s an incredible business model.

Anything above bare bones will be opinionated, imo this is the best solution for infrastructure provider - maximum freedom, but also providing a UI for simpler access.

The annoying part about AWS putting so little effort into smoothing out the rough edges in their tooling, ux and services, is that the whole ecosystem of helpers/wrappers developed to make the services usable by humans needing to get something done (rather than deep dive again and again into many rabbit holes of AWS's idiosyncrasies) - inevitably only cover some tiny subset of the services a typical shop is going to use.

Not to take anything away from the author of this project - Bash-my-aws looks fantastic - but it only helps you with a few core services. Same appears to be true of the commandeer tool that has also been mentioned in this thread. And the same is true for localstack, and on and on.

I really wish AWS would devote some resources to filling in these gaps themselves, and comprehensively.

How is this different than all of the git extensions that people publish.

Verbosity isn’t unusable, it has a steeper learning curve.

turning the flywheel

Windows users have commented at how Bash-My-AWS's innovative use of unix streams reminds them of PowerShell.

The listing functions output lines of tokens. The first token is the resource identifier. Piping that output into functions for that resource type results in the resource IDs only being used.

  $ instances
  i-03dfa28fc8235df7b  t3.nano  running  prometheus  2019-12-31T14:10:45.000Z  ap-southeast-2a  vpc-9def06f8
  i-0fd7a4c81051f2718  t3.nano  running  huginn      2019-12-31T14:10:44.000Z  ap-southeast-2a  vpc-9def06f8
  i-0abcd6e9c302f35bb  t3.nano  running  rails-demo  2019-12-31T14:10:47.000Z  ap-southeast-2b  vpc-9def06f8

  $ instances | grep rails-demo | instance-asg | asg-capacity
  rails-demo-AutoScalingGroup-14SBR6O3W1FBL  0  1  2

  $ instances | grep rails-demo | instance-asg | asg-
  asg-capacity              asg-launch-configuration  asg-processes_suspended   asg-stack
  asg-desired-size-set      asg-max-size-set          asg-resume                asg-suspend
  asg-instances             asg-min-size-set          asg-scaling-activities

  $ instances | grep rails-demo | instance-asg | asg-desired-size-set 2
  $ instances | grep rails-demo | instance-asg | asg-capacity
  rails-demo-AutoScalingGroup-14SBR6O3W1FBL  0  2  2

Do you have any insights on how someone who is used to the text-only world of Bash transition to using Powershell cmdlets?

The problem I run into is that it just feels like so much typing to me. I have to read documentation. All the attributes HaveReallyLongNamesThatContainCapitalLetters. By the time I've made my beta version of the command I want to run, I feel like I need to open a text editor to finish it. Maybe add some error checking. Some comments too. Maybe a unit test or three. And now I have an entire project and all I wanted to do was add a line of text to the end of a file.

Part of the problem on my part is my own ignorance of the APIs and what commands are available to me. But it all seems too verbose to use practically. The Powershell language seems very good for what you would write a shell script to do, but for interactive commands, I have a hard time believing that people use it. It's just so verbose.

Hasn’t AWSCLI supported toggling the cmd output to either text, json or csv for quite some time now or have I misunderstood your comment here?

jq is only used in three of the >120 functions (for sort-keys functionality). All the rest use JMESPath.

If anyone can help with a solution I'd be delighted to remove the dependency on jq.

https://github.com/bash-my-aws/bash-my-aws/blob/b74d92a902bb...

jmespath has quite a few limitations, even the official AWS CLI documentation states that for the more advanced stuff `jq` is probably the go to tool.

https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-o...

"For more advanced filtering that you might not be able to do with --query, you can consider jq, a command line JSON processor. You can download it and find the official tutorial at http://stedolan.github.io/jq/."

I find jq’s language a lot nicer than JMESpath and trend to use it whenever possible

My HN comment detailing several limitations of JMESPath: https://news.ycombinator.com/item?id=16400320

Here's the video from my talk on Bash-my-AWS at LinuxConf 2020 in the Gold Coast Australia.

https://www.youtube.com/watch?v=UbH_cg7Ev1Q

for anyone on this thread that is interested. I run https://getcommandeer.com which is a tool to manage your AWS and IAC infrastructure from a desktop GUI. I love this bash-my-aws, as we are about to release Bash, Docker Compose, and Terraform Runners. We already have Serverless and Ansible runners. They enable you to run your command line system from a GUI, so that you can instantly switch between AWS accounts/regions and even LocalStack. Because it is a desktop, under the hood we are really running cli tools mixed in with some AWS JS SDK.

They're good questions. I can tell you how I manage regions and accounts but am interested in learning how people think Bash-my-AWS might better support users in this regard.

The AWCLI, as well as SDKs all support grabbing Regions and account credentials from environment variables.

For Regions, I work tend to use the following aliases:

  alias au='export AWS_DEFAULT_REGION=ap-southeast-2'
  alias us='export AWS_DEFAULT_REGION=us-east-1'
  alias dr='export AWS_DEFAULT_REGION=ap-southeast-1'

I normally work in a single Region and swap when required by typing the 2 character alias.

To run a script or command (doesn't have to be Bash-my-AWS) across all Regions I use region-each:

  $ region-each stacks | column -t
  example-ec2-ap-northeast-1  CREATE_COMPLETE  2011-05-23T15:47:44Z  NEVER_UPDATED  NOT_NESTED  #ap-northeast-1
  example-ec2-ap-northeast-2  CREATE_COMPLETE  2011-05-23T15:47:44Z  NEVER_UPDATED  NOT_NESTED  #ap-northeast-2
  ...
  example-ec2-us-west-2       CREATE_COMPLETE  2011-05-23T15:47:44Z  NEVER_UPDATED  NOT_NESTED  #us-west-2

For AWS accounts, I type the name of the account and I'm in. For accounts using IDP (ldap/AD backed corporate logins) I generate aliases so I have tab completion and simple naming.

In accounts that are only setup to use AWS keys, I use aliases that export credentials kept in GPG encrypted files. Last time I looked, AWS docs suggested keeping these long lives credentials in plaintext files readable by your account. That's asking for trouble IMO, especially if they're kept in a known location that a compromised node library could exfiltrate them from.

AWSCLI v2 beta includes support for SSO so it's probably a good time to look at how BMA could include support for auth.

AWSCLI is amazingly flexible and powerful.

Bash-My-AWS thinly wraps AWSCLI commands that would otherwise be too long to type. So you're still using AWSCLI and can improve your skill with it by inspecting the source of Bash-My-AWS functions.

https://news.ycombinator.com/item?id=21931298

I don't like to rely on my memory either! I forget the names of commands and use tab completion to list them.

You can just type bma[TAB][TAB] and it will list them all.

If you know the type of resource you are working with, you can use TAB completion to see it's commands:

  $ stack-
  stack-arn            stack-exports        stack-tag-apply
  stack-asg-instances  stack-failure        stack-tag-delete
  stack-asgs           stack-instances      stack-tags
  stack-cancel-update  stack-outputs        stack-tags-text
  stack-create         stack-parameters     stack-tail
  stack-delete         stack-recreate       stack-template
  stack-diff           stack-resources      stack-update
  stack-elbs           stack-status         stack-validate
  stack-events         stack-tag

Thanks!

The intent has always been to enhance rather than replace AWCLI (which is an amazing tool!).

If you're ever wondering how a Bash-My-AWS command works, use `bma type` (it even supports tab completion for all the commands).

  $ bma type instances
  instances is a function
  instances () 
  { 
      local instance_ids=$(__bma_read_inputs);
      local filters=$(__bma_read_filters $@);
      aws ec2 describe-instances $([[ -n ${instance_ids} ]] && echo --instance-ids ${instance_ids}) --query "
          Reservations[].Instances[][
            InstanceId,
            InstanceType,
            State.Name,
            [Tags[?Key=='Name'].Value][0][0],
            LaunchTime,
            Placement.AvailabilityZone,
            VpcId
          ]" --output text | grep -E -- "$filters" | LC_ALL=C sort -b -k 6 | column -s' ' -t
  }