BDD testing for golang: Ginkgo and Gomega (opens in new tab)

I wouldn't mind a BDD tool written in Go, but not for this purpose.

What I would like is a tool that allowed testers and QA people to define tests using BDD for RESTful APIs.

Something along the lines of this syntax:

    Using https://example.com/myapi
        With access_token [accessToken]
        With content_type application/json
        When I create:
        {
          "somejson": true
          "foo": "bar"
        }
            Expect status 200
            Expect returnData.somejson = true
            Expect returnData.foo = "bar"
            Store returnData.id as myid
    And
    Using https://example.com/myapi/{{myid}}
        When I get
            Expect status 200
            Expect returnData.somejson = true
            Expect returnData.foo = "bar"
    And
        With access_token [accessToken]
        With content_type application/json
        When I update:
        {
          "somejson": true
          "foo": "foo"
        }
            Expect status 200
            Expect returnData.somejson = true
            Expect returnData.foo = "foo"
            Store returnData.id as myid
    And
        With access_token [accessToken]
        When I delete
            Expect status 200
    And
        When I get
            Expect status 404

I've seen a few things like this written in NodeJS, such as apieasy and vows. But nothing with this level of succinctness.

And I'd like it to be in Go as I'd like to just give a single binary to the testers (rather than an "install node, get project, fetch dependencies, resolve any issues, run the project").

When I read the headline, this is roughly what I hoped for. If I can find the time (unlikely), I'd build this. But then... maybe someone reading this knows something very close to this.

I can't help but feel like the authors of this tool (and ones like it) are fundamentally misunderstanding either the language, the purposes of testing, or maybe even static type systems in general.

Looks great. I'm a huge fan of RSpec and really miss it in other languages. It didn't occur to me before reading this post, but the ability to implement a testing framework like RSpec is a great way to gauge the expressiveness of the language.

Interesting choice of the Omega symbol for the actual symbol. Not sure if I like that or not. Vim users can use CTRL-K W * in insert mode to get a nice block character (at least on my build of vim) on their screen.

I prefer to use Gherkin syntax for by bdd testing, but the matchers are nice. The bootstrapper and helpers seem to fail hopelessly on windows, and the console writer outputs a lot of control characters which do not work in windows command prompt.

Here are 3 alternatives:

1. Mao: https://github.com/azer/mao

2. Zen: https://github.com/pranavraja/zen

3. GoConvey: https://github.com/smartystreets/goconvey

I love that Go is getting the kind of attention and use that leads to the creation of a framework like this from a relatively well known company. Although I can agree with some other comments here that it doesn't particularly feel native to Go.

This looks pretty cool, but for some reason I tend to like using the built in tools. Maybe I hate external dependencies or I have trust issues, but I'm way less likely to use this for that reason alone.

Please no with this Ruby madness in the Go world.

First of all, good job.

Are you planning to add selenium support?

This is totally awesome.

Hey all, this is the author of Ginkgo & Gomega. I'm excited to see all this thoughtful conversation and figured I'd chime in with a response to keep the conversation going. (Apologies for the long post!)

- Godoc documentation is coming soon. Godoc is great for API-style documentation but not particularly appropriate for tutorial/structured/narrative-style documentation. Think golang.org/pkg vs golang.org/doc. I wanted to write the /doc first as I believe it to be more valuable for beginners and is often overlooked by the go community. /pkg is coming soon (a few days) and is much easier to write.

- @buro9: Golang is BDD-style in the same way that Cedar and Jasmine are BDD-style. This is not Cucumber (way too much DSL for me!) With that said, asynchronous testing support is baked right into Ginkgo and Gomega so the API-testing that you'd like would be easily expressed in Ginkgo and Gomega.

- Assertions that I have fundamentally misunderstood Golang or the distinction between dynamic and static typed languages are interesting and I'd like to address some of them.

BDD-style testing need not be limited to dynamically-typed language. Yes dynamically-typed languages need far more comprehensive test coverage to make up for the missing compiler. That's why I tend to prefer statically-typed languages. To me BDD-style (vs the XUnit style) isn't primarily about addressing these deficiencies in dynamic languages -- it's about expressiveness.

To that end, I think that BDD and Golang go hand in hand. Let me explain. BDD is exceptionally good at describing the behavior of branching code. Golang is filled with branching code that needs to be described in test. Here's a classic example in pseudo-go:

    func DoSomethingAwesome(dependencyA DepA, dependencyB DepB) awesome Awesome, err error {
        stuffA, err := dependencyA.Gimme()
        if err != nil {
            return Awesome{}, err
        }
        stuffB, err := dependencyB.Gimme()
        if err != nil {
            return Awesome{}, err
        }
        ....
        return awesome, nil
    }

With Ginkgo you can cover these branches expressively:

    Describe("Doing something Awesome", func() {
        BeforeEach(...) //common happy case setup

        Context("When all is well", func() {
            It("should be awesome", ...)
        })

        Context("When dependencyA fails", func() {
            BeforeEach(...) //set dependencA up for failure
        
            It("should return a zero Awesome", func() {
                ...
                Expect(awesome).To(BeZero())
            })

            It("should error", func() {
                ...
                Expect(err).To(HaveOccured())
            })
        })

        Context("When dependencyB fails", func() {
            //etc...
        })
    })

Compare this to the XUnit style:

    func TestDoingSomethingAwesomeWhenAllIsWell() {
        //setup
        //some sort of assertion
    }

    func TestDoingSomethingAwesomeWhenDependencyAFails() {
        //setup + tweak
        //some sort of assertion that awesome is zero
        //some sort of assertion about err
    }

    func TestDoingSomethingAwesomeWhenDependencyBFails() {
        //setup + tweak
        //some sort of assertion that awesome is zero
        //some sort of assertion about err
    }

I prefer the former (note: that's a subjective statement). IMO there's nothing fundamentally more Golangish about the latter (if anything the fact that func's that begin with Test are special is kinda weird), and since Gomega's matchers are fluent in Go they know about things like zero values and errors having occured. Moreover, both Ginkgo and Gomega have been built to make testing asynchronous code (read: concurrent goroutines) easy and expressive. This isn't a carbon copy of RSpec/Cedar/Jasmine, it's a synthesis of the best ideas from those testing frameworks expressed in ways that cater specifically to Golang.

- Concerns about using interface{} and reflection seem odd to me. First off, the Go authors provide both of these things to precisely solve the sort of problem that Gomega is trying to solve. Nobody wants to write and maintain matchers that look like:

    Expect(foo).To(EqualInt64(3))
    Expect(bar).To(EqualString("blarp"))
    Expect(baz).To(EqualMyCustomAwesomeType(awesome))

besides the reflect package's `DeepEqual` does a great job comparing `interface{}` to `interface{}` correctly.

Most importantly of all: we're all using interface{} all the time: `fmt.Sprintf("%d %f %s %v", ...)`!

And thanks for seeing the point @karma_fountain: Gomega's matchers have excellent reporting about what precisely went wrong when a matcher fails. Bare assertions lack this and a look through the Go tests shows a lot of reinventing-the-wheel to provide what probably amounts to inconsistent error output to the developer. Why not put that all in one place, call it a matcher library, and make it dead easy to write custom matchers?

- And @shoo: yes, thanks for pointing it out: the example is somewhat fabricated and isn't a compelling argument for BDD vs XUnit. It's hard to cook up a good BDD example in very short space!