Metamorphic Testing (opens in new tab)

The nice thing about metamorphic testing is that it seems easy to implement. You don't need a complex testing framework, like Quickcheck.

Just alter the input in some fairly simple ways and check the output.

Good categorization. However, the example of sound -> text uses a set of expected invariants for the problem. Invariants abound in sensory data related problems as they are inherently high dimensional as measured but our interest lies in a vastly reduced space. You could squint and look at these examples as PBT as well. Such powerful invariants have been hard (caveat: for me, personally, not done research) in general systems dev areas.

Other examples more illustrative would be useful.

Also, this is commonly done as "data augmentation" in ML ... which isn't surprising given the ML related examples linked in the post. When talking to software engineers without an ML exposure I often ask them to think of ML as "extreme TDD" and gave a basic level talk with that perspective [1].

[1]: http://sriku.org/blog/2019/03/07/talk-machine-learning-as-ex...

edit: by "ML as extreme TDD", I don't quite mean the algorithms themselves as they're not central. I refer more to the data work involved.

The place where I saw this was in testing compilers. The idea is you get some piece of deterministic code, and some input to run it on. You compile and run that code on that input, and get an output.

Now, you modify the code so that the behavior ON THAT INPUT is not changed (and does not become undefined). For example, you can make arbitrary changes to parts of the code that weren't executed on that input. You now compile the modified code and see if it does the same thing on that input. If not, you have a compiler bug.

What's wrong with ACM Queue?

Not every ACM journal is excellent and I know we're supposed to hate everything that's not arxiv, but TEAC is a good one too.