Show HN: Rust test harness that measures energy consumption (opens in new tab)

Hey, I wanted to share a project that we've been working on! Coppers is a custom test harness for Rust that allows you to measure the energy consumption of your test suite. A use case for this could be to identify regressions in energy usage, or to do more targeted energy optimizations. Our goal was to make it as seamless as possible to integrate it with existing Rust projects. To make that work, we had to rely on some unstable and internal Rust compiler features that are only available in nightly. But the current implementation seems to be able to measure the energy consumption of almost every existing Rust crate we tested! (with the exception of embedded and system-specific crates, but that is a limitation we're looking into)

Counting instructions is very accurate and roughly approximates power usage. The CPU's self-reported power usage is comparatively pretty noisy. Unit tests will probably be done running before you can get meaningful data. I have to wonder if a test runner is the best point of integration for this. It might make more sense to expose it as a bench harness, like criterion.

EDIT: another benefit of a criterion-like approach is that you wouldn't require nightly

This reminds me of an idea I wanted to submit to the systemd team (or wherever it would be more appropriate) to have a Linux service report on the current power usage of the OS and maybe even translate it into currency-per-hour to show people how much they were spending over time with the aim of reducing power wastage. Seems like it would be more relevant than ever given the global situation around energy these days.

Many benchmarking systems face measurement issues that make it difficult to produce solid results. Any given run might not be running on the same hardware, the same OS, built with the same compiler, running with the same runtime, with the same versions of dependencies, with the same system load, at the same temperature, and so on.

One robust solution is to instead do pairwise comparisons, many times in a round robin fashion. The results aren't quite as nice to plot, as you don't get a single consistent speed value, but they are much more reliable and true, and you still get useful information, like ">95% chance that this test is at least 20% faster at this commit than at the previous one".

A project I contribute to uses this strategy: https://github.com/Polymer/tachometer, but I'd love it if more benchmarks took this approach.

Very nice!

Any ideas on how to measure energy consumption of programs in a GNU/Linux OS? I know of `powertop` but it measures total power consumption (its per-program table is inaccurate)