I don't have a comprehensive suite of numbers at the moment (which is why I qualified it with "I believe"). My tentative conclusion is based on experience by a flux maintainer benchmarking forwards and backwards passes compared to pytorch, along with reports like this: https://discourse.julialang.org/t/flux-came-a-long-way/62288