My bytecode optimizer beats Copilot by 2x (opens in new tab)

Yes exactly and that was the hard part (extract and verify the invariants). Still it's surprising because llm needs to be able to do that for any complex code.

What you wrote is great can I copy/paste it in the blog post? (Referring you of course)

Do you have a link to my surprised? I would be surprised if I were surprised by a purpose built thing beating something more general purposed. :P

Isn't the original reason for LLMs, language translation, the classic example where LLMs handily beat out bespoke translation tools?

For simple string search (i.e., not regular expressions) ripgrep is quite fast. I just generated a simple 20 GB file with 10 random words per line (from /usr/share/dict/words). `rg --count-matches funny` takes about 6 seconds on my M2 Pro. Compressing it using `zstd -0` and then searching with `zstdcat lines_with_words.txt.zstd | rg --count-matches funny` takes about 25 seconds. Both timings start with the file not cached in memory.

I have an older breach data set that I loaded into clickhouse:

  SELECT *
  FROM passwords
  WHERE (password LIKE '%password%') AND (password LIKE '%123456%')
  ORDER BY user ASC
  INTO OUTFILE '/tmp/res.txt'

  Query id: 9cafdd86-2258-47b2-9ba3-2c59069d7b85

  12209 rows in set. Elapsed: 2.401 sec. Processed 1.40 billion rows, 25.24 GB (583.02 million rows/s., 10.51 GB/s.)

Peak memory usage: 62.99 MiB.

And this is on a Xeon W-2265 from 2020.

If you don't want to use clickhouse you could try duckdb or datafusion (which is also rust).

In general, the way I'd make your program faster is to not read the data line by line... You probably want to do something like read much bigger chunks, ensure they are still on a line boundary, then search those larger chunks for your strings. Or look into using mmap and search for your strings without even reading the files.

What about AlphaEvolve / OpenEvolve https://github.com/codelion/openevolve? It has a more structured way of improving / evolving code, as long as you setup the correct evaluator.

I would start by figuring out where there is room for improvement. Experiments to do:

- how long does it take to just iterate over all bytes in the file?

- how long does it take to decompress the file and iterate over all bytes in the file?

To ensure the compiler doesn’t outsmart you, you may have to do something with the data read. Maybe XOR all 64-bit longs in the data and print the result?

You don’t mention file size but I guess the first takes significantly less time than 45 seconds, and the second about 45 seconds. If so, any gains should be sought in improving the decompression.

Other tests that can help locate the bottleneck are possible. For example, instead of processing a huge N megabyte file once, you may process a 1 MB file N times, removing disk speed from the equation.

You can't just tell LLM "make it faster, no mistakes or else". You may need to nudge it to use specific techniques (good idea to ask it first what techniques it is aware of), then give it comparison before and after, maybe with assembly. You can even get assembly output to another LLM session and ask it to count cycles, then feed the result to another session. You can also look yourself what seems excessive, consult CPU datasheets and nudge LLM to work on that area. This workflow isn't much faster than just optimising by hand, but if you are bored with typing code is a bit refreshing. Like you can focus on "high level" and LLM does the rest.

Regex.

Great idea. Which agent to use?

I tried with opus and o3 but I had to copy/paste the code and I wasn't sure it was the best way.

I tried 10 prompts and the simplest was the best (probably due to the code being simplistic)

Yes I did and it solves tons of problems of coding agents

Oh good point which one should I use? I ran it alos with o3 and claude sonnet but the results were similar or worst.(Some are in the repo)

Good point I should have mentioned. I was so "in it" that I didn't realize it was a valid question.

This thing outperforms llvm (on o3 and O2) and the JVM too. I didn't explain it because I wrote it in a previous blog post but that's a fair point.

And it's not that obvious because the previous example was handled better by chat gpt while llvm couldn't handle it.

Thank you for this idea.

Are they? AFAIK, the “etc” includes using hardware that produces the same results for a given input every time. Once you start to multi-thread/multi-process in combination with floating point math, that can be hard to accomplish.

For example, the result of summing a stream of floats depends on the order the floats arrive in, and that order can change depending on what’s in your CPU cache when you start a computation, on whether something else running on your system such as a timer interrupt evicts something from cache during a computation, etc.

If you’re running on your GPU, even if the behavior of your GPU is 100% predictable (I wouldn’t know of that’s true on modern hardware, but my guess is it isn’t) anything that also uses the GPU can change things.

Yes but the analysis it's going to run is probabilist vs the one I'm running is closer to smt