How to spy on a Ruby program (opens in new tab)

You know, sometimes i get slightly eye-rolly at the tone of Julia's articles, but fuck, this sort of attitude in programming is so much better than the alternative (i.e. lkml). Bringing more positivity and jolliness into programming is important. Fuck the traditional macho programmer attitude. I'm gonna try to be more like Julia.

For Python, you can just use Pyrasite (https://github.com/lmacken/pyrasite) and inject a script which prints a stack trace for all threads.

I've done this to debug OpenStack in the past and it worked very well. There are many similar projects for Python, I used this one since it's in the RHEL repo.

Useful summary on StackOverflow: https://stackoverflow.com/questions/4163964/python-is-it-pos...

If this sort of stuff is relevant to you, http://patshaughnessy.net/ruby-under-a-microscope is an essential book to read.

When I've needed to dump a stack trace, I've just included an interrupt handler which prints the call stack to a debug log.

That has been good enough for the problem of "wtf is this ruby process doing for _minutes_ at a time?" That doesn't get you flame graphs, but you can take a few snapshots and get an idea of what is happening.

For more involved perf debugging I've used ruby-prof.

From the article:

“I'm constantly surprised by how many people don't know you can do this. It's amazing.”

I'm probably nitpicking, but sad to see this in the article. One of the things I love about Julia's writing otherwise is that it is free of this sort of 'I'm surprised that you don't know this simple thing.' expressions.

Don't forget if you're on a system with DTrace, Ruby supports a nice bunch of probes for various events: http://ruby-doc.org/core-2.3.0/doc/dtrace_probes_rdoc.html

If you're planning ahead you can have your application load rbtrace which then allows connecting to a running process to see what it's doing, with options to limit to slow calls, IO, of specific method calls.

I hadn't heard of perf - very handy indeed. It's in the linux-tools package for distros using apt. You will need to install both the generic package and the one for your kernel version.

Of course there are equivalent programs for C. On GNU/Linux it's pstack aka gstack. Also gcore if you want an entire core dump you can analyze later with gdb.

http://linux.die.net/man/1/pstack

One issue I've run across is spying on JRuby's native Java classes. Anyone have a solution?

Nice write. The FlameGraph link in Julia's article is broken, maybe points to this one? https://github.com/brendangregg/FlameGraph

> maybe it doesn't exist because a Linux-only Ruby debugging tool is sort of a weird thing?

Linux-only seems perfectly acceptable, especially as more and more development moves to Docker containers.

Great write up, looking forward to seeing a completed tool!

"I was going to paste the strace output of what gdb is actually doing, but it is 20 megabytes of system calls."

I think this is why you shouldn't run this on production server itself. Each call is very resource intensive on the production server.

I believe the right way to analyze memory is to use "gcore", dump the memory, download it to the local machine's VM instance that's running the same OS as the production using scp. Also download the same ruby binary that production is running, and use gdb on the VM to analyze memory dump.