I wouldn't be surprised if it wasn't so much "brittle" as that Julia internals were too much of a moving target before reaching 1.0 to make it worth the effort of keeping it working between releases.
It was a bit of both but by far the bigger issue was that compilers emit terrible debug info. It's barely good enough to debug C/C++ code and for Julia code it was impossible to reliably get values of local variables and set breakpoints.