Ghidra, NSA's reverse-engineering tool (opens in new tab)

I don't think I've ever seen an application where a working undo function was added in a posteriori. You either design it in from the start or you don't get a working one or you have to rewrite most of the application.

And they didn't take my money, break my key in an update, and ghost me while I was still in the support period. So they've got that going for them.

Being open-sourced is a big advantage. I just fixed a bug in GHIDRA relating to trackpad scrolling which makes it MUCH more usable for me. I could never do the same with IDA or Binary Ninja.

I do so love the shell code compiler of Binary Ninja, though. It works very well and has definitely saved me a lot of time.

It's not Open Source.

For which they charge a per-CPU fortune https://www.hex-rays.com/cgi-bin/quote.cgi

It does, but it costs an extra $1.5k on top of the main software, per architecture.

I use r2 almost all the time, it's just so fast and convenient, and it makes working with Binary Ninja easier. Also use IDA when all else fails.

And the decompiler is dope for IDA

Yeah, my mouse was hovering over the download button eager to test it out when my brain suddenly went "wait, don't do that!"

If you wanna run this thing, you should probably build it from source yourself (don't trust the binaries) and even then run it in a pretty well sandboxed virtual machine. I would not be surprised at all if the NSA left some surprises in that thing.

BAP is not a competitor to IDA Pro or Ghidra, it's a platform for implementing automated analysis, while IDA and Ghidra are more like reverse engineering tools that are focused on human interactions. We do support IDA Pro so that you can run BAP analysis from it and have the best of two words. We will soon roll out the support for Ghidra too (the issue is created [1]).

What is a really great contribution of Ghidra, to my opinion, is the detailed specification of all supported ISA in Sleigh (their terse and concrete ISA specification language). Ghidra ships with about ~200kLOC of instruction descriptions and this is the most valuable contribution to the community. We're planning to support Sleigh in the nearest future, and I believe that Sleigh might become a standard de facto for instruction semantics specification.

[1]: https://github.com/BinaryAnalysisPlatform/bap/issues/929

The download includes a zip file containing source code next to every jar file.

> If this is expensive to you, then it’s not for you. This is for people who are making real money with these tools, not hobbyists dicking around.

That's an odd perspective. Imagine if this type of sentiment were applied to paint brushes. There is a lot of useful work that is not economically viable per se, and to discount that and to be pejorative feels wrong.

What's the average wage of a cyber security professional in SE Asia or Africa compared to these tools?

What advantages does the paid version of Metasploit offer? I have been able to do some pretty incredible stuff with the free offering...

That's enterprise penetration testing software, IDA is bench reverse engineering software. It's a totally different market.

I wonder if there were some mainframe "professionals" with this opinion before the advent of the personal computer.

I agree, they have a big potential. But if you speak about FOSS alternative - there is already radare2[1]+Cutter[2]+radeco[3].

[1] https://github.com/radare/radare2

[2] https://github.com/radareorg/cutter

[3] https://github.com/radareorg/radeco

Reverse engineering the firmware for an embedded product where someone lost the source code.

Bonus points available for:

  * "the source control is ZIP files on a network share"
  * "yeah we use forced squash commits on everything to keep the Git history nice and linear"
  * "it was designed by a contractor who is now uncontactable"

Malware analyst, vulnerability research

Well there are these, at my workplace: https://news.ycombinator.com/item?id=19055183

That includes malware analysis, vulnerability research, and emulator development.

Audits for Intellectual Property.

Too often companies pay 6 digits for a feature that some supplier rips directly from an open source on the Internet (often GPL) and then sells as his own.

Chinese software dev firms.

A job at NSA, for instance.

Red team?

Auto analysis when you have barely any information. Any tool can make nice output if you feed it nice input. Try a partial dump from an exotic device and then you’ll see IDA shine.

To be fair, without undo, Hex-Rays can only move forwards. This explains their advantage.

Not sure about everything, but last i looked IDA had a lot more support for different architectures and file formats compared to most of the open source stuff (not sure about other proprietary ones).

I mean having a good UI is great but without the features to back it up, you can’t do anything serious. I tried cutter again a few months ago and went back to ida after an hour of frustration. When handed a binary dump with no executable format or symbols, cutter just chokes while IDA was able to quickly find 90% of functions in memory as well as data xrefs and strings and so on.

I’m sure everything performs well on ELFs built with -O0 -g but in most real world usage, Ida is queen.

Since everything is open source, if ghidra is as good as people say it is, I’m sure people will make better guis for it (and tui) in no time.

It really is a job for a GUI, but even IDA lets you type commands. You can use Python, or a built-in language that is in the C family, or anything custom that you have attached to the plug-in interface. I would be surprised if one of these tools is lacking such a capability.

I use cutter with r2 and consider it a better UI than IDA; but then again I don't do a lot of work in it, just CTFs and crackmes.

4. Managing licenses is a huge PITA, presumably especially in environments with lots of classified information.

Operational reasons? They found a compromisable worker was employed there, or they somehow put modifications in IDA making the software compromisable and so not safe for them?

There’s a third possibility: they wanted a piece of software they could customize to meet their needs. Admittedly, for simple bugfixes and the like, Hex-Rays’ support is known for being quite responsive (as suits the small number of customers). There’s also a quite large (albeit poorly-documented and crash-prone) SDK, which can handle a wide variety of needs and has gained functionality over time. But if you want to add new functionality that can’t be implemented using the SDK? (That includes just about anything related to the decompiler.) You’re at the mercy of Ilfak’s priority list. Well, mere mortals are, at least; the NSA has enough money that it might be able to set up some sort of special contract with Hex-Rays, but I suspect that’s easier said than done.

From that perspective, the ideal is what the NSA ended up with, a codebase whose development is fully in-house. Notably, though, second best would be to just have access to the source code of an existing tool, so you can at least make your own patches if necessary, even if you’re not in control of the codebase’s overall direction. Did the NSA ever seek that in IDA’s case, and could they have obtained it if they did? I don’t know the answer to either question… but source access certainly isn’t offered to typical customers. In general I’m surprised that “paid + source access for customers” isn’t a more popular model of software development.

From my perspective, which admittedly is very different from the NSA’s, I was never very interested in low-cost IDA competitors like Hopper or Binary Ninja, but I’m very excited about Ghidra. Why? Partly because it’s a more full-fledged competitor in terms of feature set, I admit – but the competitors I mentioned are bound to narrow the gap over time. Partly because of cost: I myself am at a point where I could justify the $600/y for Binary Ninja’s commercial edition, or even the order-of-magnitude-higher cost of the Hex-Rays decompilers, without wincing too badly. but I believe that reverse engineering should be accessible to beginners and amateurs. (Piracy is a partial solution, including in IDA’s case, but some people don’t like to do that.).

But the main reason I’m excited about Ghidra is that I have the source code. As a concrete example, I’ve spent a good amount of time reverse engineering software for the Nintendo Wii and Wii U. Both consoles have a main CPU based on the PowerPC architecture, but with a custom ISA extension for an extremely barebones version of SIMD. Well, both Hex-Rays and Ghidra support PowerPC decompilation (although that’s a relatively recent development), but unsurprisingly, neither of them have full support for that ISA extension. IDA actually does have built-in support for disassembling it, but AFAIK not for decompiling; Ghidra doesn’t seem to support it at all (but I may just need to configure it properly). What can I do? Well, in practice, nothing, because I don’t care about the Wii U anymore. But if Ghidra had been released a few years ago, I’m pretty sure I would have gone and implemented support for the extension myself; I haven’t looked at Ghidra’s source yet, but since it already supports other vector ISAs, it probably wouldn’t be that hard. With IDA, I was stuck. The SDK supports adding custom instruction sets for disassembly, but the decompiler SDK is so limited that supporting them there is either impossible or at least would be a huge hack.

And that’s just one of many customizations I‘ve wanted over the years. Some of them are probably easier said than implemented, but at least now I can put that to the test!

Alternative take. Want to secure national infrastructure? Release the tools to do it for free.

Viva la open-source revolution

I don't think that is possible. I used early versions of IDA Pro on MS-DOS in 1996... and it had the core analysis and interactive disasm mode then. Ghidra seems to be based on this very concept, all the way down to the "XREF" labels. Even the sub-panels in the UI look the same: Imports, Exports, Symbols...

I believe the included decompiler predates the one that goes with IDA Pro, so that could be a motivation. IDA Pro itself is really really old, having once been a 16-bit DOS shareware program, which explains why IDA Pro doesn't use the GUI shortcuts that were standardized much later.

IDA Pro still doesn't support collaboration, although there are very broken hacks that attempt to add it. Binary Ninja supports collaboration if you buy the enterprise edition.

> Kind of seems like the equivalent of running Flask apps in debug mode, which by default will handle exceptions by showing a traceback with an interactive debugger that can be used to execute arbitrary code.

As an aside, this is no longer precisely the case, though it was for quite some time.

With modern Flask (> 1.0.0), the debug server will start with a randomly generated PIN output to STDOUT when the server starts. In turn this PIN must be entered on the web interface to execute commands.

I wonder if they run Ghidra on a remote machine and run it with some sort of command and control center to automate tasks (IE, run regular some basic automated stuff).

This makes the whole release even more interesting, I wonder if we'll get a statement on why they have that debug mode.

I also don't think it's a backdoor, but the best way to hide a backdoor is to make it look like a mistake.

Yes this doesn't really seem like a big deal

Bad actors have had access to this tool too... it was leaked previously.

This is cool because:

- It's legal and free - It's open source

You mean there is nothing new here? Then why is this news?

I am not wondering about the concept of reverse engineering but the specific (and hopefully novel) feature set that this may enable.

I am not sure I completely agree. If I know how my adversary detects and studies stealth code, I may be able to design better stealth code that is better at evading their methods of detection.

I mean the evolution of stealth tech in military has followed a similar path. As radar systems improve over decades, they keep on working on new ways to evade detection for aviation/missile tech.

I understand the high level point of good tools being more widely available to the white hat crowd, but I am trying to understand the argument that this is 100% better in all cases and there are no downsides.

I don't think they would burn some 0-days for this, one container should be enough.

AFAICT all the source is there, beside every `.jar` there is a `.zip` with the corresponding source. The source in a more usable form should be posted here soon: https://github.com/NationalSecurityAgency/ghidra/

(And if not I'm sure the community will reconstitute it)

I'd expect most people who use software like this to be using things like Qubes OS anyway?

Seeing 403 (I'm in Russia). Maybe some export restrictions, but more probably just a glitch in CDN (Cloudfront). P.S. I'm not in Crimea, never encountered software export restrictions before.

I was able to see access the first link in France. It might depend on the country?

Downloaded fine in UK

The freeware version supports x64 as well.

My personal reason for not pirating IDA Pro is because I don't want to contribute to the problem. It's one thing to argue about the effects of piracy on things like video games, where the unit price is much cheaper, and a large number of users are casual users who mostly are going to buy legitimate copies if it's convenient and not exorbitantly expensive.

Power user software, like Photoshop, IDA Pro, VMWare, etc. are a different story. They provide tremendous value to both companies and individuals and yet I have no doubt an enormous amount of their poweruser userbase simply have never paid for them. As a young adult or child with no practical way to get a license, this is pretty innocuous since frankly it's hard to argue any sale was lost. But there's plenty of cases where large companies and of course hobbyist users end up pirating the tools they use. I believe Windows XP shipped with some audio files that were produced with a pirated version of Sony Soundforge, for example. That's just silly, but.. it happened.

IDA Pro is an excellent piece of software. They provide a freeware version, which is a pretty nice thing to do. And while the licenses are expensive I have no doubt it is worth it to the companies that purchase it, many times over.

Sadly, I can't afford IDA (as I've discussed eerily recently in HN comments, actually) so I've been mostly avoiding it for now, but I do buy other software, including Windows licenses, Adobe Creative Suite, VMWare, etc. If they're useful enough for me to use, then as an adult with decent income, I pay for them.

Somebody sort of casually pirated a copy of IDA Pro back in the mid-2000s (IIRC, he shared his copy on a public server). The IDA people (DataRescue, at the time, but from what I recall the page survived the move to Hex-Rays) found out, banned him from using IDA, and then put a page on their website threatening to rescind IDA licenses from any company that employed him. The IDA team is pretty aggro.

Uh, no. So if you are making a small plane hobby project, it is okay to steal from the department store nearby.

A hobby project is for your enjoyment. Pay for your shit. Simple.

I can understand a 14-year-old teenager doing that after his parents refused to purchase the game.

Is the NSA talk about this available online?

maybe the NSA don't like non-US citizens using open-sourced software :)

this is what I get -

403 ERROR The request could not be satisfied. Request blocked.

Generated by cloudfront (CloudFront) Request ID: i_Sa-b1Fj2C4ZAUtBsmAp-7mVg9LerDTlD9t1_00ex4fQyCUhrYvdw==

I'll go one better: I've contributed patches to retdec.

Retdec is ... okay.

On small binaries it's usable. On even average sized windows binaries (a few meg), not really.

Like on things that IDA takes 10-15 minutes and a reasonable amount of memory (like a 7 meg windows binary), retdec can take forever and unlimited amounts of memory.

I started fixing a lot of the memory issues (completely recursive CFG traversal, etc), but there are also very serious algorithmic issues (N^3/N^4 algorithms in the optimizers).

If i disable a lot of the backend optimizers, i can make it work okay.

But then the output is also a lot larger/worse. To be fair: It used to be about 50x bigger than similar IDA output. The latest development version of retdec now has a new backend IR converter, and the output is only 5x-10x bigger than IDA output.

So as a TL;DR: retdec in its default state is unusable for anything but small binaries. If you understand what is going on, you can get it to work on a lot of binaries as long as you have a ton of memory and time to spare.

I believe Capstone is merely a disassembly framework, and retdec's decompilation process is custom implementation which works on LLVM IR.

Binary Ninja has most of a decompiler and is expected to get the rest soon.

Binary Ninja offers multiple views of the code, each with an API that gives you the same access that the GUI has. The different views vary in how much they are like assembly or C. Only that last step, real C code, is still missing. Those other views are quite good if your goal is to understand things, but less good if you were hoping to throw the results into a C compiler.

I did some minimal stuff it and I like it. Its slow and has Java UI is as bad on Windows as expected (in part OpenJDK) but I'll reach for it next I have the need. The main thing that I seem to have missed was some sort of scripting like jython which presumably can be added via extensions or just via code if it is actually missing. Its been a while but I'm happy to see some of my tax dollars at work and this stuff being released back to the public for free rather than used against it.

Can they? How's that work, surely a resident in USA could take the work to the other country and make it available, if they can't then it wasn't public domain in the first place?

Can you point me to any caselaw/ analysis please?

Totally, that's why I was hopping for something better looking. Looks like Binary Ninja is the only one that cares about that.

"Eschew flamebait. Don't introduce flamewar topics unless you have something genuinely new to say. Avoid unrelated controversies and generic tangents."

https://news.ycombinator.com/newsguidelines.html

Likely nothing, it's the source code for an RE toolkit with an NSA sticker right on the box. There is literally no worse place to try to hide back doors.

At worst they will know how to mask their real malware from analysis with their own tools.

That's a great idea!! I can try.

Your comment makes no sense whatsoever. Let's say you're an NSA target. You're probably already hacked. If not, then you are very smart or you haven't been an NSA target for long. Let's assume you're a very smart malware researcher - that means you 1. Already have tools like IDA and don't need this, 2. Have an in depth knowledge of how to acquire and run potentially malicious code safely, 3. Have experience figuring out if that code is malicious.

Do you think the winning strategy for the NSA here is to attack you in a way that you're perfectly equipped to deal with?

As I see it's licensed under Apache 2.0. I don't know about any regional restrictions for this type of license. But it's could be a real reason because of our stupid government.

Funny, I'd assumed it was a reference to the movie monster.