What Facebook could have done to avoid that issue is to enforce their ToS to get his IP address, then contact and hand those logs to LEO. (A company can follow its own processes as far as possible before contacting LEO, but once they start working together, they become an agent and the process is changed into something with less independence.)
The idea of creating a new OS to trap an end-user is one of the weirdest things I've ever heard of, on several levels, frankly.
Source: previously the LEO contact at a large Silicon Valley company. Typically you meet with them quarterly or as necessary, but you don't casually "work together" on cases to avoid the appearance of being their agent instead of a company representative.
But developing it in the first place, handing it to those legally authorized to use it, and catching someone like this - I don't understand how anyone could be against that... again, as long as the exploit is burned afterwards.
> raises difficult ethical questions about when—if ever—it is appropriate for private companies to assist in the hacking of their users.
I am happy Facebook did this. They made the world a better place.
These types of exploits are usually specially crafted files that trick the code responsible for parsing and displaying the video file into running whatever the creator wants. The terms "buffer overflow/underflow+" and "shellcode" might help narrow down a definition for you. Below is an overly simplistic version .
The video might contain, inside of it, a specially written computer program that sends the IP address of the current computer to whatever location the attacker wants. (This is the shellcode). This code could be really simple.
The video could also have parts in it that do not make sense. the video player code makes assumptions about the video that the video purposefully violates. When the video is processed by the computer, the video player code misunderstands what it needs to do and will accidently treat the video as code. (this is the buffer overflow). Since parts of the video are actually special shellcode, the computer has been tricked into running code hidden inside the video.
The article below implies that is what this was https://www.vice.com/en_us/article/gyyxb3/the-fbi-booby-trap...
+Buffer overflows / underflows are just one of many techniques for exploiting a program. it's the main one I know in passing.