I agree, in this case it's hard to defend against a rogue script or container image, as you need to give it read-write access to your source code, so it could add a malicious payload to your source code or install a Git hook to break out of the container into your host or get some malicious source code onto your company's Git server.
There are measures that could defend against this (run all your development tools inside containers, and mandatory PRs with reviews) but they are probably beyond many/most developers are willing to do security-wise.
There are a lot of scenarios where I think security through isolation/containerization makes a lot of sense (e.g. for code analysis tools, end-user applications like video games, browsers, etc.) but not too much for this particular one.
