If your colleagues are leveraging substantial undocumented overtime, you're going to find out eventually. If you know someone is in the office 40 hours a week and they submit massive pull requests first thing in the morning, even Monday mornings, for problems you know they hadn't solved or started the night or week before, you should suspect something's up. Sooner or later, if someone's breaking their neck, they will resent team members who aren't putting in the same effort, and they'll slip and admit to the amount of time they're putting in, directly or indirectly.
Granted, it's easier to hide this now when everyone's working from home.
> I’ve worked with people who could accomplish as much in two hours as I did in two days. Would it be “toxic” to work in such an environment?
No, why would that be toxic?