Even if some of the components in Mathematica are not state-of-the-art, there's a huge synergy benefit from having everything together in one environment, using the same datatypes. I think that's the big benefit of Mathematica and Matlab. Python can match (or exceed) a lot of their capabilities but you spend all this time with package managers, converting data types, etc.

What you said plus the accuracy and precision to which results are computed. They're effectively limited only by the amount memory you have and your patience. And even patience isn't required unless you're calculating pi to trillions of digits. It's really fast.