The downside to using a mutable reference instead of passing around ownership is just that it would be awkward.
You'd have to write two extra lines of code to set up y and z separately from calling add(), and you'd have to make them mutable which is an extra mental burden.
And an immutable reference is bad because it would force extra objects to be allocated, wasting time and cache space.