Yes.

Usually size comes first: big red metal ball.

red big metal ball sounds wrong to me and so do metal red ball and red metal big ball.

But I'm not sure if it can be succinctly expressed as a general rule. It's just something that native speakers do without thinking, seeing it written in the wrong order or hearing it said in the wrong order just feels wrong to me. Perhaps a linguist could provide a rule, or at least a rule of thumb.