Under the CFAA, they do have the right to determine what constitutes authorized access. If they say you're unauthorized for using the wrong buttons on your mouse, then you're unauthorized. It's treated very similarly to trespass on private land.
You can try telling the judge it's baloney, but if he's going by current precedent, he probably won't agree with you.