ARC-AGI has a hidden private test suite, right ? No model will have access to that set.

Its almost certain that it was, but the purpose of this puzzle benchmark is that it shouldn't really be possible just to be memorized by the amount of variations that can be created and other criteria detailed in it.