You are conflating rights with entitlements. The nature of non-enumerated rights is clear: Everything that is outside the enumerated powers of government, until the people decide to add to those powers through the amendment process.
We are safe from judges doing "anything."