> What's to stop one from coming up with novel properties which are unique to these systems?
Interestingness. Usefulness, perhaps, if you have a specific definition of 'useful' in mind.
> Is our commonly used system basically an arbitrary set of conditions?
It's an arbitrary set of conditions with interesting properties, and it's not always obvious which arbitrary conditions will have interesting properties.
> Do you think that some systems correlate better to how nature does things than others, and if so, which ones would they be?
Well, that is an interesting question, which means anyone who claims to know an absolute answer to it is a moron. We do know, for example, that vectors are a very useful tool to model a lot of what happens in physics, and that complex numbers are a compact way to talk about rotation, especially complex exponentiation (taking a real number, such as e, to complex powers).
> Ah, so it's unbounded like how Haskell's lazy evaluation treats infinity.
Yes. For example, a definition (the Peano axioms) of the set of the natural numbers (either the positive or non-negative integers; the set may or may not include zero, as conventions vary) states that the set contains 0 (or 1) and that for every number x it contains, it also contains x+1.