According to the text of Article 50, a withdrawing state remains bound by the Treaties (and thus by EU law) until either (1) an exit agreement is made and enters into force; or (2) two years pass after Article 50 is invoked.

So the UK can't 'withdraw unilateral whenever it wants' without creating a giant legal problem as its erstwhile partners attempt to enforce its (still extant) obligations.