Unless you were coerced or the contract came into being in some other illegitimate way, you certainly have a moral obligation to fulfill any promise you make, contract or otherwise.

<Quick moral theory> To be moral, an action must preserve agency of those involved. When any action is performed, a reciprocal standard of action is established. (When I do something, I am willing that it is ok for you to do it to me.) If the action in question strips another person of their moral agency, the actor initiating the action is also accepting that it is permissible to be stripped of their own agency. This is where the contradiction exists. If we authorize the removal of anothers agency, then we are authorizing the removal of our own agency as well, and thus our ability to determine morality.

Cliffs Notes: If you screw someone over, you're authorizing others to screw you over. This perverts your sense of morality and you are no longer sufficiently able to decide morality.