The founders believed in private property rights as much as free speech. Property was even a requirement for voting in most states.
This explains why Samuel Adams and the Sons of Liberty happily destroyed private property in the Boston Tea Party.