If you can retain board control there isn't a ton of leverage unless the VC wants to hurt your business out of spite... in which case you picked the wrong partner.

What the VC can do is ignore you, in which case why did you take VC money when you intended to pass up the networking opportunities? What if you change your mind about raising another round of funding in three years but you already burned the bridge?

You should also remember that your cofounders will be on the board. Often the VC only needs to convince one of them you need a change in direction. A lifelong friend as cofounder will stab you in the back without a second thought if they smell a billion dollars at stake. Hence the advice to be careful about who you get into business with.

If you take money at a high valuation, the VC will only be happy if you try to grow the company significantly and exit at some multiple of that valuation. If the founder decides not to do that, then one of the largest shareholders in the company (the VC) will be angry. At a minimum I imagine the founder would be negatively impacted by losing some of his/her reputation in the investing space and having angst knowing that he/she took a VCs money and then ignored the VCs interests. At worst, there could be issues at the board level and loss of control of the company.