If the button image is hosted on Facebook's servers (and it commonly is), they have a log of your request for it, including the page it was on (from the "Referer" header). This request is sent when you load the page, without the need to click on the button. Every site you visit that includes a Facebook resource gives them the ability to collect data about you and your habits. These are often part of a site's template, included without regard to the page it might appear on. You'd be surprised what a Referer URL can reveal about you.