It started last Thursday. Suddenly any post or chat message containing the string "laut.fm" would be blocked with the following error message:
You can't post this because it has a blocked link
The content you're trying to share includes a link that our security systems detected to be unsafe:
laut.fm
Please remove this link to continue.
If you think you're seeing this by mistake, please let us know.
In addition our facebook page disappeared. It reappeared on Sunday, but any posts between last November and now where missing.We have no further information about what might be offending facebook. Our site is pure organic user generated content. We don't do any special SEO tricks.
We and several of our users reported the blockage via the link provided in the error message to no avail. We reported an issue with our facebook page here: https://www.facebook.com/help/contact/164405897002583?rdrhc . When trying to debug the issue via the facebook debugger (https://developers.facebook.com/tools/debug/og/object?q=http%3A%2F%2Flaut.fm) one gets the error message "Failed to get composer template data.". Using a link shortener to post laut.fm links won't work either. I worked through the points on this list: http://stackoverflow.com/questions/13185406/facebook-wont-share-a-link-to-my-site without success.
The most frustrating part is not even being blocked, but being left in the dark about the reasons.
We are out of ideas. I use this HN post as a last resort. Any help or hint on how to proceed would be highly appreciated.
Niko.
Issuing df on a "completely filled" disk will give you this:
$ df Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 9611492 9123252 0 100% /
Notice the difference between "Used" and "Available"? That's the space reserved for root processes, so that careless users can't crash their system just by filling up the disk. See stackexchange for further discussion (http://unix.stackexchange.com/questions/7950/reserved-space-for-root-on-a-filesystem-why)
While df does Used/(Used+Available) to calculate the disk usage percentage, NewRelic does Used/1K-blocks.
This means when our disk got filled by a rogue process which wrote a huge logfile, the NewRelic disk usage measurement got stuck at 94,9%. As the NewRelic default threshold for disk usage alerts is 95% the alert never got triggered, the alert email never got sent and we had a service outage because the streaming server process crashed when it couldn't write to the disk.
End of story (to put it in NewRelic support staff words): "[…] you should really set to threshold under 95%, or tune your filesystem so that you have <5% reserved […]". For them the current state is intended behaviour. So heads up if you rely on NewRelic disk usage alerts!