http://www.amazon.com/Good-Great-Companies-Leap-Others/dp/00...
http://www.bloggingstocks.com/2009/02/23/from-good-to-great-...
I find "best practices" to be a useful checklist, especially in an area new to me. If someone says it's a best practice, I take a look. Ultimately I may decide their advice is of a little value.
Moreover, it's just a non-sequitur to conclude from 'mediocre company X does Y' that 'doing Y makes a company mediocre'. What makes a company mediocre is in the things they cannot single out in clear statements of the form 'we should change practice Z, as it is not good.
Today's "best practice" may be tomorrow's fad - or it may be tomorrow's common knowledge. The differentiating factor is usually the ratio of actual experience to wild-eyed exuberance that its proponents have. So I'm betting that Rails is just a fad, but several of the XP practices like unit testing and standup meetings will have lasting impact.
This is based on an assumption that at given point in time there isn't an ideal solution (or an approximation which is nearly indistinguishable) for a given problem.
I would disagree and say that while "best practices" can be a buzz word for industry consensus, there are plenty of "best practices" which are genuine best solutions.
This can be for a number of reasons, such as learning inertia on the part of users. You might be able to invent a "better" user interface convention but if it's unfamiliar or confusing to the majority of users used to an "inferior" convention it's not necessarily a better practice to use the newer convention.
State of the art sounds impressive, but it merely means the best we have at the moment. Inventive people constantly improve it.
"Crossing the Chasm" talks about how most people in a given industry prefer to do what everyone else is doing. Progress comes from the technology "enthusiasts" (who simply love cool new stuff) and "visionaries" (who are excited by new application of the technology to give them a x10 advantage).
There's also the distinction between competitive advantage and hygiene factors. For the latter, it's cheaper and more reliable to just follow the crowd than to experiment with risky new ways - save that for improving your competitive advantage.