If you demand your pet assertion, you have to maintain some extension on top of the library you use, or wait for upstream to add it -- leading to the maintainer having to maintain a 2000 function library of everyone's pet assertions, which isn't fun, and which they probably won't do.
Another problem with assertion libraries is that edge cases are buried in details far away from the test you're writing. What equality function does To.BeEqual() use? Is null equal to undefined? You really have to hope it doesn't matter. But if the assertion is the if condition, you're not adding any new semantics to your language; you know what == does, and the code says ==, so you don't have to guess.
Finally, error output suffers the more generic you make it. You can review a screenful of hand-written assertions, but autogenerated error output always seems to be one screen per error, meaning that you have to do more looking around to fix multiple tests. Minor complaint, but it's the nature of writing a formatter for interface{} vs. exactly what your test tests.
Overall I find it not worth it. Most of the time you'll be writing a table-driven test, and you're only writing the assertions a handful of times for tens of test cases. (A comment below complains that table-driven tests don't point you at the failure; giving you a vague error like "test 42: fail". I personally add a free-form "name" to every test case to avoid that particular problem.)
My main complaint with Go is how the official style guides don't make conventions around the hand-rolled assertions clear. At least when I was learning Go at Google, you pretty much had to use the form:
if got, want := len(foobars), test.wantFoobarCount; got != want {
t.Errorf("foobar count:\n got: %v\n want: %v", got, want)
}