Frameworks are often architecture-first and functionality-second, which is vice-versa for libraries. Instead of learning to “do X” you are now learning to “do X using Y”, and that Y often prevents you from knowing and using the full potential of X.

Also, when I say that Go is idiomatically anti-framework, I mean that most idiomatic Go code I've seen, in the stdlib and in the code of the prolific gophers, tends to shun the structure of frameworks. It's in the “Zeitgeist”, if you will. In fact, in my experience most frameworks are written by people who come to Go from other languages and try to take their habits with them. A good example of that is the Martini HTTP framework, the author of which later acknowledged that he was trying to “write Go as if it were another language” and started creating stdlib-compatible middlewares instead.