All the time? Most "RESTful" (meaningless term, by the way) APIs have a series of convoluted steps that are annoying and tedious to hand-roll.
Authentication is a good basic example. Different APIs will have different requirements for authentication headers. Successful authentication frequently requires multiple steps, like an initial request to get a session token. Many APIs will also require your request to be signed according to their specifications, and to stuff that signature in a header with a special name.
Every API expects to receive data in its own format. I don't want to have to make a bunch of validators that lay on top of your data model. These can and should be provided, and Swagger makes that automatic.
As eropple states, if something is making me do all of that by hand, I'm already looking for something else that will allow me to just say "import project_lib; project_lib.authenticate(); ..."
If you're just talking about something that is read-only and that only emits a single field I care about, sure, no need for a library. Something like the free version of ipinfo.io would be a good example.
Stuff that's more complicated than that, yes, it needs a client library.
