Unless you are using malloc; and/or you don't need to do anything when the pointer is freed, (the pointer doesn't reference anything else that needs to be freed or released,) there's no way that the library written outside of your runtime knows how to free your memory.

Or to put it in a different way: Passing pointers to a native library is a small amount of what your application does and you still benefit from the garbage collector when you are running inside of your own language.