The downside to them is that there is that the base class in the inheritance chain (BaseModel -> Model -> YourModel) has some magic that is difficult to reason about (the upside being the other side of that same token!). Thankfully, it's reliable and there's rare occasion to dive in.
Maybe this is a me problem. I don't know how to make a nice abstraction layer like that without inheritance, so I think inheritance is sometimes the answer.