Seems like out of the major libraries (TF/Caffe/Theano/pytorch), pytorch is the only one to have a core that is C (the TH*). It's not exactly a small library, though. One small library that is in C and has some state-of-the-art features is Darknet (https://pjreddie.com/darknet).

That said, seems like directly using the C++ API was a major use case here, and it looks fairly clean to me.