It is, but this sort of approach is easily applied to transformations (in fact, an earlier GAN HN submission was "Generative Visual Manipulation on the Natural Image Manifold"
https://people.eecs.berkeley.edu/~junyanz/projects/gvm/ , Zhu et al 2016b ). So you can start with a lousy sketch and transform it until it works, or you can start with a pre-populated set of handbags, recognize the one closest to what you want, and tweak/sketch that.