>Have you run your test multiple times with varied training sets?
Some background - I've been working on this for several months now, experimenting with various CNNs (settling on a modified LeNet), hyperparameters and parameters. The bulk of the experiments have been failures - a typical scenario is the loss function decreasing in the training phase, but winding up unable to correctly predict labels in the test phase. There has been a progression however - from predicting either label A or B (but not both), to predicting both (but no better than chance), to doing a little bit better than chance. Maybe I'm fooling myself - I don't know. That's why I've been scouring the Internet for similar kinds of work (and not finding anything truly useful) and now reaching out to HN. If there's work out there that definitively rules out working in this space, I'm all ears. Otherwise I'm going to keep on experimenting. (edit:formatting)