An In-Depth Guide to Contrastive Learning: Techniques, Models, and Applications (opens in new tab)

I've been messing with contrastive learning in the CLIP-like "minibatch" style for applications where correspondences between modalities within an example are a relevant source of information, and have been pleasantly surprised by how much this can actually in some cases speed up or otherwise aid with learning different objectives framed as decoding each modality's embedding for a different downstream objective. It seems that tying the representations together in this way can serve as a kind of semi-orthogonal correcting force on the embedding's ability to retain its original reconstructability while not constraining it too much to shift the representation for other goals, and many properties of the resulting models (like the obvious one of having interchangeable downstream decoding functionalities regardless of which modality is used to encode) suggest to me that it might be another tool to use when pursuing a better theory of objective balancing, as well as significant practical use in applications obviously. I'm surprised people aren't making more random contrastive embeddings for sets of "non-obvious" coupled representations

I really gotta find a job where they pay you but people won't freak out if you can't make a simple business case for every detail of what you're doing in a given week. Does that even exist anywhere without like a PhD from rich people schools? Aside from the moral issues this myopic mindset of businesses just refusing to trust or support their people in any sense sure makes it hard to believe they're actually capable of "innovating" anything. Necessity may be the mother of invention, but focusing on speedrunning exploiting will always leave exploring neglected