The recent progress in multi-modality retrieval is to augment image search with text instruction, so called Composite Image Retrieval (CIR). Related research includes
https://huggingface.co/BAAI/bge-visualized and Google's
https://open-vision-language.github.io/MagicLens/. We built a live demo for people to play with it and get a feeling of the quality of this retrieval approach.
