You have to realize that search is not AI. It's pattern matching. And the string "shirt without stripes" matches really good with "shirt with stripes". Levenshtein distance is 3.
If vendors would use the term "shirt without stripes" than it would match great, but they call it "plain shirt".