Well, the reality is complex and OSM wants to be precise :-)
Apps shall then make it easy to look for stuff and OSM labels are quite low level, they usually have their specific/internal representation of things. All this to say:
- I doubt CoMaps allows searching for "amenity:drinking_water", at least I haven't found a way to do this.
- That's what the higher level and more user-friendly Water category is supposed to be for. This will limit possibilities, but optimize for the common cases. If the Water category surfaces wrong things or doesn't surface things it should, it's a bug that needs to be fixed.
- You might have better luck with apps for editing OSM for working with OSM labels directly but appart from basic use of Street Complete and Every Door, I haven't explored complex OSM app editors like Vespucci on mobile.