Having it in the tile list makes it super easy - make sure the Modes tile is near the top, so it becomes a single swipe to switch it on and off.

But app contrast is generally better on Android than on iOS (in particular, Google Maps is perfectly usable in greyscale on Android) so you might find you don't need to switch it as often as you might think.