It's not difficult, but why do they make it difficult to "just go to the previous window", whose default shortcut was alt-tab on almost all systems (windows + gnome + kde)?
Maybe you have a specific workflow with a specific set of apps you use in a specific order in which case your brain is hard-wired to know whether you'd like to switch app or window within an app. I personally find it much easier when i don't have to think about it and can just "go back" to what i was doing previously.