what you're describing would make end-running the GPL absurdly easy. Vendors would just distribute vanilla Linux source code and in another file distribute their not-a-derivative patches under their own proprietary license.
Patches are obviously a derivative work. No one spontaneously describes deleting several lines of code & then replacing it with other lines of code.