You'll always need platform-specific recompilation for native executables, as the executable formats are different on different operating systems. The only exceptions are emulation systems like WSL and Wine.
I should have added it to my first post - SDL provides precompiles libraries for different platforms. All I have to do is to distribute them with my game. Does it solve the problem?