In the end what matters to perception may be the attendant fuss, as you say with commentators and organised competition, but I grew up with the Football Pyramid†, which provokes healthy distrust of such things in trying to define a sport.
† Unlike with US sports like the NFL, in England Soccer is organised in a vast sprawling hierarchy of leagues, the Pyramid, with a system for teams to be promoted from a lower league to a higher one, or contrariwise demoted to a lower league, depending on their performance each season. Anybody can start at the bottom, if they can put together a group of people to turn up and play. In principle a group of friends or colleagues could create a new side, "Speen and Dean FC", that was just ridiculously good, and get promoted every season, until after a few yeares they were playing against semi-professionals, and then facing actual professional footballers in stadiums, and then playing internationals, it's all possible. Incredibly unlikely of course, but possible. And so this emphasises that actually the game you're playing kick-about with some friends, is the exact same game somebody gets paid eyewatering sums of money to play on national TV. If it's a "serious sport" is clearly just a matter of perspective.