I remember watching Olympic diving years ago and the commentator was making a big deal out of the fact that the divers were spreading their knees apart (apparently a flaw) but that the judges weren't picking it up because their viewing angle on the dives didn't allow them to see it. So the divers were taking advantage of a literal blindspot in the human judging to make their dives easier to perform.
So really, changing to AI judging is just going to change what the competitors optimize for in an arbitrary way. It's not going to fundamentally change things.
It will take away most of the fun(?) of arguing a judge's ruling and gaming them, but I will fundamentally always support making games fairer and objective'er.
After, they will do stuff that the "AI" will notice.
Goes for any physical sport that needs human judges instead of something measurable, like running speed, for their competitions.
Pretty sure if you do competitional martial arts you train a subset of techniques that will be very visible and score you points as well.
Basically catchers will try to catch a pitch using a motion that slightly tricks the umpires eyes into calling a marginal ball a strike.
It does have a statistically significant effect and since it has become widespread catchers are judged on how good they are at pitch framing since that is a small but significant part of the value to help the team win.
That whole concept goes out the window if MLB ever changes to just using cameras to call balls and strikes!
The problem comes when one (or multiple) measure(s) are an approximation of some ideal measure for the target. For example the famous example [1] of the measuring rat tails to determine killed rats (target). The faulty assumption here was that the only way to hand in a rat's tail is by killing them.
AI might be able to radically change sports like fencing and HEMA. I could see the development of match judging AIs trained by experts. Anyone who has fenced knows the frustration of the fallibility of human judges.
It's also possible that AI judging could benefit martial arts, bringing them closer to their practical roots, by counteracting the rule-gaming competition rules. ("Sport-ification" of martial arts.) Instead of allowing just any touch to count as a point, only hits with enough body structure to allow damaging force could be allowed.
Many HEMA competitions try to implement this, but expert calibrated judging AIs would make this an order of magnitude better.
LOL! I think the assumption that an ML system would be robust across these contexts is not well founded. Things as innocuous as the lighting, background, or placement of athletic tape could easily wreck a model like this.
The thing about human judges is that the modes of failure are pretty well known at this point; and human perception and visual processing is pretty robust to perturbation.
Right, professional athletes will go to much greater lengths to exploit a scoring sytem if they can get an edge from it. The AI sensors might be much better than a Wii, but that won't fix the core problem because they will be studied and gamed much more diligently.
Take figure skating as an example. Since 2006, ISU has introduced a similar judging system - IJS – that uses points for elements and an increasingly complex code of rules on how to assign those points. Each element has a so-called "base value" and judges determine GOE value in the range from -5 to +5 for each element. The computer averages that and takes trimmed mean to produce the final points for the element.
Those base values and points are logically assigned by the complexity of elements. More complex elements cost more. Great skaters can perform more complex elements. Sounds logical, right?
What has happened, though, is that using these points as a target has changed the way how skaters train. Why spend time learning "easy" elements that give little points? All time must be spent now learning "complex" elements that give more points. In a way, before, IJS skaters were chasing overall greatness; now, they chase points.
"Skater's greatness" – whatever that means – is hard to define, let alone measure. The ability to perform complex jumps was a consequence of it, not a cause. Using points as a metric changes the whole system of training, and a new generation of skaters can perform insane quads (often because of pre-puberty body sizes, though), but no one would call them great skaters.
I'm not even starting with how these points affected artistry. "Program components" (PCS) that should measure artistry are practically a joke now. You basically can multiply jump scores by some constant and get PCS. [1] All skating programs look the same now and are very predictable because of points maximization.
Back to AI. The rules for figure skating become more and more complex each year in order to duct-tape the constantly emerging issues. Many judges admit that complexity is so high that the human brain can't even possibly apply them in real time and has to resort to cognitive shortcuts. There are some early flirtations with using AI for judging figure skating [2], but nothing serious yet.
As outlined in the "Categorized Variants of Goodhart's Law" [3], one of the major reasons of this effect in sports is not just the fact that elements != greatness, but the fact that there is no shared definition of what is a True Goal of this sport. What exactly judging system try to measure? What would constitute the best skater?
Where there is no explicitly stated True Goal, any proxy variable used as metrics (like "ability to perform complex elements") will be far from good and will be distorting the system even further.
Eventually, this boils down to the structure and decision-making in the international federation that governs this sport.
And these things cannot be solved by AI. In a way, using AI is a yak-shaving for ever-increasing rules of an inherently wrong judging system.
[1] http://libjournals.unca.edu/OJS/index.php/mas/article/view/2...
You agree that "Skater's greatness – whatever that means – is hard to define", but I would say that the problem is then the idea of competition for this sport and not "how you judge" / "how you measure".
There is no inherent need of competition for every sport in my opinion. There can be prizes, juries, etc, but if people would accept that in the end it is subjective might make it easier than overly complex system to rate "fairly" something unclear (doubt anybody would want to put rules on how you rate a film for an Oscar - they just know it is "fuzzy").
I disagree. Competition is a defining property of ‘sport’.
What’s wrong with calling figure skating an art form? Nobody calls ballet or violin playing sports [1], but there are “prices, juries, etc” for them.
I would split various activities in a group where objective measurements are more or less possible (they never fully are) and one where they aren’t.
The first would consist of:
- games, where your opponent has significant influence on your behavior or performance (basketball, judo, billiards, etc). Basically, if you can play defensively, it’s a game.
- sports, where they don’t (archery, skiing, all forms of athletics; I would include the jumping events, even though your opponent’s performance may significantly affect your strategy. Etc.)
The second would be ‘arts’.
[1] if you look at the modern Olympics, painting and other art forms were events at some time, but that, but even then, did people call those sports?
So, again, the goal of the competition is to incentivize athletes to show their best. Accidentally, we need some way of comparing results here, and that's where the judging system comes into play. It has (or rather should have) a nice bonus of giving feedback to the athletes on what to improve upon.
Unfortunately, the majority of people in sports equate sports with competitions. Even some definitions of sports contain clauses like "...in a competitive fashion...". They strongly believe that without competition, there is no sports component.
Yet, every four years, the world rediscovers a unique Norwegian sports policy for children. [1][2] They banned competitions for kids till the age of 11 in the late 1970s. You can have competitions, technically, but you are prohibited from ranking kids and publishing results with rankings. That, of course, is based on the studies of the reasons behind drop-out from sports rates and the science of motivation (self-determination theory, mainly). As a result, Norway has 1st rank in medals in the Winter Olympics and 9th in both summer and winter (they even won gold in beach volleyball!) and the highest sports participation rates in the world (9 out of 10 kids and 7 out of 10 adults, I believe). Sports != competitions indeed.
Another example I like is Magnus Carlsen refusing to participate in the Worlds Chess Championships for the 11th time because it simply doesn't challenge him anymore. [3] Again, competitions are not about "finding who's the best" (that's a byproduct). It's about challenging to show an athlete's best. In the case of Magnus it failed to achieve this goal.
[1] https://www.washingtonpost.com/kidspost/2022/03/03/norway-is...
[2] https://www.theglobeandmail.com/sports/olympics/article-norw...
[3] https://www.dailymail.co.uk/news/article-11034789/Magnus-Car...
In Figure Skating, HNers should also be aware of the issues surrounding the current sport of figure skating:
1) Women's Free Skate is the marquee event in the winter Olympics. Not other event comes close in the ratings, and therefore, the ad buys.
2) Russia. Figure skating, and especially women's skating, is completely jumbled up by the interference of Russia. To sum up a very complex topic (poorly), women's figure skating currently has a strange and deep relationship with Russia's ruling elite and has had one for a few decades. That leads to many scandals and a lot of bribing of officials. It's still playing out from the 2022 Olympics. The US team still has not gotten their medals from the team event.
If you've not seen the end of the 2022 women's free skate, do yourself a favor and watch it: https://www.youtube.com/watch?v=Pgiuo4lHFt4 . The meltdowns here are just so emblematic of everything that Russia is today and the travesty of the Ukraine war. The drama leading into this event and the drama coming out of it are much better than any reality TV show.
3) Declining interest. The laws of physics are not kind to figure skaters. The push for quads is only leading to the push for qunits and hexes. The effort required to advance the sport is going to be larger than time allows for training. At which point, probably sometime around the 2050 Olympics, the sport will be 'solved'. Combine this with the explosion of new sports full of frontiers to be broken, and figure skating will no longer be the marquee event. Training so intensely for such meager gains is going to be a hard sell to the boys and girls that are to be the medal winners.
I would just add that the russia thing is a direct consequence of the judge's system change (where russian officials are very much involved). The reason why russia was dominating medal standings for women (and not for men or ice dance), is precisely because russia could afford a stream of pre-puberty girls who win a medal at the cost of their physical and mental health.
Also, needless to say, the russian sport system is still a direct copy of the soviet sport system, which was essentially created by Stalin when USSR first decided to participate in the Olympics in 1952 (guess, why USSR didn't care about the international sport before WWII). The way how sport is organized in russia goes against the sports autonomy principle that is sacred in democratic countries and is embodied in the Olympic Charter and European Sports Charter.
I've watched a figure skating routine recently for the first time in almost 20 years (I think the last show that I had followed was the 2006 Winter Olympics) and I was surprised to see how technical it had all become, how routine, there's no "artistry" left. It didn't use to be like this.
Similar thing happened with the NBA, now everyone chases the 3-point shoots and in so doing whatever it is that had made the NBA one of the best sports shows in the world is losing its shine. In this day and age Jordan would be seen just as a gimmicky player.
That’s happening to, like, all of life, across the world. Beancounters have conquered the planet
I very much disagree with your Jordan comment. I think MJ could develop a 3-point shot. If anything, I think he could have shined even more in this day and age .
Today's NBA is more dynamic and "positionless" than previous rigid coaching, giving rise to players that would have otherwise been forced to play a role. Jokic & MJ, Haliburton & MJ, or even Sengun & MJ would be scary.
All amazing passers that could feed the ball to MJ, but also put the offense in a bad spot. If you guard MJ, then Jokic, Haliburton, Sengun, etc. could all shoot a 3 if left wide open.
No part of this is true or even really makes sense.
Skaters being rewarded for performing as difficult elements, as close to perfection as possible? I'm all for it!
Same for gymnastics.
Is this really a result of the current points system? Even in old-school, 6.0 human judging, wouldn't it be preferable to reward a much more difficult routine with higher scores? Is there any reason not to reward, say, a triple-triple combo better scores than a similar routine which tops out at a double-double?
In a way it was similar to how classical music contests are judged. No one could claim that they can measure the performance, so they rely on the subjective and imperfect judgments of the humans who are considered experts in music. One judge might be consistently high-scoring, another might be consistently low-scoring, and they might look at the different aspects (technicality, musicality, etc.). They just try to capture their ranking (voting) distribution.
In a way, the old system was more like voting. Even if the judge can't formally explain why s/he thinks skater A has better skating skills than B, the system would capture their preference. (And of course, this comes with all sort of biases – sequential bias, national bias, etc). But an important distinction is that in this system there is no proxy variable to optimize for.
In the IJS it's very clear what to optimize for. So everyone is doing it. Also, one of the major changes, of course, was eliminating compulsory figures from competitions. Modern skaters don't even know they existed as a thing. Nowadays, everyone knows that you win a figure skating competition not by having Patrick Chan level skating, but by rotating your body in the air quickly – that is what gives points. So in modern clubs all over the world people start learning jumps in the first half of a year of practice. Which, ironically, leads to the lower probability of them actually mastering the jumps, because you still need insanely good edge control for take-offs and landings.
MLB is weird because everything needed to ref a baseball game does not need any advanced AI. MLB simply refuses to let robots ref games.
I'm sceptical. An AI that could view the whole playing field at all time and instantly spot and call every rule infraction with 100% accuracy would pretty much ruin any game. Knowing when to ignore an insignificant rules infraction in favour of letting the game flow is part of being a good ref. Plus, as a player, trying to get away with some sneaky bullshit when the ref isn't paying attention is part of the game.
If anything, I think that refs need to be enforced more to call games a particular way. The officiating that occurs during the playoffs is much better, and IMO should be done that way during the whole season rather than only the playoffs. Refs let defenders be more physical, they call more rule violations, etc.
I'd fear that an AI ref could possibly lead to even more fouls being called, since a ref could miss things depending on where they are in the floor. More fouls being called would just slow down the game.
We tried it, it was shit, we lost the most important part of the game for fans (celebrating goals, of which soccer has few), let's go back.
Even though it’s a subjectively judged event, the scores somehow have three decimal places!
(Nevertheless, there are still issues with bias, where athletes or teams who benefit from a particular judge's biases might have their errors more gently scored. For example, on the bars, gymnasts are required to hit a handstand position within a 10-degree range of vertical. But a judge who is impacted by a bias, or a certain viewing position, many not notice that a particular handstand was at 15 degrees, and not deduct accordingly. This is where the AI judging may be particularly helpful.)
[1] https://www.gymnastics.sport/publicdir/rules/files/en_2022-2...
So at what point does a sensor of some kind that does some computation become 'AI'? Is this just marketing buzzword bandwagoning?
By going backwards in time — I'd guess sometime between 2010 and 1995.
I use speeding in vehicles a lot. The point of a 55MPH speed limit isn't really that cars should be going exactly 55MPH at all times. It's an approximation of a complex situation that could handle only one number. But it's not actually a law of physics. Rigidly handing out tickets for people doing 56MPH is missing the point of speed limit. (As are many of the "You're doing X over" flashing signs, generally put in places that are precisely where the speed limit is too low and what people are doing is actually fine.)
Now that we finally have the technology to determine whether a gymnast was or was not one millimeter over some line, the question should be raised, is that what we actually want to judge on? Because just because we humans thought we were judging on that, just because we thought that's what we wanted, is it really?
It is this examination step that has been missing in every rush to insert computers into some judging system. We thoughtlessly reify accidents of history and what used to be easy to judge. At least in the case of gymnastics we're not going to send anyone to jail because of such thoughtlessness, but the sport could be destroyed. In the end, it still needs views, however abstracted the sports may seem to be from that, and after the novelty wears off, seeing humans out there competing for the most robotic and soulless performances runs the risk of gradually, but steadily, destroying all support for the sport.
Now, personally, I'm not sure this wouldn't be a net gain for humanity in this particular place. But it's certainly not what the gymastics community as a whole wants and they really need to slow down and apply some thought to what they really want before they let someone reify that into code.
They won't. Nobody else is either, for things far more important than this. But they should.
Everybody recognizes it's not perfect; but in reality a cop isn't going to a write a ticket for 56 in a 55 (unless they have some ulterior motive). They might write a ticket for dangerous driving at 45 if the conditions are bad enough. There's still some human element, but we try to minimize it because it creates an opportunity for corruption and inequality in enforcement of laws.
In the case of sports the judging is supposed to have a human element, but the judges are supposed to only care about the performance and routine and not stuff like the nationality of the gymnast.
Which is an unrealistic hope frankly. So pick your poison; either we take subjective human judgement and accept it will wanted and unwanted subjective elements, or we do this stuff and remove subjectivity.
[0] There's always measurement error but that's a separate issue.
I immediately recognized that this essay would’ve been accepted by my teacher when I was a kid, and then remembered just how useless I thought essays were back when I had to write them. It feels like, with AI, I had proof that these essays were just “busy work” rather than anything else. As a student, I was just taking the prompt that my teacher gave me and expanding it into an essay, and there was no information in the essay that wasn’t already in the prompt itself. Why is it useful to take something that could be said in a single sentence and say it in 5 pages? I wasn’t encouraged to think about the book I was writing the essay about because my thoughts were seen as secondary to doing the technical work and simply copying the style and format of the essay, but now, with AI, that part had been automated and all that education was for nothing.
It seems like Gymnastics really is, at it’s heart, a very technical act. Your goal is to achieve perfect normal angles for some reason. But as robots begin to dispel the haze around technical work, we stop being so interested in technicality. You can see it as an attack if you’re really invested in that technical stuff, but really you gotta ask yourself why you were so interested in it in the first place.
https://www.youtube.com/watch?v=CinAYBZYANg
To me it looks very shaky, e. g. at 0:20-0:25. It seems like they're not using an underlying physical model that takes momentum into account. Also it looks like the system is interpreting the movement of the ponytail as head movement.
Does FIG have powers of prosecution like UK Postal Office? Asking for a friend.
Oh boy, considering the Horizon debacle going on in the UK, I can expect a similar story in Olympic gymnastics in the near future...
