undefined | Better HN

0 pointsphkahler1y ago0 comments

>> Like, that's the only REAL reason? Not the technological or ethical implications? The dangers in providing people with no real concept of how any of this works the means to evaluate themselves?

On the surface those all sound like additional reasons not to make it available. But they are also great rationalizations for those who want to maintain a monopoly on analysis.

Personally I found all the comparisons to other AI performance bothersome. None of those were specifically trained on diagnostics AFAICT. Comparison against human experts would seem to be the appropriate way to test it. And not people just out of training taking their first test, I assume experts do better over time though I might be wrong on that.

0 comments

jarrelscy1y ago

Developer here - its a good point that most of the models were not specifically trained on diagnostic imaging, with the exception of Llava-Med. We would love to compare against other models trained on diagnostic imaging if anyone can grant us access!

Comparison against human experts is the gold standard but information on human performance in the FRCR 2B Rapids examination is hard to come by - we've provided a reference (1) which shows comparable (at least numerically) performance of human radiologists.

To your point around people just out of training (keeping in mind that training for the FRCR takes 5 years, while doing practicing medicine in a real clinical setting) taking their first test - the reference shows that after passing the FRCR 2B Rapids the first time, their performance actually declines (at least in the first year), so I'm not sure if experts would do better over time.

1. https://www.bmj.com/content/bmj/379/bmj-2022-072826.full.pdf

rscho1y ago

Someone downvoted the author !? This site never ceases to amaze.

j / k navigate · click thread line to collapse