I don't understand why so many people in these discussions are so keen to ignore the impact of scale and accessibility when it comes to new technology, and specifically this new technology. Yes, most dangerous things that can be done with AI can also be done by humans.
Is it not meaningful that these dangerous things can now be done far more cost effectively? It would've made no financial sense in the past to spend hours/days creating a fake mushroom identification book. You'd almost certainly never get enough sales to make it worth it, especially once people realized your book was nonsense and potentially dangerous (getting you delisted, as it seems like the seller did with this book). Now you can just ask an AI to "generate 100 book ideas, scripts, and images." Who cares if almost all of them make very little money when the time and $ cost to creating them is near zero (it looks like this book was physical but especially for digital books, videos, etc.).
Is it not meaningful that dangerous things can now be done (or may soon be able to be done) by more people with less skill? The time/money investment required to learn skills with the potential for destructive use is IMO a strong filter against the people who would do those destructive things in the first place.
OpenAI and I think Google have talked about making intelligence "too cheap to meter." I'm not sure that is a good thing. I could be convinced, but every poorly thought out dismissal of AI dangers only makes me increasingly sure that we aren't ready for it.
For those who are less concerned about AI dangers, consider maybe that premature reckless deployment of AI also has serious potential for generating social backlash that might end up slowing or halting the adoption of AI for positive purposes. The average non-tech person I know thinks of AI as a useful improvement over Google, and maybe thinks AI generated images are cool, but they aren't in love with it. It could disappear tomorrow and they'd shrug. In fact, public sentiment against AI (in the US) seems to be rapidly increasing, and is bipartisan [1]. If the promise of AGI is achieved this will go into overdrive should (when?) mass-unemployment happens. Look how far right-wing politicians are able to go by drumming up fears about immigrants stealing jobs.
https://www.pewresearch.org/short-reads/2023/11/21/what-the-...