> An autonomously self-optimizing postbiotic system has emerged, the rapidly growing factual knowledge and the general, domain-independent intelligence of which has superseded that of mankind, and irrevocably so.
> The BAAN-scenario is a logical instrument that can perhaps help us to think about some of the deeper aspects in the applied ethics of artificial intelligence.
You know, my big problem with AI theory is that people make handwavy and nonsense statements, then back out by saying it's just a "logic instrument" and doesn't need to be explained. In science we just call this fraud; the author writes assertions that have no basis in theory, ignore the limits of conventional research, and blatantly stoke the reader's fears of AI armageddon. If you want me to fear situations where AI might have control over my life, start by introducing realistic stakes or circumstances where it might happen, not godlike entities in a post-dystopia AI-run world. Self-driving cars or AI-assisted medical care is subtle enough to get the point across.
I know this might seem novel for 2017 literature, but anti-natalism (or applied nihilism) and AI fearmongering are not new concepts. Also - they're just kinda wrong. There isn't really a logical or historically successful basis for nihilism besides the inevitability of suffering. But that alone doesn't deny the concept of life, and humanity stands here now as evidence. If society was built by absolute nihilists, we would have nothing. AI owes it's existence to people who, however foolishly, believe in a future for humanity.
