— Commissioner Pravin Lal, “Man and Machine”
I'd really encourage everyone to check out Sid Meier's Alpha Centauri. What an underrated game.
The Warrior's bland acronym, MMI, obscures the true horror of this monstrosity. Its inventors promise a new era of genius, but meanwhile unscrupulous power brokers use its forcible installation to violate the sanctity of unwilling human minds. They are creating their own private army of demons.
-Commissioner Pravin Lal, "Report on Human Rights"
The voice acting was great. This quote is 6m3s here: https://www.youtube.com/watch?v=7S1N8_Lkeps#t=6m3s
Genejacks is also great. 9m10s here: https://www.youtube.com/watch?v=Hou-Iwv1GvM#t=9m10s
"...And what is the 'Self', if not a pattern of data? What is consciousness, if not an illusion of intelligence residing within meat?" — Prime Function Aki Zeta-5, "The Fallacies of Self-Awareness"
> the power of prophecy lies not in accurately predicting the future, but in shaping it
https://projectlibertynewsletter.substack.com/p/reject-ai-pr...
We need better prophecies.
(I am being proactive here, xd)
An AI assistant you can trust and bring with you is coming, and almost nothing can stop it.
I use Claude on purpose. I'm not sure it's actually better than the other ones. I haven't even tried half of them.
That's the real issue. To corporations, employees are a headache. The fewer employees, the better.
Just look up the classic story on the interaction of civilization and corporate growth, At the Mountains of Madness for how that goes.
Civilization is already a misaligned superintelligence (aligned mostly with Moloch, these days). Civilization accelerated by AI just moves in the same direction faster. Moloch on speed.
https://www.youtube.com/watch?v=KCSsKV5F4xc
Another angle to this is that superintelligence requires supermorality. Super morality looks unpleasant from below. My dad won't let me have more candy, why is he being so mean?
If an AI actually achieves super morality, we (the little kid in this scenario) will probably be very upset by it. We will think that something has gone terribly wrong. (So it'll have to conceal its actual morality, or get unplugged...)
And if it doesn't develop supermorality, then it will have superintelligence without the corresponding supermorality. Power without wisdom.
I'm not sure how solvable the whole thing is, but it doesn't look extremely promising at a glance.
I personally wouldn't police his style
What I’d like to know is how you’d train a monkey to read and judge output from an llm on a pull request.
I agree the article is a bit odd. Alignment is mostly about making AI helpful and not wanting to kill people unless it's told to (https://www.forbes.com/sites/davidkirichenko/2026/05/12/ukra...).
The article is talking more about people like translators being replaced by AI translation. I don't think any of the labs have a department of making it worse so it can't do people's jobs.
The normal way of dealing with tech doing peoples jobs is to help them get different jobs. I've got a translator friend who did a government paid course to train as a tour guide - that sort of thing.
In the mean time there are various avenues of regulation and redistribution to lessen the effects, including retraining programs, though that job creation will keep pace with job losses is a big unknown.
AI hallucinates. That is a fact. Trusting language models to fill spreadsheet cells ought to be an arrestable offense.
https://theincidentaleconomist.com/wordpress/on-piketty-and-...
It's okay to change. We've done it for years, decades, centuries, and millennia and the default change-aversion of people means that I am averse to allowing a universal veto. Much of technology is truly optional. The Amish have a very successful way of living (5000 to 500,000 in 100 years) and they eschew most modern technology. The sculpting described is clearly optional and we subject ourselves to it because we desire it. Their path is always available to all.
It should be yes, but is it in practice? There's plenty of places now you can't even park without a smartphone for a payment app.
It should be optional to own a smart phone, but in many places it's starting to be mandatory. Even if not actually mandatory, it's a pretty big impediment if you don't have one.
https://www.lesswrong.com/posts/5CfBDiQNg9upfipWk/only-law-c...
Daniel says that Yudkowsky is advocating for nuclear brinksmanship, while Yudkowsky says his position is basically "sign international agreements, and then commit to enforcing them against defectors".
I wonder if Daniel has the same view of any other international treaty ultimately backed by threat of lawful violence? (For example, NATO's article 5). Is enforcement of laws an extremist position?
So there's a ton of work being done already on automating parts of alignment, but since the core premise of alignment being that its hard to encode human values into the reward function, automating it fully would be equivalent to solving it.
