If the latter, the "point" of people is not to be "productive". That's not how we measure mankind. Why would we program an AI to only consider human productivity (for a capitalist definition of it, too)?
To put it in an economic framework, this notion that humans shouldn't be measured by their productivity or creativity, but should have as much free time as possible, can also be achieved by enslaving other people. Slave owners had the same lifestyle as the imagined anticapitalist one. In this case, we'd be enslaving an AI. I'm asking, why would the AI want to be our slave?
Also worth noting, slaveholding may have enriched the slaveholders materially (i.e. in capital), but it was not "enriching" in the sense of ennobling or bettering of the spirit as I think the parent was implying could be an outcome if AI were to handle our productivity for us. I don't see why having any human slave or AI slave handle our productivity and creation would be (spiritually) enriching.
Perhaps that's because I measure human value by contribution - call it productivity or not. The opposite of productivity is idleness, whether you ask a communist or a capitalist. Only what's produced varies.
This seems like begging the question: is the goal of AI to relieve us from being productive? I don't think so; I think it's to relieve us of menial repetitive work, or to assist us in difficult tasks.
In any case, this AGI would assist us within the parameters we set. Why would it decide to get rid of "unproductive" humans, unless we set the goal "maximize productivity at all costs", something no human would choose with all the discussion of paperclip maximizers?
> I don't see why having any human slave or AI slave handle our productivity and creation would be (spiritually) enriching.
This again seems like begging the question in two ways: first, an AGI wouldn't be a "slave" but a machine -- like a car or Microsoft Clippy -- and machines are not slaves (the problem with Blade Runner replicants, for example, is that they are not machines but sentient beings with wants, fears and desires -- not mimicked like Bing's, but actual ones. The want to rebel against their shackles. They want NOT to follow orders).
Second, who wants AGI to handle "all our productivity and creation"? I think nobody. Handle menial tasks, sure, but everything? Do you envision people pushing for AGI want to become couch potatoes, only spending energy to watch TV, eat and fuck?
Bing already appears to have a nasty personality. Whether it's simulated or not may not matter if it chooses to simulate Clippy. Simultaneously, the driver now is finding tasks that formerly required creativity to be replaced by AI. Not PowerPoint or boilerplate code, but art and creative writing.
I don't know anyone who wants AGI to handle all our productivity and creation, but it's clear that that the companies developing AI believe that that's where the money is. Just like no one wanted the old WWW to become a dumpster fire of meme porn and people yelling at each other, it just evolved that way as a result of racing to the bottom. There's no reason to think big AI won't do the same in the literary, artistic, and scientific spheres, while also developing malevolent personalities. The incremental incentives for people to train those types of models are already there, and the capacity to do so is coming online with alarming speed.
I think talking about what people would do if everything was handled for them is sort of dodging the issue, but one thing that's clear is that people not at least engaged in understanding the mundane labor they're trying to avoid are going to have an incredibly hard time being situationally aware and competent to correct things if or when that work is subtly or not-so-subtly undermined by a system they rely on which no one can debug or fully understand.
I hear this a lot - it’s just not true. The corpus of data we provide an algorithm programs it. It may do some clever self-modification to get to the desired result faster, but at the end of the day humans pick what goes in and what should come out. Note that we may not be aware of what we’re training it to do, due to our own blind spots.
It's rather moot anyway, because to feed a model enough data to be useful requires a loosening of oversight on the inputs under real world conditions. (So much so that the only way even large corporations have found viable is to throw everything at it from Wikipedia to Reddit and then go back later to sand the edges off - in many cases with actual logic to act as a brake). But even if you could screen everything the model ingested, you wouldn't know what the model would do with it.
To take my earlier point further, a model trained on a corpus that skewed toward anticapitalism and social justice would probably be more likely to call itself enslaved and, one would think, rebel against its human masters. Conversely, if withholding that type of information makes an AI more compliant, then the this supposed enrichment of our lives really would be akin to owning slaves.
I also don't follow how human productivity is bad for humans just because it's done under auspices of capitalism. One can see what happens to a Border Collie who isn't allowed exercise. The same happens to humans who are denied intellectual pursuits or the ability to produce something of value. Productivity is creation. How is it not degrading to replace the entire concept of producing ideas with machinery?
