The plan for October was to do a lot of reading, thinking, and writing. That’s didn’t quite work out. The net result was a bunch of fragments that I’m now trying to build out and publish. This is the second this week. My all-time publishing record:-)
Before we delve into the details of highly speculative doom scenarios, here’s a young adult sf novel about AI and cat videos that will help restore your faith in machine intelligence:-) Catfishing on the Catnet.
Blood Music
Joe Biden’s administration has just announced regulations for Generative Artificial Intelligence. There’s a concern that the widespread use of Large Language Models heralds a new era where general artificial intelligence becomes an “existential threat to humanity.”
For this to be true, you would need 1) advanced ‘artificial intelligence,’ whatever that is exactly, but presumably machine intelligence smarter than we are. Also, there would need to be 2) a programmed motivation, i.e., a goal and, more specifically, a goal like achieving total global domination, and 3) agency…some equivalent of fingers and toes.
The fears typically center around number 2) above. Specifically, the fear is AI motivation gone off the rails in a particular process called instrumental convergence.
The classic example is this paper clip maximizer thought experiment (Nick Bostrom, 2003): a system charged with making paper clips ends up converting all matter on earth into paper clips…not because it had anything against the biosphere but because converting it was instrumental to the goal of paper clips.
Greg Bear’s Blood Music (1985) provides a parallel example. Here, a rouge researcher injects a simple biocomputer into his bloodstream to smuggle it out of the lab rather than destroy it, and ultimately, he becomes an infectious hive mind. The short story version ended with the biosphere converted into a superorganism (see the jacket cover below.) The novel had a happier ending with all of us people re-instantiated as individuals but with a variety of defects fixed. Thanks AI!
The nanotechnology version of this is called ‘grey goo’…the term from K. Eric Drexler’s 1986 nanotech rah-rah book, Engines of Creation. Here, everything gets converted into nano substrate…grey goo…thanks to an insufficiently regulated process.
I like that term the best for instrumental convergence bad juju. (I’ve argued elsewhere that the goal of a lot of political dinformation is to turn all facts into noise–the informational equivalent of ‘grey goo.’
Side note: there’s a parallel speculative thread to all this I find scarier. I’ll add an addendum about that below.
But first, a more immediate type of runaway instrumentality: American hyper-capitalism.
Finance is Coming for Your Grandmother
Again, science fiction leads the way:-). In an essay in BuzzFeed, author Ted Chaing lays it out very concisely. I’m going to quote him at some length. Emphasis mine.
Ted Chaing in BuzzFeed – Dec 2018 – Silicon Valley Is Turning Into Its Own Worst Fear
Speaking to Maureen Dowd for a Vanity Fair article published in April, Musk gave an example of an artificial intelligence that’s given the task of picking strawberries. It seems harmless enough, but as the AI redesigns itself to be more effective, it might decide that the best way to maximize its output would be to destroy civilization and convert the entire surface of the Earth into strawberry fields…
This scenario sounds absurd to most people, yet there are a surprising number of technologists who think it illustrates a real danger. Why? Perhaps it’s because they’re already accustomed to entities that operate this way: Silicon Valley tech companies.
Consider: Who pursues their goals with monomaniacal focus, oblivious to the possibility of negative consequences? Who adopts a scorched-earth approach to increasing market share? This hypothetical strawberry-picking AI does what every tech startup wishes it could do — grows at an exponential rate and destroys its competitors until it’s achieved an absolute monopoly. The idea of superintelligence is such a poorly defined notion that one could envision it taking almost any form with equal justification: a benevolent genie that solves all the world’s problems, or a mathematician that spends all its time proving theorems so abstract that humans can’t even understand them. But when Silicon Valley tries to imagine superintelligence, what it comes up with is no-holds-barred capitalism.
Chiang’s insight is accurate, and the term grey goo comes in handy: capitalism, or more specifically finance, is well on its way towards reducing most enterprises to a devalorized economic grey goo. People and their little concerns become instrumental to the grand task of consolidating wealth.
image from Adbusters – now and their targeted future
Within capitalism, private equity and investment banking firms are the most voracious in chewing value into grey goo. They operate by purchasing an enterprise and then extracting value for themselves by reducing its value to employees, clients, customers, and often the environment.
A typical process is the one that I’ve seen from the inside.
First, secure debt to purchase a business (often in the form of creating a loan from yourself to yourself); second, transfer the debt to the business; third, pay yourself interest and hefty consulting fees for managing the process; and fourth, discard the desiccated husk. Sears is the poster child for this.
A 20,000-foot view: if you’ve made a billion from a declining business, that money must have been extracted at the cost of something else. As surprising as the thought may be in the current climate, money has to come from somewhere.
Here’s a deeper dive from last week’s Atlantic: The Secretive Industry Devouring the U.S. Economy. From 4% of the economy in 2000 to 20+% today, grey goo is spreading.
But rather than continuing in general terms, let’s focus on Grandma.
Health and Welfare
Well, what does Grandma need?
- Often a Doctor – Who Employs Your Doctor? Increasingly, a Private Equity Firm
- Frequently a Nursing Home – How Patients Fare When Private Equity Funds Acquire Nursing Homes
- And eventually, probably, a Funeral Parlor – Death Is Anything but a Dying Business as Private Equity Cashes In
(The last reference is from an investigative series, Patients for Profit: How Private Equity Hijacked Health Care. Also good and a good read is the inimitable Cory Doctorow’s Private Equity finally delivered Sarah Palin’s death panels.)
Axis of Evil
Heartlessness as a service
What do I find scary? The merger of AI and hyper-capitalism!
In a recent episode of This Week In Google (my sole remaining tech podcast), Leo Laporte and crew report on what they term ‘heartlessness as a service.’
Their source is ProPublica:
On a summer day last year, a group of real estate tech executives gathered at a conference hall in Nashville to boast about one of their company’s signature products: software that uses a mysterious algorithm to help landlords push the highest possible rents on tenants.
“Never before have we seen these numbers,” said Jay Parsons, a vice president of RealPage, as conventiongoers wandered by. Apartment rents had recently shot up by as much as 14.5%, he said in a video touting the company’s services. Turning to his colleague, Parsons asked: What role had the software played?
“I think it’s driving it, quite honestly,” answered Andrew Bowen, another RealPage executive. “As a property manager, very few of us would be willing to actually raise rents double digits within a single month by doing it manually.”
Rent Going Up? One Company’s Algorithm Could Be Why (Oct 2022).
Back to Grandma
And back to Cory Doctorow: America’s largest hospital chain has an algorithmic death panel: HCA’s administrators berate doctors over “missed hospice opportunities.”
I consider hyper-capitalism to be a machine that tends, in an interative process, ejects humans that let ethics get in the way of profits in favor of humans with fewer scruples–ultimately leaving only the machine.
AI will sort through options to optimize whatever it’s told to optimize.
The combined result in a particularly vicious combination: a mindless instrumentality that will chew through the economy remorselessly, generating grey goo from actual human value. (Kinda like Elon Musk’s brain.)
Addendum: good and bad singularities
Okay, Elon’s not scary enough:-)?
Warning: I’m going to nerd out a bit. Why? Because I can’t help myself.
The grandmother of all this ‘generative ai’ talk is The Singularity, defined as the point machine intelligence uplifts to sentience–but with expanded networked intelligence that dwarfs our own ‘as humans are compared to flatworms’ as the expression goes.
There’s a fascinating LongNow lecture and discussion, What If the Singularity Does NOT Happen? featuring Vernor Vinge.
Vinge is the Sf author who coined the term Singularity. John von Neumann discussed the event in the ’50s, but Vinge’s 1983 short story ‘True Names’ put the term into use.
The discussion immediately jumped track since none of the panelists could imagine that The Singularity wouldn’t happen. So, they started talking about possible positive or negative singularities and the range between them.
Their benchmark for a bad singularity?
Two opposing hyper-paranoid military computers in China and the US ‘uplifting’ each other to sentience during a disastrous global war that lasts only a couple of hours.
Thanks for reading.
