Grey Goo and You – Capitalism Wants to Eat Your Grandma

Disney's Big Bad Wolf: 1933
Big Bad Wolf. (2023, September 15). In Wikipedia. https://en.wikipedia.org/wiki/Big_Bad_Wolf

The plan for October was to do a lot of reading, thinking, and writing. That’s didn’t quite work out. The net result was a bunch of fragments that I’m now trying to build out and publish. This is the second this week. My all-time publishing record:-)

Before we delve into the details of highly speculative doom scenarios, here’s a young adult sf novel about AI and cat videos that will help restore your faith in machine intelligence:-) Catfishing on the Catnet.

Blood Music

Joe Biden’s administration has just announced regulations for Generative Artificial Intelligence. There’s a concern that the widespread use of Large Language Models heralds a new era where general artificial intelligence becomes an “existential threat to humanity.”

Scary artificial intelligence

For this to be true, you would need 1) advanced ‘artificial intelligence,’ whatever that is exactly, but presumably machine intelligence smarter than we are. Also, there would need to be 2) a programmed motivation, i.e., a goal and, more specifically, a goal like achieving total global domination, and 3) agency…some equivalent of fingers and toes.

The fears typically center around number 2) above. Specifically, the fear is AI motivation gone off the rails in a particular process called instrumental convergence.

The classic example is this paper clip maximizer thought experiment (Nick Bostrom, 2003): a system charged with making paper clips ends up converting all matter on earth into paper clips…not because it had anything against the biosphere but because converting it was instrumental to the goal of paper clips.

Greg Bear’s Blood Music (1985) provides a parallel example. Here, a rouge researcher injects a simple biocomputer into his bloodstream to smuggle it out of the lab rather than destroy it, and ultimately, he becomes an infectious hive mind. The short story version ended with the biosphere converted into a superorganism (see the jacket cover below.) The novel had a happier ending with all of us people re-instantiated as individuals but with a variety of defects fixed. Thanks AI!

The nanotechnology version of this is called ‘grey goo’…the term from K. Eric Drexler’s 1986 nanotech rah-rah book, Engines of Creation. Here, everything gets converted into nano substrate…grey goo…thanks to an insufficiently regulated process.

I like that term the best for instrumental convergence bad juju. (I’ve argued elsewhere that the goal of a lot of political dinformation is to turn all facts into noise–the informational equivalent of ‘grey goo.’

Side note: there’s a parallel speculative thread to all this I find scarier. I’ll add an addendum about that below.

But first, a more immediate type of runaway instrumentality: American hyper-capitalism.

Finance is Coming for Your Grandmother

Again, science fiction leads the way:-). In an essay in BuzzFeed, author Ted Chaing lays it out very concisely. I’m going to quote him at some length. Emphasis mine.

Ted Chaing in BuzzFeed – Dec 2018 – Silicon Valley Is Turning Into Its Own Worst Fear

Speaking to Maureen Dowd for a Vanity Fair article published in April, Musk gave an example of an artificial intelligence that’s given the task of picking strawberries. It seems harmless enough, but as the AI redesigns itself to be more effective, it might decide that the best way to maximize its output would be to destroy civilization and convert the entire surface of the Earth into strawberry fields…

This scenario sounds absurd to most people, yet there are a surprising number of technologists who think it illustrates a real danger. Why? Perhaps it’s because they’re already accustomed to entities that operate this way: Silicon Valley tech companies.

Consider: Who pursues their goals with monomaniacal focus, oblivious to the possibility of negative consequences? Who adopts a scorched-earth approach to increasing market share? This hypothetical strawberry-picking AI does what every tech startup wishes it could do — grows at an exponential rate and destroys its competitors until it’s achieved an absolute monopoly. The idea of superintelligence is such a poorly defined notion that one could envision it taking almost any form with equal justification: a benevolent genie that solves all the world’s problems, or a mathematician that spends all its time proving theorems so abstract that humans can’t even understand them. But when Silicon Valley tries to imagine superintelligence, what it comes up with is no-holds-barred capitalism.

Chiang’s insight is accurate, and the term grey goo comes in handy: capitalism, or more specifically finance, is well on its way towards reducing most enterprises to a devalorized economic grey goo. People and their little concerns become instrumental to the grand task of consolidating wealth.

image from Adbusters – now and their targeted future

Within capitalism, private equity and investment banking firms are the most voracious in chewing value into grey goo. They operate by purchasing an enterprise and then extracting value for themselves by reducing its value to employees, clients, customers, and often the environment.

A typical process is the one that I’ve seen from the inside.

First, secure debt to purchase a business (often in the form of creating a loan from yourself to yourself); second, transfer the debt to the business; third, pay yourself interest and hefty consulting fees for managing the process; and fourth, discard the desiccated husk. Sears is the poster child for this.

A 20,000-foot view: if you’ve made a billion from a declining business, that money must have been extracted at the cost of something else. As surprising as the thought may be in the current climate, money has to come from somewhere.

Here’s a deeper dive from last week’s Atlantic: The Secretive Industry Devouring the U.S. Economy. From 4% of the economy in 2000 to 20+% today, grey goo is spreading.

But rather than continuing in general terms, let’s focus on Grandma.

Health and Welfare

Well, what does Grandma need?

  1. Often a Doctor – Who Employs Your Doctor? Increasingly, a Private Equity Firm
  2. Frequently a Nursing Home – How Patients Fare When Private Equity Funds Acquire Nursing Homes
  3. And eventually, probably, a Funeral Parlor – Death Is Anything but a Dying Business as Private Equity Cashes In

(The last reference is from an investigative series, Patients for Profit: How Private Equity Hijacked Health Care. Also good and a good read is the inimitable Cory Doctorow’s Private Equity finally delivered Sarah Palin’s death panels.) 

Axis of Evil

Heartlessness as a service

What do I find scary? The merger of AI and hyper-capitalism!

In a recent episode of This Week In Google (my sole remaining tech podcast), Leo Laporte and crew report on what they term ‘heartlessness as a service.’

Their source is ProPublica:

On a summer day last year, a group of real estate tech executives gathered at a conference hall in Nashville to boast about one of their company’s signature products: software that uses a mysterious algorithm to help landlords push the highest possible rents on tenants.

“Never before have we seen these numbers,” said Jay Parsons, a vice president of RealPage, as conventiongoers wandered by. Apartment rents had recently shot up by as much as 14.5%, he said in a video touting the company’s services. Turning to his colleague, Parsons asked: What role had the software played?

“I think it’s driving it, quite honestly,” answered Andrew Bowen, another RealPage executive. “As a property manager, very few of us would be willing to actually raise rents double digits within a single month by doing it manually.”

Rent Going Up? One Company’s Algorithm Could Be Why (Oct 2022).

Back to Grandma

And back to Cory Doctorow: America’s largest hospital chain has an algorithmic death panel: HCA’s administrators berate doctors over “missed hospice opportunities.”

I consider hyper-capitalism to be a machine that tends, in an interative process, ejects humans that let ethics get in the way of profits in favor of humans with fewer scruples–ultimately leaving only the machine.

AI will sort through options to optimize whatever it’s told to optimize.

The combined result in a particularly vicious combination: a mindless instrumentality that will chew through the economy remorselessly, generating grey goo from actual human value. (Kinda like Elon Musk’s brain.)

Addendum: good and bad singularities

Okay, Elon’s not scary enough:-)?

Warning: I’m going to nerd out a bit. Why? Because I can’t help myself.

The grandmother of all this ‘generative ai’ talk is The Singularity, defined as the point machine intelligence uplifts to sentience–but with expanded networked intelligence that dwarfs our own ‘as humans are compared to flatworms’ as the expression goes.

There’s a fascinating LongNow lecture and discussion, What If the Singularity Does NOT Happen? featuring Vernor Vinge.

Vinge is the Sf author who coined the term Singularity. John von Neumann discussed the event in the ’50s, but Vinge’s 1983 short story ‘True Names’ put the term into use.

The discussion immediately jumped track since none of the panelists could imagine that The Singularity wouldn’t happen. So, they started talking about possible positive or negative singularities and the range between them.

Their benchmark for a bad singularity?

Two opposing hyper-paranoid military computers in China and the US ‘uplifting’ each other to sentience during a disastrous global war that lasts only a couple of hours.

Thanks for reading.

Privacy is Doomed

[This article is, also, available on Medium. If you read there, I would appreciate some Claps. Thanks.]

Here’s a story about the future

– Technology is increasingly empowering the individual.
– 30 or so years out, some Columbine Killers wannabes will be able to use a virus or dirty bomb.
– The only real solution is the surveillance state. 
– There won’t be good individual counter-measures: trying to block surveillance will only make you stand out.
– One path to that solution is panic and partial collapse of our democratic standards similar to the dynamic of post-9/11 legislation.
– It would be nice to do better than that.

Continue reading Privacy is Doomed

Science Fiction as Erosion – Meditation on Gibson’s Spook Country

I recently reread William Gibson’s Spook Country. I read it first years ago shortly after publication and it has lurked the back of my memory ever since. Its characters, plot elements and moods would surface even though only vaguely connected to the matters at hand. I wanted to get back into it for another full immersion.

It was a very satisfying re-read.

Now then, I read it on the Kindle  and the Kindle has a dangerous aspect, particularly for insomniacs: you can buy a book on impulse at 2 am and be reading it minutes later. Spook Country (of which I had vivid memories) lead to Pattern Language (which I remembered hardly at all despite some overlap in characters and vibe) and then on into a mini-sf reading spree of early Gibson, newer Ian Banks, and mop up Kage Baker.

Noting what remained vivid in memory and what didn’t  lead to some thinking about what sf does.

I’ve read various theories over the past decades about the pulps and the  limits and appropriate role of genre fiction. My theory of sf matches none of that and simultaneously works as a description of why I like all the interrelated genre’s of sf, fantasy, & horror.

The role of sf is, quite simply, to erode the present.

  • Imagine that we sit down with a new volume by a favorite author on the flat plain of the present.
  • Hills and valleys have been flattened since last time by the familiarity of the everyday world.
  • We read.
  • If the volume is effective the world begins to erode.
  • In the best possible case, like a stroll atop Sheep Mountain in the South Dakota, flat prarie drops away into a deeply erode landscape of shape and color.

The gradient of how things will move forward has now been changed!

The present flows into the future along an altered channel with different resonance and open possibilities.

And finally that new future feeds back and revises the present. Our world becomes more eerie, more open, more wondrous, more strange.

Kage Baker: An Appreciation

A favorite author of mine, Kage Baker, died young at 57 on 1/31/2010.

I took the news as a call to chase down and read anything I could find of hers that I’d somehow missed. There was very little to find. I’d been following her work assiduously since reading her In the Garden of Iden in 1997. Not much had slipped by me.

I was particularly fond of her popular Company series but I’ve liked everything she’s written.
The Company series began with a premise introduced in Garden of  Iden of a secret organization using a highly constrained version of time travel to ‘guide’ human history. Its agents are recruited (by other agents) from among otherwise doomed young children who are trained up, given immortality and near invulnerability, and then deployed to rescue cultural treasures and prevent the derailment of human history. Or at least that’s the story the agents are given.
The master organization exists in the 21st – 24th Century and stays there since time travel is one way and it would doom the non-immortal company officers to life in a primitive past.
The agents slowly realize things are not necessarily as they seem, factions form, and different strategies constellate all focused on 2355…the point where something happens and the news and instructions channel from the future goes dark.
The beauty of Baker’s writing is that she focuses on the human side of her cyborg agents. She gets to play with a clash of cultural backgrounds and character formation from different epochs with agents recruited in periods ranging back to the Neolithic but deployed potentially anytime or anywhere forward…although she focus on relatively recent missions (1500 and on).  She deals with the challenge to relationships and the strains on the personalities caused by her agents inhuman condition and there alienation from the regular human population who are, nonetheless, producing the magnificent cultural artifacts they are raised to protect and treasure.
It’s a great use of a nerdly science fiction premises to anchor an exploration of personality and culture. Oh, and did I mention that a key thread through all books is the romance of the two characters introduced in Iden, one of which repetitively dies and then reappears.
If I have any criticism, it’s that her later books have too much plot to get through. She seemed to be racing forward faster and faster to cover the full narrative to the events of 2355 in fewer and few pages. The final book in the series appeared in 2007. I don’t know the events of her personal biography: perhaps this rush was related to her illness; or it could be she was simply bored with the series but wanted to give her fans a conclusion and get on with other projects. In any case, I liked it better when she took her time and did a slower dive into culture and character.  There she shone although it was all interesting.
My favorite of her novels is Sky Coyote. Imagine an competent version of Rowling’s Gildaroy Lockhart, surgically altered to appear part coyote, and sent to convince a coastal, mercantile-oriented, California tribe to do the Companies bidding and move from their rich home territory. This is the second book in the Company series. I give it a strong recommendation. If you’re a purist, read Iden as a warm up.
Creative Commons License
Except where otherwise noted, the content on this site is licensed under a Creative Commons Attribution 4.0 International License.

Useable (share, remix, etc.) but with attribution