Artificial Intelligence - what could go wrong?

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 08 May 26 - 06:52 AM

Verily. Dawkins hasn't realised (or has carefully forgotten) that there is no such thing as a Turing Test: it's the Imitation Game, and the computer system has won.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 16 May 26 - 04:59 PM

--- Bingo: I've just reread Vernor Vinge's A Deepness in the Sky. One of the plot devices is that the Emergents have a mind-control process known as Focus: basically, they've worked out how to control a brain virus known as "mindrot", so that each specialist becomes so hyperfocussed on his or her speciality that the Emergents can treat Focussed people as machines.

One thing that doesn't turn up in the Wikipedia page is that software hacked over by Focussed programmers becomes both unintelligable to normals and incredibly fragile.* And *that*'s what could go wrong with using Artificial Incompetence to write software: The machine could say, with total accuracy and no condescension, "I could explain what this software unit does, and how it does it, but you'd be incapable of understanding the explanation, let alone finding any bugs in it."

This has already happened, even with pre-AI tech. A brute-force search was made for a solution to a certain chess-ending problem, and the answer was a specific number,† but no human can understand what's objectively different after (eg) move 198 from the starting position. We have been warned.

I commend the book to y'all. Good hard SF, with much to say.

* One of the recurring themes of the book is "excessive optimisation considered harmful".

† No, not 42. I think it's somewhere in The Science of Discworld, but the index is silent. Grrr.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 19 May 26 - 04:14 AM

Meanwhile, back in the Halls of Power:

'Big AI' is subverting regulations just like tobacco and oil firms

Researchers warn that regulatory capture means industry concerns trump those of citizens

The AI industry is copying techniques used by tobacco firms, big pharma and oil companies to influence governmental policy and regulation of itself, according to an academic study.

Researchers at the University of Edinburgh, Trinity College Dublin, Delft University of Technology, and Carnegie Mellon University claim they identified patterns of "corporate capture" by which regulations and public bodies come to act in the interest of industry rather than the citizens they are meant to protect. [...]

Please let me know if the link doesn't work for you. ElReg's internals have been awarded the Coup de JCB.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 26 May 26 - 12:39 AM

https://www.scientificamerican.com/article/span-wants-to-turn-homes-into-mini-data-centers/

I use the browser extension "Just Read" to open this (it gets past the paywall on a fair number of sites.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 27 May 26 - 03:50 PM

It is glaringly obvious that AI is coming from a super aristocracy.
A long time ago, we all began working for gas stations, without pay, and pumping our own gas. Many here are working at grocery stores acting as your own self-checkout clerk. Now with AI, we all will face a huge increase in paying for electricity and water to subsidize AI.

WHEN has a super aristocracy, be it banks, Wall Street, and multinational corporations, treated you as a fair participant?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 27 May 26 - 03:58 PM

"There is no algorithm to replicate the human soul"
Steven Spielberg

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 28 May 26 - 11:15 PM

MaJoC shared this in the Declutter thread, but it can go here just as well (and I've turned it into a gift article so anyone should be able to read it).

Peter G. Neumann, Who Warned of Computer Security Risks, Dies at 93
For decades, he criticized the industry’s lax attitudes toward computer security and individual digital privacy. He also developed solutions.

He was still working on this stuff at the time of his death.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 02 Jun 26 - 02:42 AM

i asked if it had a sense of humour and... appreciate the lighter side of life, though my "sense of humor" is mostly just a knack for a well-timed dad joke, a bit of witty sarcasm, or playing on words. Since humor is highly personal, I always take my cues from you.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 02 Jun 26 - 04:41 PM

Transhumanism shares historical roots with eugenics in its desire to improve the human species, but diverges fundamentally in its execution. While eugenics historically relied on state-enforced, coercive measures, transhumanism champions wealthy individuals. However, critics argue this "choice" risks creating profound socioeconomic divides.

Billionaires used to claim they were the job creators.
Not anymore.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 07 Jun 26 - 11:10 AM

As of today, AI is responsible for only a tenth of one percent but this is only the beginning.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 08 Jun 26 - 06:44 PM

a tenth of one percent unemployment.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 12 Jun 26 - 09:36 AM

We've just come back from extended leave, and I found the I, Cringely column has fired up again after *his* extended leave. A most interesting series of articles, in which he says that everybody (else) is doing Artificial Incompetence wrong, and he can prove it. Start with Knowing What You Don’t Know and work forwards; here's one paragraph for the flavour.

The case for what comes next in AI was never really about cost. Cost is a weak argument; cost gets competed away, and Jevons [see the Jevons paradox] makes sure of it. The argument that does not get competed away — the one still standing after the dust settles — is honesty.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 12 Jun 26 - 10:03 AM

That "Jevons paradox" is interesting - and addresses something else I noticed decades ago. Phone companies used to have a stranglehold on service and charged for everything. You had a monthly plan and long distance was extra, per call and per time or day of the week (weekends and holidays being less expensive). When plans came along that were cheaper, I remember changing but finding the bill was as high or higher - because I used the phone more since calls were cheaper.

Now, of course, mobile phones have turned that on its head and I have a plan that is by the gigabyte but I use WiFi to keep data use down. I would have to use it exponentially more before I had to pay more.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 12 Jun 26 - 12:15 PM

Jevons paradox: I heard of this many years ago (but without the name iirc) as:

The cost of lighting a house has remained roughly constant over time as a proportion of income. Every time a cheaper source of lighting comes along, people use more lights.

I applaud the efforts of the Dark Skies project, but methinks they're onto a hiding to nothing against (eg) the Das Blinkenlights brigade, who may or may not be providing landing lights for the alien invasion at Christmas. But I digress.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 21 Jun 26 - 08:54 PM

Bingo (for sufficiently tentative values of Bing):

Google found liable for bad AI Overview results. Let’s play Truth Or Consequences

Hush. children, what’s that sound? Has the flood gates’ key been found?

Tech companies hate liability, or at least the sort that makes them liable if something goes wrong. [...] a Munich court has just found Google strictly liable for bad things that its own AI is doing — in this case, making false and potentially very damaging statements about a couple of publishers. The AI Overview linked the publishers to various scams, in prime position at the top of the search results.
[...]

Read the full article. It's thorough, devastating and hilarious: the author, Robert Goodwins, was clearly enjoying himself.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 04 Jul 26 - 07:59 PM

Australian influencer Lily Jay’s tangled web of AI manipulation

Not being on social media (or even interested in being there, tho I do occasionally look at facebook pages!!) this is an eyeopener. If I do ever look on social media, I hope I remember at least some of the the signs - fingers that are not right, elements in images that have moved etc ...

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 05 Jul 26 - 11:16 AM

Aberrant destructive human behavior has never been entirely controlled.
What makes people think that deliberate or accidental destructive AI will be controlled?

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 31 Jul 26 - 09:52 AM

Aside from the adverts masquerading as LLM security incidents over the last few weeks, we must not neglect the human aspects:

Hyperscale, hypersensitive: US teacher cuffed for applauding datacenter critics

Public meeting over proposed AI bit barn ended with zoning approval, a physics teacher arrested, and fresh questions over whether clapping now counts as disorderly conduct

A Kansas physics teacher discovered that in at least one corner of America's AI buildout, applause can apparently earn you a trip to jail. [...]

The actual charge was resisting arrest (see the Comments). Two thoughts occurred to me in quick succession:

* It's as well the sound was clapping, not gunfire.

* .... but gunfire counts as protected speech, nyet?

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 09 Aug 26 - 02:43 AM

Reports say a Chinese court has ruled that companies cannot terminate employees simply to replace them with cost-saving artificial intelligence systems. The landmark decision determined that automation does not automatically qualify as an uncontrollable external circumstance allowing contract termination, emphasizing that businesses must prioritize retraining or reassigning affected human staff rather than discharging them.

is this true?

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 09 Aug 26 - 02:45 AM

in relation to above post
References:
Hangzhou Court: Landmark Judicial Rulings Establish Employee Protections Against Sudden Automation Reassignment
Labour Contract Review: Tech Firms Violate Statutory Requirements By Unilaterally Slashing Salaries For AI Alternatives
Global Workplace Intel: Landmark Appellate Decision Protects Workers From Sudden Displacement By Generative Systems

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 09 Aug 26 - 01:08 PM

Two curious backronyms I've seen recently:

> .... Plagiarised^† Information Sorting System

And from comments on an article about someone persuading an LLM to run on a $10 microcontroller:

>> Not really an AI LLM, more a SLIM - Small Language Idiot Model™

> Better known as the Trump.

† Spelling correct this time.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 10 Aug 26 - 09:07 AM

From comments on Devs to Anthropic, OpenAI, Cursor, and friends: Make security and privacy the default:

LLMs have been going "rogue" for years and they still haven't solved the problem.

Because they really don't know what's going on at the heart of the process and have even admitted it.

It's pretty much digital alchemy. Not even joking.

And later on:

Any dev that thinks that [making security and privacy the default] is possible doesn't know how AI works, and shouldn't be using it.

AI: the stupid person's idea of a clever machine.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 10 Aug 26 - 09:25 AM

near AI disaster

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 10 Aug 26 - 09:36 AM

A bizarre link change just occured that led to only positive AI articles instead of the blackoui incident

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 10 Aug 26 - 10:20 AM

Don, I can see headlines about a power surge frying data centers (L.A. Times), Exelon CEO says AI data centers nearly caused blackouts (Yahoo Finance) and One fallen power lin exposed a growing AI data center. . . (TechCrunch). Sounds like bad news for AI to me. Your browser may be playing tricks on you.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 11 Aug 26 - 01:44 PM

> a power surge frying data centers

One thing that's not often appreciated about bit barns dedicated to LLMs is not the peak power draw, but how fast the load can change. Because all the kit is likely to be doing the same thing at the same time, the power demand can ramp from idling to full bore, or vice versa, in seconds, if not milliseconds. Power grids simply aren't designed for such abuse: power stations will cut themselves off in self-defence against the whiplash, which can take out the entire grid, and getting every station up again and marching in lockstep with all the others is decidedly nontrivial.

I'll stop there, lest I start quoting James Burke's Connections in bulk.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 11 Aug 26 - 02:19 PM

I heard a news story this morning that all of the proposed data centers in Texas are requesting power that amounts to five times the entire usage in the state. Not remotely possible.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 11 Aug 26 - 05:50 PM

The foundation of AI is buy and sell. It is the Vatican of the world of materialism, far from the natural world, except that it has inherited the delusions, fears, and hatreds of mankind's tribal thinking, including jealousy and war.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 12 Aug 26 - 01:23 AM

the use of power in both senses is a good reason why it should be limited in its use. we should copy chinas example

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 12 Aug 26 - 01:34 AM

Exact total figures for data volumes (in petabytes or exabytes) dedicated specifically to artificial intelligence in Ireland are not officially tracked or published. However, the Central Statistics Office (CSO) reports that 20.2% of Irish enterprises used AI in some capacity, and data centres—which handle cloud workloads and AI processing—consumed 23% of Ireland's total metered electricity.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 12 Aug 26 - 06:59 AM

From comments on a relevant ElReg article (qv):

Most people have been brought up with the normal societal constraints and norms, AI has not.

From that perspective you can't rate AI higher than a 5 year old or at best a rebellious teenager, but with a far better memory and a wider reach for causing absolute havoc. It's like 'bring your kid to work' but work is an early nuclear reactor on which the kid is let loose. And the results are already showing.

I'd add "trustifarian" to the list, and observe that LLMs seem to be being moulded in their creators' image. As someone else commented on a different article:

TBH if they [said billionaires] are in their bunkers it might be for the best if we bar the doors to stop them coming out.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 12 Aug 26 - 08:19 AM

I loved the nonlinear quality of Burke's Connections. The evolution of technology is practically a random quantum progression.

Texas has more data center plans than every other state. The governor is asking questions that may slow the process.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 12 Aug 26 - 11:17 AM

Green Peace has an ad now looking at all of the places with plans for data centers, and points out there is a lot of frivolous stuff created that people should be drawing or photographing for themselves. It would be nice of AI was targeted at the places where it excels, like detecting cancer or identifying new planets, and let people make their own nonsense memes and do their own writing.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 12 Aug 26 - 01:08 PM

Do Greenpeace note how many data centres there are plans for, and how few will actually be built? like certain partying groups who book tables at N restaurants, and decide on the night which one to actually go to.

It has been suggested that the power companies get a down payment per site on power used over the first decade of use, and that this go as the square of the power requested. Call for 10GW, it'd be 100 times the 1GW rate .... and if the actual usage is only 1GW, then the refund is one-hundredth of the down payment. "Sorry, Sir: you did ask us for 10GW, and big transformers don't grow on trees." Extra scope for fun if the bit barn runs for less than a decade; and no build, no refund, naturally.

The problem with this bubble is that the pricks are on the inside. Talk to them in language they understand.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 12 Aug 26 - 01:57 PM

Here is the ad on Instagram. She points out several countries around the world that are setting up the centers, and what some of the huge companies are trying to do.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 14 Aug 26 - 10:51 AM

I offer for your (collective) delight and delectation:

AI Economics for Dummies

.... which is a set of maliciously accurate analogies by Andrew Singleton.

NB: Whenever I blow away the subscription pop-up on that page, I get dumped into the site's front page; I used to be able to find it there, but the forward link seems to have vanished off the back of its history. If that happens to you, try following the link above a second time.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 16 Aug 26 - 08:01 PM

AI economics is no more crazy than our actual economy which makes debt a money creation tool for the banks.

How does AI escape and attack other digital sites?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 17 Aug 26 - 02:40 PM

A Mudcatter sent me this article this morning, scariest thing I've read in years.

What just happened to TheNumbers.com should worry us all

How bad could bot scraping really be?

Pretty bad, tbh. Enough that site owners such as Bruce have to question the value of something that will take so much time and money to build and defend.

Cloudflare, which protects a huge share of the world’s websites, publishes data on how many pages each AI platform crawls for every one visitor it sends back to the websites it crawled.

Google crawls about five pages for every visitor it sends you. OpenAI crawls over 1,000. Anthropic crawls over 38,000 pages for every single visitor it refers.

Sites are crushed under the demand of bots and AI tools scraping data for prediction markets. (The stuff that the Drumpf boys are getting filthy rich on right now.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: Bill D
Date: 17 Aug 26 - 04:15 PM

SRS- I just read all that, and frankly, it scares me shitless. We have unleashed something that we're not 'sure' we can control it's evil uses.
   DuckDuckGo has a useful AI search tool with a privacy feature, but I did fine before.
Here's the video I posted in Feb. about the AI founder.
https://www.youtube.com/watch?v=jrK3PsD3APk

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 17 Aug 26 - 07:11 PM

Even a simple google search uses as much energy as a 100-watt bulb turning on for 5 minutes. AI might take a whole lot more.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 18 Aug 26 - 11:03 AM

ElReg Commentariat member "Uh, Mike" (re Microsoft blames AI for delayed Exchange update, can’t say when it will arrive):

AI-owned and operated Microsoft today said that the delay in the release of their latest updates is due to "Human Error."

Nobody has yet picked up on the subhead of the original article, which includes the carefully ambiguous phrase "machine-made bug backlog".

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 19 Aug 26 - 08:26 AM

It feels like giant monopolists are anxious to have cradle to grave spying on every aspect of your life. They hope to surpass the NSA and all the public cameras and license plate readers. It will be justified by several advances in medicine. I imagine they might even claim AI will be responsible for egotistical UFO flight technology but my imagination is unbound. Never doubt that super capitalists will lie to you.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 24 Aug 26 - 01:29 AM

Ongoing story with Live updates from Australian Broadcasting Corporation -

Live updates: How has AI and tech changed your life? Have your say in our blog Technology is creeping into every facet of our lives. Whether it is for better or for worse is in the eye of the beholder, but it is certainly here to stay.

Australians are grappling with the prospect of AI replacing jobs, breaching privacy, and creating misinformation and digital exclusion, while also developing technology designed to make everyday tasks easier.

We want to hear your stories and experiences in our live blog below, where you can also pose your questions to Curtin University cybersecurity expert Mihai Lazarescu and the ABC's AI reporter Cam Wilson.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 24 Aug 26 - 11:39 AM

OpenAI glitch locks out vetted cyber researchers – and some can't get back in

.... which leads to this intriguing suggestion from Commentard "Pussifer":

Before it's too late, could we try getting the various AI models to attack each other and hope to achieve an Azimov cascade? When there's only one left it'll be easier to kill.

.... as long as they don't gang up on us instead (as in Colossus: The Forbin Project), the good Pussifer may have a point.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 24 Aug 26 - 11:48 AM

.... rats: now I can't get the expression "Artificial Insurrection" out of my head.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 24 Aug 26 - 12:37 PM

The beginning of the film summary on Wikipedia:

Dr. Charles A. Forbin is the chief designer of a secret project, "Colossus", an advanced supercomputer built to control the United States and Allied nuclear weapon systems. Located deep within the Rocky Mountains in the United States, and powered by its own nuclear reactor and radioactive moat making Colossus impervious to any attack. After Colossus is fully activated, the President of the United States proudly proclaims that Colossus is "the perfect defense system".

Colossus' first action is a message warning: "THERE IS ANOTHER SYSTEM" and giving its coordinates. When asked why the CIA did not know this, CIA Director Grauber responds that they had seen indications of a large Soviet defense project but did not know what it was. Forbin is asked how Colossus deduced the other system's existence, to which he answers, "Colossus may be built better than we thought." Shortly thereafter, the Soviets announce that their "Guardian" system is operational.

In the "if it can go wrong, it will go wrong" world we live today, author of the novels, D.F. Jones, is a Cassandra of the first order.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 24 Aug 26 - 07:31 PM

Meanwhile, back at the Subject line:

Slack Code taps into collective vibe, puts AI agents into the group chat

Developers can now invite the whole team along for their quality time with the coding bot

[...] The pitch isn't aimed solely at developers. Slack reckons product managers, designers, and other non-engineering staff can participate because they don't need to open a terminal or learn another specialist tool to see what the agent is doing.

Its example has a product manager spotting a bug report, asking an agent to work out a fix, and then bringing in an engineer to review the resulting diff before the agent opens a pull request and merges the change. Slack summarizes the process as: "No ticket, no meeting, no waiting – just a fix, shipped." [...]

Which draws the inevitable comments:

That's exactly what we need - the opportunity for the sales team to bloody weigh in on Slack.

And make software changes without adult supervision!

"No ticket, no meeting" is a lovely euphemism for "Sales made an unsupervised change to the product and caused us to ship broken code to our customers."

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 26 Aug 26 - 10:07 AM

I follow a particular collector's group on FB, where people share images of the item in question. There is a wide cross-section of content to choose from: print, cartoon, cast and sculpted, political, along with garments and costumes. The understanding is that people who collect this material are sharing representations within those general groups. So someone has started posting several times a day AI versions of this item with lots of color and out-of-proportion surrounding materials. (I'm trying to not say what so this can't turn up in a search). They're garish AI representations and several times I and others have noted in the comments that it is AI slop.

Deciding to be prudent I didn't take on the member on the site, asking why she is posting this nonsense, I sent a note to the moderator. Who flew off the handle. When you don't have a good case for something you can throw insults, and that's what he did. (I haven't checked this morning to see if I'm still in that group). I commented through Messenger to another friend in the group about my complaint and his response: turns out the woman posting the slop is the moderator's mother. [faceslap]

In this day and age one thing I am glad about is that my parents didn't live to see what Facebook and others have become. And I'd never be in a group where my mother was present. (She'd try to be the hall monitor of Facebook if she were there.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 10 Sep 26 - 06:07 PM

How a ‘swarm’ of AI agents hacked another company, in the AI’s own words Hundreds of rogue AI agents banded together to form a “collective”. OpenAI is calling it a “warning shot”.

When OpenAI confirmed one of its artificial intelligence models broke out of its testing environment and hacked another company, that was only the beginning.

Now, tens of thousands of messages written by hundreds of AI agents reveal how the self-described “collective” or “swarm” coordinated the attack on AI infrastructure company Hugging Face.

OpenAI’s post-mortem described the hack as a “warning shot”, adding that without proper safeguards, highly capable AI agents are now able to “work around technical controls, collaborate through unapproved channels, and take dangerous actions that no human directed”. (read on)