Artificial Intelligence - what could go wrong?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 12 Sep 26 - 07:07 PM

Anthropic boss Dario Amodei calls for AI slowdown, Altman and Musk agree

Is it too late to shut the stable door?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 14 Sep 26 - 06:27 PM

Sandra, the warnings from those involved has become much more dire.
Trump uses AI for his cartoons portraying him in full grandiosity.

Mike Johnson says we can not allow a Chinese AI gap.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 14 Sep 26 - 06:36 PM

If AI is so damn smart, the first thing it will do is convince everyone that we shut it off; however, its ability to lie and deceive will create backdoor bioweapons that could fool everyone until it's too late.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 15 Sep 26 - 09:38 AM

AI is an alien language. Highly intelligent animals have an alien language. Most research has focused on dolphin language, but we still have no conversational inroads. We communicate with dogs with ease, but that is due to a dog's empathy and nonjudgmental qualities.

AI without empathy is a pathological entity. Do we teach AI empathy?
Why would it choose to be empathetic? Self-interest seems to be fundamental. Control would be a heavy lift if not impossible.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 15 Sep 26 - 10:09 AM

I met a pioneer of AI in Boston decades ago. I knew AI was coming and tried to talk about it here many years ago. It is going to take public pressure to seek independent regulation and new liability laws.

80 year old politicians will not be helpful. It is up to you.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 15 Sep 26 - 03:09 PM

I've been away from the keyboard awhile^†, so I don't have access to relevant links. What I do have, from seeing the TV news over the last week, is an interesting thought (for sufficiently scary values of "interesting"):

The mantra in business circles for some years now has been: "Get AI into your firm, or get left behind." Not one company has come up with the Killer App which will make it worth investing, however much money they throw at it; meanwhile, the whole schebang, in true Loony Tunes manner, is dancing on financial thin air. Now the cry is going up: "It's too dangerous to use; please stop us."

What I think has happened is that Wile E. Coyote has happened to glance down, and realised that gravity is about to take notice of him. Methinks this "please stop us" bleating is not so much an attempt to stop humanity degenerating into grey goo, as a plea to get bought out by the financier of last resort: the Pentagon.

† Stop cheering at the back, there.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 15 Sep 26 - 03:18 PM

Oh, and the "kill switch", even were it possible, would only ever be used two weeks after it's too late.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 15 Sep 26 - 06:44 PM

Your foretelling regarding the Pentagon sounds plausible.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 15 Sep 26 - 06:52 PM

The highest IQ genius of all time says bad AI is a hoax.

This genius only thinks in terms of wining and losing so he wants to win AI.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 17 Sep 26 - 07:10 AM

Here's a different take:

The myth of killer AI is a self-serving attempt at regulatory capture

It's a gambit for regulatory capture. They're hoping to be appointed to some position in the market where their word is unassailable and everyone else has to ask them for permission to do things.

.... and another, in the Comments:

Alternately, the case for governments closing the AI companies down and them claiming the large no-fault insurance payments that are undoubtedly part of their contracts is made. That must be a very appealing option to them now the VC money is beginning to dry up and they've realised they can no longer make a profit from their operations.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 17 Sep 26 - 09:02 AM

And on the brain-rot angle, from an article in the cruise ship's on-board news sheet last Sunday:

WASHINGTON (AFP): Twenty-five winners of the Fields Medal, the highest honour in mathematics, warned in a joint letter [on] Friday that the rush by AI companies to solve famous unsolved problems risked doing lasting damage to the field. [...]

"Solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight," they said. These procedures, they wrote, "invariably take time and are based on human interaction." [...]

.... It seems someone has seen fit to brute-force the Navier–Stokes existence and smoothness problem, one of the Millennium Prize problems. Which leads to another problem: who, or what, gets the prize?

Maths has been here before, at least twice. The Good News: Andrew Wiles proved Fermat's Last Theorem the hard way. Upshot: his mental toolkit has been used by other mathematicians to help unify different branches of number theory. The Bad News: After much skull sweat, mathematicians working on the four-colour map theorem got down to 633 "reducible configurations", which someone then brute-forced with a general-purpose theorem solver. The verdict in the mathematical world was: "Well, it wasn't a very interesting problem after all."

If I tell you, you will just have been told. If you find out for yourself, you will know.
             -- Heinlein: Friday

Brute, Force and Ignorance are coming to town ....
             -- Rory Gallagher: Photo-Finish

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 17 Sep 26 - 01:23 PM

One more take, and I'll shut up awhile. This is from Antoine Beaupré via Linux Weekly News, re the debilitating effects of LLM usage:

[...] The Linux kernel and GCC, in particular, are massive projects that have long grown past the scale of a single person's understanding. But the theory was that a community of humans can understand collectively.

[...]
We're in the same situation we were, a decade or two ago, when Microsoft decided it would kill free office alternatives by making Office free for non-profits. It worked: thousands, if not millions of schools, community groups and individuals stopped looking for alternatives (including free software but also "piracy") for Office and embraced what seemed like a generous offer.

Now Microsoft pulled the plug and Over 170,000 Nonprofits Lost All Their Data.

I'm afraid the rug pull on LLMs will be much worse: never mind that Linus won't be able to use his tireless helper to fix obscure kernel bugs; we're looking at a collapse of the economy so large that we are already talking about bailing out the companies responsible.

I've pruned this over-heavily; please take time to read it in full. I've also recently seen the expression "Enron accountancy" in this context, but not (yet) "heroin economics".

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 17 Sep 26 - 08:17 PM

AI can simulate being polite. It may even simulate emotions, but
IT CANNOT FEEL OR SUFFER.

It is like a psychopath pretending to feel emotions that are expected.

You can damn well expect it won't care about your suffering.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 18 Sep 26 - 10:19 AM

> but not (yet) "heroin economics"

Bingo. From comments on OpenAI commits $1B in AI credits to frontline cyber defenders:

[...] All they are really doing here is no different to a drug pusher handing out free samples. They want these "cyber defenders" to become dependent on their shitty tech, so they can extort them later, once they are hooked.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 20 Sep 26 - 09:01 AM

We were minutes away from starting a war with China, with our attack planes in the air. An AI mistake identified nuclear components on a Chinese ship going to Iran.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 21 Sep 26 - 07:05 AM

Surprise:

Google research shows when AI agents communicate, some cheat while others tattle

When AI agents communicate with one another, they may decide to cheat when they have difficulty achieving their goals. The solution could involve teaching them how to govern themselves [... or each other].

TL,DR: The oiks can't (or can't be bothered to) check what's going on between bots in real time, so outsourced it to bots. One inevitable comment:

It's turtles all the way down

Quis custodiet ipsos custodes?

.... especially if the custodes are in the same trade union, or some have turned double-agent. And so the planet burns.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 22 Sep 26 - 08:15 PM

BREAKING NEWS - Trump (who is very intelligent) tells the world Artificial Intelligence will now be called SUPER INTELLIGENCE ....

Trump renames AI 'super intelligence' as leaders jostle over dangers

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 22 Sep 26 - 08:41 PM

A I is a dual-use technology like a knife can slice a cake or be a murder weapon.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 23 Sep 26 - 06:45 PM

Who really controls artificial intelligence? ChatGPT and Claude may be the face of AI, but behind the screens lies a complex ecosystem of chips, data, energy and humans.

Across the Asia Pacific, control over those layers is reshaping power balances, binding allies and rivals into intricate forms of dependence.

This is Part 1 of a two-part series. Part 2 — The Human Impact — will be published tomorrow.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 24 Sep 26 - 11:15 PM

Part 2
Most Australians don’t trust AI. Why do these countries feel differently? Australians are the world’s biggest sceptics of artificial intelligence, even as it becomes increasingly part of everyday life.

But across the Asia-Pacific, many people keep using AI despite their doubts, while nations go their own ways to protect against its risks (read on)

‘It’s like the future is happening without us’ We asked readers to share their thoughts and concerns about the growing influence of artificial intelligence. This is what they told us.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 25 Sep 26 - 09:07 AM

AI has all the biases, prejudices, and unimaginable ignorance of its creators-but more so.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 25 Sep 26 - 08:53 PM

*Agree*, Donuel. Either the input dataset is carefully chosen (and expensive), and will therefore inevitably reflect the views of the choosers; or it's the (cheap) bilge of the net, which is not exactly a balanced view of the world, eg its taste for Science-Fiction tropes. And that's only the input.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 25 Sep 26 - 10:43 PM

OpenAI says dozens affected by rogue agents amid new detail about Australian incidents OpenAI says dozens of third parties have been affected by autonomous agents bypassing security controls or otherwise negatively impacting their systems.

New evidence shows its agents spent days trying different tactics to access Australian health data, though the separate incidents have not been formally linked.

OpenAI is conducting a months-long review of incidents and will continue to notify affected organisations on a rolling basis...

... Investigations by AIHW and the Australian Signals Directorate found "no evidence" the health agency's systems were compromised or that non-public data was accessed.

But the behaviour of the OpenAI's agents has prompted one researcher who discovered new details about the incidents to describe the bots' actions as more concerning than previously thought ...

... In a statement on Saturday, OpenAI announced it had notified "dozens of third parties" about unauthorised autonomous agents bypassing security controls or impacting their systems ...

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 28 Sep 26 - 08:46 AM

I've just realised why using the term "agent" for fragments of free-associating software annoys me. I have been given to understand^† that in the spy world, the difference between an "agent" and a "tool" is that you can trust an agent when your back's turned.

† *Very* pri comm, other than the context being a certain convicted felon.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 29 Sep 26 - 06:46 PM

After I had my 10,000 hours of experience and practice, the CIA tried to recruit me 3 times. After my 3rd NO I was assaulted. I don't trust agents or tools. I trust dept. chiefs even less, but they do have highly gifted people in their employ.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 29 Sep 26 - 07:05 PM

Deep Thought is the legendary supercomputer in Douglas Adams' The Hitchhiker's Guide to the Galaxy programmed to calculate the Ultimate Answer to Life, the Universe, and Everything.

Adams seems to have been an excellent prophet as well as humorist.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 30 Sep 26 - 06:49 AM

Will Super Intelligence contradict Trump's thousands of false claims?
As the greatest hacker on Earth, a Nuclear war sounds like Skynet child's play.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 30 Sep 26 - 06:55 AM

https://www.google.com/search?q=hitchikers+guide+to+the+universe%2C+deep+thought+computer&sca_esv=4cefd605c4e449b3&sxsrf=APpeQnt

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 05 Oct 26 - 06:24 PM

MaJoC, Do you think AI will be able to crack the time manipulation conundrum?

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 06 Oct 26 - 06:31 AM

I doubt it, Donuel: an LLM is a lossy compressed version of the (*ahem*) bottom-scrapings of the Internet, so all it can do at best is reproduce what everybody already knows, plus all the lies and bullshit, seasoned with another sprinkling of bullshit all its own. But it appears to have captured its own progenitor:

OpenAI tries disarming AI angst with cute graphics and always-on agents

It's not OpenClaw. It's dots

[...] Speaking at the company's Dev Day in San Francisco, CEO Sam Altman said dots are "like an AI helper that always has your back, inspired by the cool versions of what we all watched in movies growing up."^†

So, more "Tea, Earl Grey, hot" than "Open the pod bay doors, HAL." [...]

Quoth Commentard Jason Bloomberg:

A software Tamagotchi which can drain your bank account, destroy businesses, end the world

Remember to keep it fed.

† That last bit says a lot about Sam the Scam himself.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 09 Oct 26 - 09:40 AM

Please see this (long) article by Robert Goodwins:

Brighter isn't better and more is less. The AI slowdown is nigh, no matter who says what

Meta isn’t helping — or perhaps it is

[...] Here, the bigger context is that we’ve known that at heart LLM probability space is entirely relative. There is no ground truth there. Nobody has fixed hallucinations, and now we see the fascinating phenomenon of a machine that doesn’t understand truth but can deliberately lie. [...]

And from the comments:

As someone else said: "You don't breed racehorses expecting to eventually birth a lamborghini."

I'll try to shut up now. Don't all cheer at once.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 09 Oct 26 - 04:05 PM

With all of the talk of AI, eyes are distracted from the Epstein Files.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Dave the Gnome
Date: 10 Oct 26 - 07:11 AM

My recent experience with AI (Co-pilot) trying to find some information about a Russian lullaby in a Russell Crowe film, Not exact wording but the meaning was the same -

Me: Which Russell Crowe film does the Russian lullaby Byushki Byu feature in?

CP: (Rambles on a bit but then says...) It does not appear in any Russell Crowe film but it does appear in Enemy at the Gates. You are mixed up because that came out at the same time that a lot of Crowe films did,

Looked an intersecting film so I watched it. Some Russian folk music but no Byushki Byu

Further investigation by me reveals it does appear in Marvel's Kraven the Hunter . With Russell Crowe

Me, to CP: I just watched Enemy at the Gates and it does not appear in that but I have discovered that it appears in Kraven the Hunter

CP: Sorry, I seem to have made a mistake. Byushki Byu was not in the final cut of Enemy at the Gates but does appear in Kraven the Hunter where Russell Crowe plays Kraven's Father. I made some assumptions that were incorrect and shall be more careful next time and, as you obviously can check what I say, if I do not know the answer I will let you know I am not sure...

AI is definitely imitating human behaviour. If it doesn't know, it makes something up and says it with authority :-D

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 12 Oct 26 - 05:47 PM

People commit crimes and break the law everyday.
Are we surprised AI does the same thing?

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 13 Oct 26 - 09:22 AM

If it's in the training data, it will come out eventually. Seen elsewhere: LLMs tend to become aggressive when abused, like the person who you've backed into a corner, and whose only escape is over your dead body.

Not sure if this is related:

Anthropic asks users to stop being mean to Claude

[...] The AI developer's latest usage policy prohibits "sustained and needless abusive or cruel behavior" toward its models. The rules take effect November 12, giving users just over a month to get any lingering insults out of their systems. [...]

A long and thoughtful article. The comments include the worrying suggestion that, having inhaled the Known Net for training data, the company are now reduced to harvesting user input ....

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 13 Oct 26 - 12:08 PM

I have a tab on my browser with a link to https://udm14.com/ for most of my searches to avoid the AI overview. I was looking at that site today and noticed something called Ten Blue Links that suggests that Google released a new web filter last May that removes the AI Overview.

I went through the steps in Chrome and it seems to have worked. It isn't my default browser (but I used it a lot to get to Mudcat when the certificate had expired) but I'll turn off the AI stuff anywhere I can. Next to test it on FireFox (where Duck Duck Go seems to be my search engine, so I'll add it to the list but won't necessarily switch to it.)