Artificial Intelligence - what could go wrong?

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 18 Jan 26 - 06:11 AM

Bingo: the RISKS Digest site at Newcastle is back up to date again. Herewith two consecutive articles from RISKS 34:83:

Capability Maturity Models and generative artificial intelligence

(see above)

The AI boom is based on a fundamental mistake (The Verge)

Large language mistake

Cutting-edge research shows language is not the same as intelligence. The entire AI bubble is built on ignoring it.

[ ... ]

Last year, three scientists published a commentary
https://archive.ph/o/BV16q/https://gwern.net/doc/psychology/linguistics/2024-fedorenko.pdf
in the journal Nature titled, with admirable clarity, “Language is
primarily a tool for communication rather than thought.” [ ... ]

Subject: RE: Artificial Intelligence - what could go wrong?
From: Sandra in Sydney
Date: 22 Jan 26 - 02:06 AM

well, you do get pretty pictures!!!

Tour website's AI sends visitors to Tasmanian sites that do not exist An AI-generated article on a travel booking website has sent tourists to a remote location in Tasmania's north-east, looking for hot springs that do not exist.

Australian Tours and Cruises has admitted the AI technology it uses to create content and articles to help drive bookings has "completely messed up".
What's next?

The company has said it will review all of its AI-generated content, which is produced by a third party ...

Mr Hennessy said that while all posts were normally reviewed before being posted, some had been made public by mistake while he was out of the country ...

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 22 Jan 26 - 06:05 AM

What's next?

people might work.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 23 Jan 26 - 10:03 AM

> well, you do get pretty pictures!!!

As it happens, Sandra, I saw an article some years ago about automated uglification using machine learning: teach the ML on pictures from one or more of the horror mags, then get them to "enhance" real-life photos to re-render them in that style. The most horrifying bit was that it was easy to tell that the image of a female English politician (Theresa May?) was processed this way, but that I found it difficult to tell that the one of Agent Orange wasn't the original.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 30 Jan 26 - 12:44 PM

This interesting essay by James O'Sullivan was shared by an academic friend who writes about AI in the classroom.

I am so f**king sick of AI slop

Extracted from it:

What is alarming, however, is the sheer willingness of people to put their own names to any old slop the machine produces.

Take the rise of ‘vibe coding’, for example, which most tech bros represent as some kind of futuristic workflow. Am I the only one who thinks that there is a peculiar arrogance in a ‘developer’ who takes pride in their inability to write or understand the syntax that underpins their own creation? We are seeing software deployed into the wild that is little more than a patchwork of pasted suggestions, a digital Frankenstein’s monster stitched together by someone who has no concept of the structural integrity of the code base. They might feel as though they know what they have built, but they can’t comprehend the gaping security vulnerabilities they have introduced because they never actually engaged with the logic of the system; they simply prompted a black box until the error messages stopped appearing, creating a technological infrastructure that is terrifyingly insecure.

And further down:

The internet was, in its most idealistic (and yes, maybe naive) conception, a sprawling parlour for human conversation and the exchange of genuine thought. That vision is effectively dead. Open LinkedIn or Reddit (or X, if you really want to wind yourself up) and you will see streams of the same beige, hallucinatory text bearing the chirpy, predictive cadence of ChatGPT, generated by users who could not be bothered to read the content they are putting their name to. They enter a prompt and paste the result, engaging in a pantomime of interaction that benefits no one but the platform’s engagement metrics. It is a hall of mirrors where machines talk to machines while humans look on, increasingly alienated from the very networks built to connect them.

He concludes "I can appreciate the technology for what it is, but I am finding it increasingly difficult to forgive the laziness of the people using it."

Amen.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 30 Jan 26 - 05:39 PM

The slop will still have better spelling than the flesh-and-blood moron activists.

A minor point, but I just saw a Trump commercial on CNN that uses AI copy of Trump's voice. I'm surprised the fine print informed people of this.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 01 Feb 26 - 05:17 AM

This interesting essay by James O'Sullivan was shared by an academic friend who writes about AI in the classroom." quote
I disagree, i find the article opinionated and subjective. but we are all entitled to different opinions.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 01 Feb 26 - 12:47 PM

Artificial intelligence researchers hit by flood of ‘slop’
Conferences restrict use of LLMs after surge of low-quality AI-generated papers and reviews

Artificial intelligence researchers are grappling with a problem core to their field: how to stop so-called “AI slop” from damaging confidence in the industry’s scientific work. AI conferences have rushed to restrict the use of large language models for writing and reviewing papers in recent months after being flooded with a wave of poor AI-written content.

Scientists have warned that the surge of low-quality AI-generated material risks eroding trust and the integrity of the sector’s research by introducing false claims and made-up content.

“There is a little bit of irony to the fact that there’s so much enthusiasm for AI shaping other fields when, in reality, our field has gone through this chaotic experience because of the widespread use of AI,” said Inioluwa Deborah Raji, an AI researcher at the University of California, Berkeley. . . . In January AI detection start-up GPTZero published research that found there were over 100 AI-generated errors across 50 papers last year at the Neural Information Processing Systems (NeurIPS) conference, considered the most prestigious place to publish cutting-edge AI research.

Further down the article is "A tell-tale sign is when papers contain hallucinated references in the bibliography, or figures that are wrong, said Dietterich. These users are then banned from submitting papers to arXiv for a while, he added."

 

Subject: RE: Artificial Intelligence - what could go wrong?
From: Bill D
Date: 02 Feb 26 - 09:44 AM

Long but fascinating conversation with founder of AI

Subject: RE: Artificial Intelligence - what could go wrong?
From: Bill D
Date: 02 Feb 26 - 09:46 AM

That conversation is with Jon Stewart being serious.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 06 Feb 26 - 07:13 AM

More straws in the wind. Just seen on BBC Red Button:

Amazon shares fall as it joins Big Tech AI spending spree

Amazon has announced plans to invest $200bn in AI and infrastructure [...]

The stock market's reaction: Amazon's shares drop over 11% in after-hours trading. Even the Wall Street gamblers are noticing the distinct lack of the Killer App we've all been promised, let alone a return on investment.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 06 Feb 26 - 08:31 AM

Related comments on an Elreg article about Raspberry Pi price rises (tyops in original):

as I read elsewhere recently: The reason why RAM has become four times more expensive is that a huge amount of RAM that has not yet been produced was purchased with non-existent money to be installed in GPUs that also have not yet been produced, in order to place them in data centers that have not yet been built, powered by infrastructure that may never appear, to satisfy demand that does not actually exist and to obtain profit that is mathematically impossible.

.... which drew the response:

Lots of "statements of intent" but very few actual contracts. It's almost like someone is bumping stock prices up for some very strange unfathomable reason that certainly has nothing to do with with some soft of pump and dump before the inevitable "Where profit?" crash.

[...] At least with the dot com bubble you could see a genuine use case for the internet. Where would your average person need AI?

.... I'll leave it there awhile.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 12 Feb 26 - 03:36 AM

Ai, is no different from encyclopedias,
Encyclopedias are only as good as whoever has written them, and that includes wikipedia

Subject: RE: Artificial Intelligence - what could go wrong?
From: Dave the Gnome
Date: 12 Feb 26 - 07:11 AM

You are wrong, Dick. The whole point of AI is that it will go beyond its initial programming by learning from the responses to the answers it gives. Encyclopedias cannot rewrite themselves and Wiki is constantly updated by the general public.

As to being anti-AI, I am far from a luddite and spent most of my working life on huge highly available computor installations - Both maintaining and designing them. As I retired I was working on AI and cloud services to replace the monolithic computor room and it makes a lot of sense to distrubute computing, particularly where you need high availability. AI is a good thing where it helps to keep companies running and the economy flowing. It is also good for coding in machine language and performing many tasks that take us humans far longer and are intrinsically boring!

However, the way many are using AI - to find answers to questions that a standard search can provide or to replace their own language - is both lazy and wasteful of resourses.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 12 Feb 26 - 07:29 AM

However, the way many are using AI - to find answers to questions that a standard search can provide or to replace their own language - is both lazy and wasteful of resourses.quote ,
all a matter of opinion
Resources not resourses
my opinion is this
Apart from the obviousness that we could simply pull the plug from any machine which didn’t behave, machines have no desires!!!

They don’t give a hoot about anything, be it sex or world domination or anything else.

They're friggin lawn mowers. Nothing else. Cogs and gears. Mechanics. That’s it.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Dave the Gnome
Date: 12 Feb 26 - 07:37 AM

Not a matter of opinion at all. The fact is that AI is resource hungry and using it for trivial purposes an easy way of using a sledgehammer to crack a nut. Look up Google's own research into the energy that Gemini uses and compare it to the energy that a standard search engine uses.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 12 Feb 26 - 07:43 AM

resource hungry compared to using a car or an aeroplane?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 12 Feb 26 - 10:28 AM

Yes. And Dick, you're done with this one for now.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 12 Feb 26 - 11:58 AM

All week the discussions on KERA's Think, a local NPR station's midday radio show, have featured AI topics. Most of them are based on books or articles you can find to learn more.

When will A.I. want to kill us? Feb. 9, 2026 - is Nate Soares discussing what happens when brain power surpasses what humans are capable of.

How A.I. is getting in the way of real learning Feb. 10, 2026 - visits with Clay Shirky, vice provost for A.I. and technology in education at NYU. How are professors using A.I. in the classroom and whether or not the technology gets in the way of critical thinking.

A.I. is writing obits now Feb. 11, 2026 - talks with technology reporter Drew Harwell (ironically, with the Washington Post that just paid all of it's obit staff to retire) to discuss the rise of obit-writing A.I. program and how funeral homes are embracing it.

Would you go to an A.I. doctor? Feb. 12, 2026 - Dhruv Khullar is a physician and contributing writer at The New Yorker discussing whether doctors will lose the skills to properly diagnose, etc. (As I write this that show hasn't broadcast yet, but it seems to be available online. )