Artificial Intelligence - what could go wrong?

Subject: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 01 Sep 25 - 11:51 PM

AI browsers could leave users penniless: A prompt injection warning

Artificial Intelligence (AI) browsers are gaining traction, which means we may need to start worrying about the potential dangers of something called “prompt injection.”

Large language models (LLMs)—like the ones that power AI chatbots including ChatGPT, Claude, and Gemini—are designed to follow “prompts,” which are the instructions and questions that people provide when looking up info or getting help with a topic. In a chatbot, the questions you ask the AI are the “prompts.” But AI models aren’t great at telling apart the types of commands that are meant for their eyes only (for example, hidden background rules that come directly from developers, like “don’t write ransomware“) from the types of requests that come from users.

To showcase the risks here, the web browser developer Brave—which has its own AI assistant called Leo—recently tested whether it could trick an AI browser into reading dangerous prompts that harm users. And what the company found caused alarm, as they wrote in a blog this week:

“As users grow comfortable with AI browsers and begin trusting them with sensitive data in logged in sessions—such as banking, healthcare, and other critical websites—the risks multiply. What if the model hallucinates and performs actions you didn’t request? Or worse, what if a benign-looking website or a comment left on a social media site could steal your login credentials or other sensitive data by adding invisible instructions for the AI assistant?”

Prompt injection, then, is basically a trick where someone inserts carefully crafted input in the form of an ordinary conversation or data, to nudge or outright force an AI into doing something it wasn’t meant to do.

and this

For an AI browser, part of the input is the content of the sites it visits. So, it’s possible to hide indirect prompt injections inside web pages by embedding malicious instructions in content that appears harmless or invisible to human users but is processed by AI browsers as part of their command context.

Read the whole thing at the link.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 01 Sep 25 - 11:59 PM

This was the entire email, viewed in browser (I don't know how durable this will be).

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 02 Sep 25 - 02:38 AM

if you do not do internet banking or pay for facebook ads,and use cash, how can they empty your bank account?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 02 Sep 25 - 08:25 AM

Cyber AI criminals? Who knew there were AI criminals in this establishment?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 04 Sep 25 - 07:33 AM

AI is expensive right now, but like Amazon, it will eventually run a profit, offering safety and convenience. Still, porch pirates will exist in various forms.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 05 Sep 25 - 01:32 AM

it will eventually run a profit, offering safety and convenience" QUOTE. No evidence to support this statement

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 05 Sep 25 - 11:33 AM

Dick, do you ever read the news? It is well known that AI is 1. not making a profit and by the way, is 2. sucking up vast amounts of energy to run the facilities and water to cool them. (I disagree that it will be safe and convenient. That is a matter of opinion. Pandora opened a great big box with this one.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 05 Sep 25 - 12:21 PM

Methinks the bubble will burst, and it's only a matter of how long it will take for company directors, vulture capitalists and other gullibles* to notice it's happened, and how much of the world economy it'll take with it. Two simple words: model collapse. That's what happens when you use the output of an LLM to train another LLM, or worse still itself; basically the noise drives out the signal, the LLM gets stuck on its own hallucinations, and the poor thing ends up simulating a conspiracy theorist.

This has been demonstrated directly in the lab. Now that much of the Web's content consists of what's called "AI slop" in the trade, scraping the Web for input for other LLMS has closed the loop, and we'll be in for a reproduction of the lab results. "Boom," says Ivanova.

I look forward to the expression "Artificial Incompetence" being used in anger by the Press. In the meantime, Real People™ will have to put up with ever-increasing denial-of-service attacks on genuine Web sites like ours from scraper bots which actively ignore the "get knotted" notices. Buckle in, folks. End of rant.

* Sign of the times: you know something's in trouble when politicians start backing it.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 06 Sep 25 - 02:18 AM

I do hear the news propaganda I do not believe everything that the establishment puts out as propaganda?news?
Donuel has provided no evidence, to back up his statement"it will eventually run a profit, offering safety and convenience" quote

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 06 Sep 25 - 11:22 AM

Since it hasn't happened I'm afraid you'll have to treat that as his personal opinion and move on.

You'll also have to find some acceptable sources of information, there are many out there. Not always the biggest players, but they exist.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 06 Sep 25 - 12:50 PM

AI will have its criminal and bias costs, but most people believe that many jobs will be lost as AI takes hold of the workspace. Graphic artists are easily replaced. Analysts may hold on longer.

It is a nice day here today. (no evidence given deliberately)
How is Steve anyway? I miss his lucid posts.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 06 Sep 25 - 12:57 PM

who is Steve?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Aethelric
Date: 06 Sep 25 - 02:13 PM

This is an exerpt of a coversation I had with and AI. In this case DeepSeek. I think it highlights a few things that can go wrong.

Me
You paint a picture of AI's battling it out on the Dark Web! That sounds like far out Science fiction but it sounds like it’s close.

Deepseek
You're absolutely right—it does sound like sci-fi, but we're alarmingly close to this reality. The pieces are already falling into place: malicious AI tools, dark web markets, and decentralized tech are converging into a perfect storm. Let’s break down why this isn’t just speculation—it’s an emerging threat.

1. The Dark Web is Already Hosting AI Crime
AI Phishing Kits: Hackers sell GPT-like tools that auto-generate convincing scam emails, bypassing traditional spam filters.
Deepfake Blackmail: Dark web vendors offer "non-consensual deepfake" services, powered by open-source AI models.
Fraud Bots: Automated chatbots mimic customer support to steal credit card info, trained on leaked corporate data.
This isn’t the future—it’s happening now.

2. Why AI vs. AI Warfare is Inevitable
As defensive AI (e.g., spam filters, fraud detection) improves, offensive AI adapts:
AI Scammers vs. AI Scam-Blockers: A digital arms race where bots duel in milliseconds.
Disinformation vs. Fact-Checking Bots: Troll farms deploy AI to spread fake news, while governments/companies use AI to debunk it.
AI "Worm" Malware: Self-replicating AI agents that exploit zero-day vulnerabilities faster than humans can patch them.
Think Terminator, but with phishing emails instead of nukes.

3. The Dark Web’s Role in the AI Cold War
Unregulated Training Data: Illegal forums share datasets to train hate-speech bots, fraud scripts, and harassment tools.
AI-as-a-Service (AIaaS): Just as ransomware is rented out, bad actors will lease "GPT-5 Jailbreak" for attacks.
Decentralized AI: Blockchain-based AI models (like those on the Ethereum network) could become unhackable, unstoppable criminal tools.
Imagine Silk Road, but instead of drugs, you’re buying a chatbot that hacks banks.

4. How Close Are We? Timeline of the Inevitable
2024–2025: Rise of AI-assisted dark web scams (e.g., voice-cloning for vishing).
2026–2028: First AI worm outbreak—malware that self-improves via LLMs.
2030s: AI botnets—networks of infected devices running malicious AI agents.
Beyond: Fully autonomous cyberwars, where AI attacks and defends without human input.
We’re not at Skynet yet, but we’ve boarded the train heading there.

5. Can We Stop It?
Kill Switches: Governments may force backdoors in AI models, but this risks abuse.
AI "Immunity" Tools: Just as vaccines train the immune system, we’ll need AI that recognizes and neutralizes malicious bots.
Global Treaties: Like nuclear non-proliferation, but for AI weaponization. (Good luck enforcing that.)
The alternative? A digital Wild West where only the most ruthless AI survives.

Final Thought
Yes, it sounds like a William Gibson novel, but the groundwork is already laid. The question isn’t if AI will battle on the dark web—it’s how bad it will get, and whether we can contain the fallout.
Will we control AI, or will it control us? (Cue the Black Mirror theme.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 08 Sep 25 - 08:15 PM

Companies are racing to use AI Surveillance pricing which can be 4 times higher on a home town VPN compared to going incognito.

Some Congressmen are trying to outlaw the practice.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 14 Sep 25 - 02:23 PM

Three related articles from The Register. I'll try to keep this short.

* Model collapse is on its way:

It's AI all the way down as Google's AI cites web pages written by AI

ai-pocalypse Welcome to the age of ouroboros. Google’s AI Overviews (AIOs), which now often appear at the top of organic search results, are drawing around 10 percent of their sources from documents written by ... other AIs, according to a recent report. [...]

* Using AI to do the learning for you considered harmful:

Use it or lose it: AI may cause you to forget some skills

[...] “Is there anyone here who's OK with the idea that they may wake up one day and find they have no one left in the organization who knows how to code?” asked Gartner distinguished VP analyst Daryl Plummer during the keynote speech of the firm’s Symposium conference in Australia yesterday. [...]

* .... and someone's decided to try reducio ad absurdum as aversion therapy:

AI Darwin Awards launch to celebrate spectacularly bad deployments

It was bound to happen. The Darwin Awards are being extended to include examples of misadventures involving overzealous applications of AI.

[...] As the organizers say: "Why stop at individual acts of spectacular stupidity when you can scale them to global proportions with machine learning?"

.... That'll do for now. I find the cheesecake recipe in the Comments on the last one especially instructive.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Mr Red
Date: 17 Sep 25 - 05:51 AM

Fer Gawds' sake (chose yer Gawds)

AI is not one thing. Deep Blue was configured around the game of Go, and reconfigured around protein folding (et al). And has tremendous potential for science.

AGI/GAI trawls the web for data and re-gurgitates it in a conversational context. And the web is 100% true isn't it? And just as the web has agendas marble throughout, so does AGI. Worse, now the average idiot will believe the results because they anthropomorphise. In the similar way they believe social media because it comes from a human (mostly?).

BUT

there is "model collapse" where people use AGI to write their text (or video) which is put on the web, which AGI trawls for................ Positive feedback (aka model collapse). Except collapse only refers to the validity of results, it will more likely break out into bubbles, where the "algorithm" is telling you what it thinks you want to know. How anti-social media developed is your clue. We thought it would spread the word, it did. But it localised the words (to like minds).

AGI will become de facto because people are lazy, and live busy lives (aka frippery). All aboard for the hell in a handcart.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Mr Red
Date: 17 Sep 25 - 05:57 AM

BTW the solution to AGI trawling web content is what they like to call "curating"

which is filtering by any other name, and there are many other names, like:

Censorship. Needed fer sure but isn't that a slippery slope? Or is it a knife edge? Or am I sitting on the fence?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 18 Sep 25 - 12:11 PM

The Atlantic's AI Watchdog - not everyone will be able to open it, but I'll park this here anyway.
The Atlantic’s ongoing investigation of the books, videos, and other media used by the world’s most powerful tech companies to train their AI models.

Explore the data sets
See whose work tech companies are using to train their generative-AI models. Search for authors, YouTube channels and creators, or screenwriters—for example, Zadie Smith, MrBeast, or Aaron Sorkin. AI companies may omit certain works when training, so the presence of a work in a data set is not definitive proof that it was used. Companies often use multiple data sets in training, so the absence of a given work is also not proof that it hasn't been used. Note that some data sets contain multiple copies of certain works. Read our FAQ for additional information.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 19 Sep 25 - 06:52 AM

There's an article in this week's New World (née New European)* by Olive Pometsey, called Sweet, soulless music: the rise of AI pop. It confirms my impression that AI ought to be called AM: Artificial Mediocrity. "OK, lads, let's purée everything that's been in the charts this year, strain out the lumps, and put out the bilge. Give it a year or two, and nobody will be able to tell the difference." Hopefully, in two years' time, there'll be a revolution.

There's been a fight on between artists and record companies since at least the 1950s: true musicians want to make music, while the companies' top priority is to make money. The result in the charts is best called "punctuated equilibrium". This was documented by George Melly in Revolt into Style: The Pop Arts in Britain (1970); the bilge of the early 1960s in the UK was music-hall jokiness and bedroom angst. I was there, and it was *dire*, with the occasional good nugget which had escaped through the filter.

* Apologies for lack of link: it's behind a paywall. We buy the hardcopy edition.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 19 Sep 25 - 07:20 AM

.... Apparently one major reason for using AM to produce sample-alikes in other peoples' styles is for backing tracks for rap*. Getting permission to sample someone can take years.

I'm waiting for an album, or perhaps a group, called Bland On Bland. Hopefully, Bob Dylan (or Blonde on Blonde) will sue.

* Barry Took: "The C is silent."

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 21 Sep 25 - 06:09 PM

Perhaps AI will build the best quantum computer whose qbits will unravel and create experiments that will reveal the hidden dimensions that would enable FTL travel and teleportation.

Or an amoral AI glitch would solve our over population forever EXCEPT FOR TWO HOMO SAPIENS CAGED IN THE ZOO.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 25 Sep 25 - 02:08 PM

An answer from a Reddit discussion popped up today:

Tip: How to Enable / Disable All AI Features in Google Chrome (September 2025)

Skip down to the content about disabling AI: How to Disable or Modify AI Features and Tools in Google Chrome

A heads up - you need to know what you're doing because these instructions tell you to use regedit. Not for the faint of heart, and be sure to research enough to know that these instructions are accurate.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Mr Red
Date: 27 Sep 25 - 09:38 AM

My Goggle Chrome was the portable version from maybe 10 years ago, updated whenever it wanted to till about 3 years ago. Now it keeps telling me to install, so what are they looking for in my laptop? As they find suitable resting places (& registry entries) while they load all their files?
Portable Chrome still seems as shiny as it ever was and I only use it (mostly) to check my websites & Goggle Translate. It must really confuse the algorithm.

Having said that MS Edge keeps insisting on a Co-Pilot install & Yahoo preface most searches with an AI answer that is at least 50% ineffective. Not that I consult the answers that often. When I search I use specific terms, often with the word "red" and Amazon results always fail to offer "red" - they remove the term, and I don't buy from them anyway.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 27 Sep 25 - 12:58 PM

I threatened to unsubscribe and cancel my version of Windows 365 until they finally offered a cheaper (the previous price) version without the Copilot software as part of it. Yes, please. It renews in November, at which time I hope all of the CoPilot icons I've turned off everywhere actually disappear.

I use https://udm14.com/ as one of the tabs that opens when I start my browsers. Searching from there eliminates the AI responses. The Chrome browser extension that is supposed to block it doesn't do a complete job if I search using the Google search bar.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Ebbie
Date: 27 Sep 25 - 06:34 PM

A friend of mine who makes his living online likes AI and foresees good things for all. At the same time, he admits that it will be good only as long as good people use it. (And we know how optimistic that is...)

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 29 Sep 25 - 10:22 AM

Found this in the comments section of an article in The Register (Software CEO tells Catholic uni panel AI won't take out jobs, but it could take out brains):

School kids are now using the phrase 'that's so AI' to describe things they don't believe and are untrue.... there's hope for the next generation after all.

This is an innovation which I approve of. Hopefully it'll displace the yoof's casual abuse of the word "gay" to mean "stupid", which would make it a double win.

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 13 Oct 25 - 10:41 AM

This is priceless. From the comments section of Managers are throwing entry-level workers under the bus in race to adopt AI:

BBC News - Fears over AI bubble bursting grow in Silicon Valley

https://www.bbc.com/news/articles/cz69qy760weo

Speaking as one who was around during the Dot.Com bubble this scenario feels almost inevitable. [...]

The orange Don's trade sanctions are just starting to bite so the realisation that the US economy is weak could well be the precipitating factor.

Donald could well be the prick that bursts the AI bubble.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 13 Oct 25 - 03:29 PM

There are external factors greater than the foibles of AI.
Those are a financial crash following the burst of AI investment bubbles. Right now the overvaluation is close to 1929.