Artificial Intelligence - what could go wrong?

Subject: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 01 Sep 25 - 11:51 PM

AI browsers could leave users penniless: A prompt injection warning

Artificial Intelligence (AI) browsers are gaining traction, which means we may need to start worrying about the potential dangers of something called “prompt injection.”

Large language models (LLMs)—like the ones that power AI chatbots including ChatGPT, Claude, and Gemini—are designed to follow “prompts,” which are the instructions and questions that people provide when looking up info or getting help with a topic. In a chatbot, the questions you ask the AI are the “prompts.” But AI models aren’t great at telling apart the types of commands that are meant for their eyes only (for example, hidden background rules that come directly from developers, like “don’t write ransomware“) from the types of requests that come from users.

To showcase the risks here, the web browser developer Brave—which has its own AI assistant called Leo—recently tested whether it could trick an AI browser into reading dangerous prompts that harm users. And what the company found caused alarm, as they wrote in a blog this week:

“As users grow comfortable with AI browsers and begin trusting them with sensitive data in logged in sessions—such as banking, healthcare, and other critical websites—the risks multiply. What if the model hallucinates and performs actions you didn’t request? Or worse, what if a benign-looking website or a comment left on a social media site could steal your login credentials or other sensitive data by adding invisible instructions for the AI assistant?”

Prompt injection, then, is basically a trick where someone inserts carefully crafted input in the form of an ordinary conversation or data, to nudge or outright force an AI into doing something it wasn’t meant to do.

and this

For an AI browser, part of the input is the content of the sites it visits. So, it’s possible to hide indirect prompt injections inside web pages by embedding malicious instructions in content that appears harmless or invisible to human users but is processed by AI browsers as part of their command context.

Read the whole thing at the link.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 01 Sep 25 - 11:59 PM

This was the entire email, viewed in browser (I don't know how durable this will be).

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 02 Sep 25 - 02:38 AM

if you do not do internet banking or pay for facebook ads,and use cash, how can they empty your bank account?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 02 Sep 25 - 08:25 AM

Cyber AI criminals? Who knew there were AI criminals in this establishment?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 04 Sep 25 - 07:33 AM

AI is expensive right now, but like Amazon, it will eventually run a profit, offering safety and convenience. Still, porch pirates will exist in various forms.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 05 Sep 25 - 01:32 AM

it will eventually run a profit, offering safety and convenience" QUOTE. No evidence to support this statement

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 05 Sep 25 - 11:33 AM

Dick, do you ever read the news? It is well known that AI is 1. not making a profit and by the way, is 2. sucking up vast amounts of energy to run the facilities and water to cool them. (I disagree that it will be safe and convenient. That is a matter of opinion. Pandora opened a great big box with this one.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: MaJoC the Filk
Date: 05 Sep 25 - 12:21 PM

Methinks the bubble will burst, and it's only a matter of how long it will take for company directors, vulture capitalists and other gullibles* to notice it's happened, and how much of the world economy it'll take with it. Two simple words: model collapse. That's what happens when you use the output of an LLM to train another LLM, or worse still itself; basically the noise drives out the signal, the LLM gets stuck on its own hallucinations, and the poor thing ends up simulating a conspiracy theorist.

This has been demonstrated directly in the lab. Now that much of the Web's content consists of what's called "AI slop" in the trade, scraping the Web for input for other LLMS has closed the loop, and we'll be in for a reproduction of the lab results. "Boom," says Ivanova.

I look forward to the expression "Artificial Incompetence" being used in anger by the Press. In the meantime, Real People™ will have to put up with ever-increasing denial-of-service attacks on genuine Web sites like ours from scraper bots which actively ignore the "get knotted" notices. Buckle in, folks. End of rant.

* Sign of the times: you know something's in trouble when politicians start backing it.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 06 Sep 25 - 02:18 AM

I do hear the news propaganda I do not believe everything that the establishment puts out as propaganda?news?
Donuel has provided no evidence, to back up his statement"it will eventually run a profit, offering safety and convenience" quote

Subject: RE: Artificial Intelligence - what could go wrong?
From: Stilly River Sage
Date: 06 Sep 25 - 11:22 AM

Since it hasn't happened I'm afraid you'll have to treat that as his personal opinion and move on.

You'll also have to find some acceptable sources of information, there are many out there. Not always the biggest players, but they exist.

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 06 Sep 25 - 12:50 PM

AI will have its criminal and bias costs, but most people believe that many jobs will be lost as AI takes hold of the workspace. Graphic artists are easily replaced. Analysts may hold on longer.

It is a nice day here today. (no evidence given deliberately)
How is Steve anyway? I miss his lucid posts.

Subject: RE: Artificial Intelligence - what could go wrong?
From: The Sandman
Date: 06 Sep 25 - 12:57 PM

who is Steve?

Subject: RE: Artificial Intelligence - what could go wrong?
From: Aethelric
Date: 06 Sep 25 - 02:13 PM

This is an exerpt of a coversation I had with and AI. In this case DeepSeek. I think it highlights a few things that can go wrong.

Me
You paint a picture of AI's battling it out on the Dark Web! That sounds like far out Science fiction but it sounds like it’s close.

Deepseek
You're absolutely right—it does sound like sci-fi, but we're alarmingly close to this reality. The pieces are already falling into place: malicious AI tools, dark web markets, and decentralized tech are converging into a perfect storm. Let’s break down why this isn’t just speculation—it’s an emerging threat.

1. The Dark Web is Already Hosting AI Crime
AI Phishing Kits: Hackers sell GPT-like tools that auto-generate convincing scam emails, bypassing traditional spam filters.
Deepfake Blackmail: Dark web vendors offer "non-consensual deepfake" services, powered by open-source AI models.
Fraud Bots: Automated chatbots mimic customer support to steal credit card info, trained on leaked corporate data.
This isn’t the future—it’s happening now.

2. Why AI vs. AI Warfare is Inevitable
As defensive AI (e.g., spam filters, fraud detection) improves, offensive AI adapts:
AI Scammers vs. AI Scam-Blockers: A digital arms race where bots duel in milliseconds.
Disinformation vs. Fact-Checking Bots: Troll farms deploy AI to spread fake news, while governments/companies use AI to debunk it.
AI "Worm" Malware: Self-replicating AI agents that exploit zero-day vulnerabilities faster than humans can patch them.
Think Terminator, but with phishing emails instead of nukes.

3. The Dark Web’s Role in the AI Cold War
Unregulated Training Data: Illegal forums share datasets to train hate-speech bots, fraud scripts, and harassment tools.
AI-as-a-Service (AIaaS): Just as ransomware is rented out, bad actors will lease "GPT-5 Jailbreak" for attacks.
Decentralized AI: Blockchain-based AI models (like those on the Ethereum network) could become unhackable, unstoppable criminal tools.
Imagine Silk Road, but instead of drugs, you’re buying a chatbot that hacks banks.

4. How Close Are We? Timeline of the Inevitable
2024–2025: Rise of AI-assisted dark web scams (e.g., voice-cloning for vishing).
2026–2028: First AI worm outbreak—malware that self-improves via LLMs.
2030s: AI botnets—networks of infected devices running malicious AI agents.
Beyond: Fully autonomous cyberwars, where AI attacks and defends without human input.
We’re not at Skynet yet, but we’ve boarded the train heading there.

5. Can We Stop It?
Kill Switches: Governments may force backdoors in AI models, but this risks abuse.
AI "Immunity" Tools: Just as vaccines train the immune system, we’ll need AI that recognizes and neutralizes malicious bots.
Global Treaties: Like nuclear non-proliferation, but for AI weaponization. (Good luck enforcing that.)
The alternative? A digital Wild West where only the most ruthless AI survives.

Final Thought
Yes, it sounds like a William Gibson novel, but the groundwork is already laid. The question isn’t if AI will battle on the dark web—it’s how bad it will get, and whether we can contain the fallout.
Will we control AI, or will it control us? (Cue the Black Mirror theme.)

Subject: RE: Artificial Intelligence - what could go wrong?
From: Donuel
Date: 08 Sep 25 - 08:15 PM

Companies are racing to use AI Surveillance pricing which can be 4 times higher on a home town VPN compared to going incognito.

Some Congressmen are trying to outlaw the practice.