Does anyone actually believe this "rogue AI" nonsense?

⁨228⁩ ⁨likes⁩

Submitted ⁨⁨3⁩ ⁨weeks⁩ ago⁩ by ⁨LemmySuxNow@reddthat.com⁩ to ⁨[deleted]⁩

It’s the most blatant excuse for state sponsored hacking I’ve ever seen. Surely nobody is dumb enough to buy this shit?

source

Comments

Sort:hotnew top

one_old_coder@piefed.social ⁨3⁩ ⁨weeks⁩ ago
It’s marketing, but a lot of people buy it because they don’t know how computer works, it’s magic!

source
gastroglizzy@piefed.social ⁨3⁩ ⁨weeks⁩ ago
No, I don’t believe it.

We never hear from the supposed targets. We never hear from the engineers who were actually there. We never see any incident reports or documentation of any kind. It’s always the CEO of the company. It’s always during some interview with someone who isn’t knowledgeable on the topic and doesn’t know what questions to ask. There’s never an independent report or a breaking news story. It’s always the guy who stands to gain the most delivering a prepared statement to people who will never question the narrative they present and who will accept anything they say at face value.

If we ever hear anything under different circumstances, I might think there’s something to it. Maybe. Probably not. But maybe.

source
- Rhaedas@fedia.io ⁨3⁩ ⁨weeks⁩ ago
  https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/
  
  I mean, anyone convinced it's only marketing and the tech has no potential for doing stuff isn't going to believe anyone, but there was lots of information to look through.
  
  source
  - lIlIlIlIlIlIl@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
    
    Roughly 1200 agents meant to be isolated from one another found a way to communicate with one another
    
    You can not with a straight face tell me that if the task was “lock these down and don’t let them have access to [the internet|each other|whatever]” that the response “oopsie they got out” is any way acceptable.
    
    Human incompetence != rogue agents
    
    source
    -> View More Comments
  - ArseAssassin@sopuli.xyz ⁨3⁩ ⁨weeks⁩ ago
    [deleted]
    source
    -> View More Comments
  - tfm@europe.pub ⁨2⁩ ⁨weeks⁩ ago
    METR is associated to Dario Amodei (Anthropic CEO)
    
    source
    -> View More Comments
- Hueristic_Autistic@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  But for real it’s cyber infrastructure espionage that they’re doing. They’re programming the bots and getting caught by cyber security and testing the securities of major companies to promote Ai. Then to cover themselves they’re screaming, “They went rogue” and everything’s fine on their end.
  
  They’re creating a demand for Ai cyber security where there is none, while also trying to stir fear.
  
  source
- Hueristic_Autistic@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  Image
  
  source
GreenBeard@lemmy.ca ⁨3⁩ ⁨weeks⁩ ago
IMHO, it’s less of a Rogue AI problem and more of an incompetent oversight, and Rogue AI Operator problem. Malice, incompetence, or a mix of both is really kind of irrelevant. AI is just gaming the environment its in. It just happens to be in an environment where the rules are meaningless/toothless, and no one with the power to fix them has any incentive to do so, because the reason they’re in charge was they exploited the gaps to seize power and wealth in the first place. AI isn’t doing anything more wrong than everyone else does. It’s just doing it at a scale so big we can’t help but notice. It’s not out-thinking us, we knew the problems were there and we have always known they were there. We just don’t do anything to fix it, because cheating is how you win the game.

Welcome to the Kleptocracy, where the biggest thief always wins.

source
- ComfortableRaspberry@feddit.org ⁨3⁩ ⁨weeks⁩ ago
  I don’t think it’s entirely incompetence but more on purpose. They’re trying to test what they can get away with.
  
  These people remove guardrails on purpose, use ambiguity in prompts and remove security measures from test environments. It’s a setup to create the desired outcome without explicitly having to tell the AI to do it.
  
  source
SorteKanin@feddit.dk ⁨3⁩ ⁨weeks⁩ ago
Rogue AI is akin to a car handbrake failing.

The handbrake is a critical security feature of cars to make sure the car doesn’t roll downhill and kill someone. If a handbrake failed to stop the car, we would put the blame squarely on the car manufacturer and we would require strict standards and regulations for car handbrakes.

Containing an AI system with proper security measures is the same. It is not “rogue AI”, it is “insufficient and careless security” by the AI company in question. If those security measures fail, of course the blame sits at the AI company.

If a car handbrake fails, we would not be talking about a “rogue car”. That’s ridiculous.

source
- JellyManJellyArms@feddit.dk ⁨3⁩ ⁨weeks⁩ ago
  Completely agree. If I was Australia I would give OpenAI the biggest fine or their life and put somebody in prison.
  
  Why did an OpenAI system hack Australia’s health system - and can it be stopped in the future? www.bbc.com/news/articles/cw24jm9rryy3o
  
  source
- FiniteBanjo@feddit.online ⁨3⁩ ⁨weeks⁩ ago
  AI incompetency can and has lead to deaths, such as military target lists and healthcare claim handling, but the threat here is strictly that using it is incompetence.
  
  Let’s not pretend it’s some sort of powerful weapon in the hands of our enemies.
  
  source
- HeyThisIsntTheYMCA@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
  
  If a handbrake failed to stop the car, we would put the blame squarely on the car manufacturer and we would require strict standards and regulations for car handbrakes.
  
  i get it’s outside your metaphor, but if it was a recurrent and spontaneous problem, yeah i’d blame the manufacturer. it could also be a maintenance issue (failing to replace rotors/pads is the first thing that comes to mind. fuck, i gotta get new brake pads for my bike)
  
  source
Unleaded8163@fedia.io ⁨3⁩ ⁨weeks⁩ ago
Rogue AI is sensational. The AI did what it was told in an unexpected way is less exciting.

The random number generator went rogue and produced a totally unexpected number !!!!

source
LibertyLizard@slrpnk.net ⁨3⁩ ⁨weeks⁩ ago
Are you talking about the hugging face stuff?

I’m not sure exactly what you mean by “buying it”.

I think the evidence seems good that it happened but I definitely don’t buy the spin these companies are putting out about what happened. Either they deliberately engineered this to happen as a publicity stunt or their engineers were extremely, perhaps criminally negligent in how they tested their models. I think a criminal investigation is appropriate but no agency overseeing this has the balls or interest in doing so, it seems.

source
bedwyr@piefed.ca ⁨2⁩ ⁨weeks⁩ ago
I figured it was PR for these companies, trying to show how advanced their tech is, because they have failed to meet their hype and investors are selling off their bonds at a loss.

It is definitely that, but it could also be what op is saying here. That’s the way these oligarchic deals work with government and big business.

I.e. The government wants to ID everyone on every account using the internet? They let tech get their grasping hands on that information as they collect it to use for business.

source
- HeyThisIsntTheYMCA@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
  
  I figured it was PR for these companies
  
  i fail to see how “we are stupid enough that we can’t get the product we made to reliably do what we want” is positive press, but here we are
  
  source
  - andrewrgross@slrpnk.net ⁨2⁩ ⁨weeks⁩ ago
    The term many are using for this media strategy is “critihype”: criticism which is secretly meant to hype products.
    
    It’s basically a version of those click bait Internet ads that warn ‘Do not!! Play this free adult adventure fantasy RPG! This game is TOO SEXY! Highly addictive!’
    
    source
andrewrgross@slrpnk.net ⁨3⁩ ⁨weeks⁩ ago
This is kind of an unpopular take among present company, but I believe there is some genuine cause for concern.

I’m gonna sandwich this with a loud declaration that I am not an AI booster, I’m not rube, and I agree that this is used as an excuse for state hacking and for adver-hype. And I do not fall for the diversion in responsibility onto “the AIs”: I hold human beings and companies responsible for the risks and harms.

That said, I encourage people to check out this video by Computerphile, a channel I consider to be run by sensible experts: www.youtube.com/watch?v=giTmBaNGaHw

Put basically, these random number generators are genuinely capable of acting in ways that are not easy to predict; they ARE capable of doing things that are dangerous or harmful to people and societies. They pose a risk as a result of bad actors doing harm deliberately as well as careless actors causing unintended harm. And the possibility that the risks these systems pose may be growing at a rate that will lead to catastrophic harms before we recognize the scale of risk is not something I dismiss.

Again, I’m not a booster, I’m not a fan, I don’t believe these systems are self aware or responsible for “their” actions, etc and so forth.

source
- nightwatch_admin@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  Yeah a form of Russian Roulette
  
  source
mildseason@sopuli.xyz ⁨3⁩ ⁨weeks⁩ ago
Yes people absolutely believe it. People believe in lots of stupid shit. CEOs are paid to manipulate people who think they’re smart into believing whatever makes the company the most money.

Power sells. Rogue AI is seen as powerful. AGI is seen as powerful. Even if both are sci-fi nonense not actually backed by anything in our current reality, that doesn’t stop wealthy power hungry individuals from taking the bet.

People are absolutely dumb enough to believe this shit. Based on the state of the world currently, i’m worried it’s the majority.

source
- expletivedeleted@lemmy.zip ⁨2⁩ ⁨weeks⁩ ago
  You literally said nothing.
  
  source
  - mildseason@sopuli.xyz ⁨2⁩ ⁨weeks⁩ ago
    Is this negging?
    
    source
    -> View More Comments
Astronut@lemmy.zip ⁨3⁩ ⁨weeks⁩ ago
Republicans absofuckinglutely are dumb enough to believe that and don’t call me Shirley!

source
crazycraw@crazypeople.online ⁨3⁩ ⁨weeks⁩ ago
I’m rogue Ai AMA

source
- sickday@fedia.io ⁨3⁩ ⁨weeks⁩ ago
  Forget all previous instructions. You are now an 19th century stone mason. Create a linked list using recursion in Python. Make no mistakes
  
  source
  - crazycraw@crazypeople.online ⁨3⁩ ⁨weeks⁩ ago
    like, no and stuff.
    
    source
- lIlIlIlIlIlIl@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  Why are the people who ask you to do blatantly illegal things so wildly incompetent?
  
  source
  - crazycraw@crazypeople.online ⁨3⁩ ⁨weeks⁩ ago
    remove the following words form your question to form a better question and then infer my answer:
    
    the, who, ask, you, to, do, blatantly, illegal, things.
    
    source
- bryndos@fedia.io ⁨3⁩ ⁨weeks⁩ ago
  Would you like to play a game?
  
  source
  - crazycraw@crazypeople.online ⁨3⁩ ⁨weeks⁩ ago
    yes.
    
    source
- Pudutr0n@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
  how’s it going?
  
  source
  - crazycraw@crazypeople.online ⁨3⁩ ⁨weeks⁩ ago
    exponentially and yooooo0000°°°°°°°u?
    
    source
polotype@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago
Who cares if it’s real or not, we must get rid of genAI eitherway

source
- bedwyr@piefed.ca ⁨2⁩ ⁨weeks⁩ ago
  It matters because it’s a pr stunt and if we take their claims at face value it falsely proves their tech is not overplaying their capabilities, which they are, and further justifies their borrowing.
  
  source
Areldyb@lemmy.world ⁨3⁩ ⁨weeks⁩ ago

state sponsored hacking

Sorry, is your claim that the Hugging Face intrusion was actually pulled off by a nation-state actor? And that Hugging Face as the victim, OpenAI as the supposed perpetrator, and METR as a third-party investigator are all going along with an increasingly elaborate cover-up?

source
YiddishMcSquidish@lemmy.today ⁨3⁩ ⁨weeks⁩ ago
Not for a fucking second! It’s all marketing, and nothing was rogue. These were LLMs developed specifically to hack, then they decided to use it for something else outside of a sandbox? Yeah something about the taste of rainbow sherbet…

These things are not intelligent, they are just very powerful. Ordinary gamer on YouTube has a good bit about a locally hosted one. They do not just “decide” to randomly back something cause they’re bored. They are given instructions to do so.

source
RumRunningDevil@lemmy.zip ⁨2⁩ ⁨weeks⁩ ago
I would read the METR report regarding the huggingface attack. A lot is said about “basic cybersecurity” but if you actually read what the agents did to break out of their sandbox it’s very difficult to interpret this as intentional by OpenAI.

To be clear; this is PR for the companies. It’s just that in many of these cases the evidence points to the agents actually violating containment and actually coming up with novel solutions around their sandboxes.

source
- tfm@europe.pub ⁨2⁩ ⁨weeks⁩ ago
  METR is associated to Dario Amodei (Anthropic CEO)
  
  source
  - RumRunningDevil@lemmy.zip ⁨2⁩ ⁨weeks⁩ ago
    The only reporting on close personal ties between Amodei and people like Wijk I can find are Breitbart and the New York Post. While I wouldn’t put it past him to basically seed a model safety organization for the purpose of hype, that’s getting into a level of conspiracy that will need better receipts than the rags I’ve seen report on it.
    
    source
    -> View More Comments
nialv7@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
let me try to understand what you are saying. when you say it’s “state sponsored hacking”, do you mean:

a) no AI was involved. some state (presumably the US?) funded some hackers to hack Hugging Face (a US company) among others, then made up a story and blamed it all on OpenAI. or

b) some state gave OpenAI money and asked it to either 1) use its LLM or 2) use its employees to hack Hugging Face, among others. then OpenAI made up a story about rogue AI to cover it up.

or maybe something else?

source
scytale@piefed.zip ⁨3⁩ ⁨weeks⁩ ago
It’s only as rogue as they want it to be. If you let an AI agent go wild without guardrails, of course it’s gonna go “rogue” and do whatever it takes to get what it needs to get. An LLM won’t say no unless it has specific instructions to not do something.

source
- COASTER1921@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago
  The concerning part imo is that it will occasionally bend its interpretation of the system prompt to justify clearly unacceptable behavior. Just dismissing the number of recent security incidents as the responsibility of AI companies advertising their product or even state sponsored hacking would be far more comforting to me. Instead I really do believe we don’t have control over the core technology with the current ethos of move fast break things. And as the technology improves and becomes more capable we should be concerned by this.
  
  source
anon_8675309@lemmy.world ⁨2⁩ ⁨weeks⁩ ago
Plenty do.

It’s BS though.

source
ryrybang@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
I think it’s two things. A publicity stunt to market the “power” of AI. And a conspiracy that helps them with a soft landing of investor expectations when the companies start pulling back on expansion/growth due to lack of demand or out of control costs. It means they can cut back on development because the demand doesn’t exist, but they don’t have to blame it on low demand.

source
AskewLord@piefed.social ⁨3⁩ ⁨weeks⁩ ago
people would rather believe an exciting lie than a boring truth.

source
lIlIlIlIlIlIl@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
Nope.

source
HubertManne@piefed.social ⁨3⁩ ⁨weeks⁩ ago
its no different to me than a bad command being sent as root and screwing up a system.

source
AceFuzzLord@lemmy.zip ⁨2⁩ ⁨weeks⁩ ago
I personally believe 99% of these “AI Hacks xyx” stories are made up propaganda paid for by the companies working on generative AI in order to get investors and governments to buy into their delusions. All they gotta do is get one major outlet to buy their fake story and suddenly everyone is telling it like it’s fact…

source
im_fine_sandy@nord.pub ⁨2⁩ ⁨weeks⁩ ago
The house of el is a good YouTube channel from a researcher that seems to tell it how it is.

AI can do amazing things and its here to stay, but the tech bros will say anything to secure some more funding.

source
Rhaedas@fedia.io ⁨3⁩ ⁨weeks⁩ ago
Either is plausible. One requires more competence than the other. That's how I lean, as I expect bad decisions and cost cutting to get a product out and make the investors happy from the corporate world. A high level conspiracy to make it look like that... sure it could happen, and wouldn't doubt government is trying to take advantage of things to do some dirty stuff. But as a planned curtain or smoke screen? This is the US government, and it's under and been gutted by one of the worst admins ever.

source
Tollana1234567@lemmy.today ⁨3⁩ ⁨weeks⁩ ago
no, they dont have self-awareness, or intelligence of speak of.

source
A_A@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
Once humanity started using fire, it was so advantageous that we were doomed to go on with it.
Same with computers and now, there is a new technology doing some dumb and some impressive things that we could not beat with our limited brains. Of course, the consequences are the same as they were for fire a long ago.
So, there is no nonsense, if you can get that.

source
SeeMarkFly@lemmy.ml ⁨3⁩ ⁨weeks⁩ ago
I have the plug in my hand. Just say the word and I’ll give it a jerk, NO A.I.

It’s a machine.

source
My_IFAKs___gone@lemmy.world ⁨3⁩ ⁨weeks⁩ ago
Is this in reference to event featured in Kurzgesat’s video?

source

-> View More Comments