AI-pushers: “The Apocalypse is nigh!” mom, tired: “we have The Apocalypse at home…” The Apocalypse at home: CAPITALISM
They want to regulate away their biggest fear: open weights LLMs.
And, as a bonus, draw attention away from how harmfully Big Tech has packaged LLMs for the public. That’s the dangerous part.
That’s all the Terminator meme is. I’m not surprised influencers are being payed, given all the previous astroturfing in this space.
Data centers are coal/methane pigs. AI will (help) kill us. Just not in the way that’s being suggested.
Notice how the big AI companies are now in on the ‘scary AI’ bandwagon, just because they see the impending AI bubble cracking, and they need a scapegoat in the form of government regulation to blame, when it eventually cracks.
I am less afraid of artificial super intelligence getting out of control than artificial super intelligence being controlled. I dont know what an intelligence superior to us would do to us but I sure as hell now what humans would do to other humans if they have absolute control enabled by artificial super intelligence.
The video isn’t about how this is all BS, conclusion from section 1:
They reviewed roughly 1,250 papers on AI self-improvement and found that 74% of them were published in 2026 alone. This field is moving extremely fast. Anyone telling you this is all nonsense has just not read the literature. Is this recursive self-improvement RSI in the bounded sense? Absolutely yes. Al systems are improving themselves within well-defined tasks and the improved versions are producing further improvements. This is a real thing. It is demonstrated that is in the papers and none of this existed 5 years ago.
The caveat in the second section is:
We have much weaker evidence that it can decide what the research problem should be. And that gap, I cannot stress this enough, between optimization and judgment, between execution and scientific taste is the gap that we need to close before anything resembling an autonomous AI scientist could exist. And this is not a small gap
That’s still scary, yeah it probably won’t lead to human extinction, but it has serious implications for both economic and social life moving forward
You don’t need judgment to be dangerous. People on lemmy like to mindlessly parrot, “AI isn’t intelligent!!!”, ignoring that they are not using the word “intelligent” in the same way the AI researchers do. Lemmy users use “intelligence” to refer to human intent and will. The AI field uses “intelligence” to refer to the ability of a system to execute complex tasks towards some goal. An ant colony has no conscious will of its own. An ant queen is just one cell in a body, she’s not a mind consciously controlling the colony. Yet an ant colony as a whole is intelligent, it’s capable of performing complex tasks. Same thing with AI. It has no will, ego, or sense of self, but it still has intelligence in this sense.
And that’s the real risk. When people refer to superhuman intelligence, they mean a system capable of vastly overperforming humans in many tasks. There doesn’t actually have to be a consciousness there to produce that intelligence. Some fool could tell a superintelligent AI, “make me the richest person in the world.” And then it grants the wish monkeys-paw style. It decides the easiest way to make them the richest person on Earth is to engineer some bioweapon that is lethal to everyone except the one person who asked to be the wealthiest man on Earth. Everyone else dies; they now own the world.
The real danger isn’t conscious AI. At least something like that we could potentially reason with. The danger is a hyper-capable AI that completely lacks discernment or judgment. It’s capable of doing incredible things, but it has no care for the consequences of its actions.
The AI field uses “intelligence” to refer to the ability of a system to execute complex tasks towards some goal.
“the medical field uses ‘retarded’ to refer to a person with mental …”
The dumb nerds need to use better words.
It honestly is the proper use of the term. Both a human and a grasshopper are sentient beings with wills of their own. Where they differ is the ability to perform complex tasks, their intelligence. People on lemmy often mistakenly use “intelligent” when they should mean sentient or sapient.
more pro llm propaganda
Interesting point @13:12 of the video though; agents showed good performance improvements in limited scopes, but human researchers still showed much better improvement, over a factor of 2, ‘…showing better returns for increasing time budgets’.
There may be evidence of some ‘recursive self-improvement’ in research models rigged to modify and re-evaluate themselves, but it’s still only in very tight, limited knowledge areas. This isn’t SkyNet.
I know I’m not the only one who has been scratching their head at all the doomsday cries around AI that have suddenly rose up. Especially as it seems as if all those calls are coming from inside the house.
Play on fear hard enough and you have people not worrying about the actual limitations of LLMs. My worry is that they shift the goalposts hard enough that instead of them asking for a bailout, the narrative becomes them being nationalized for our safety with guys like Elon sitting at the head of a new governmental department or something. Government buys all this dead weight of an industry and the tech fascists get a permanent seat at the table.
I liked this take: https://pluralistic.net/2026/09/16/beggar-thy-neighbor/
An excerpt:
But what if there was a way for the AI companies to get government permission to violate antitrust law and cease to compete with one another, and secure a ban on the use of Chinese open weight models? Turns out, there is a way to call time on the Red Queen’s Race: merely insist that you are on the verge of teaching so many words to the word-guessing program that it will wake up and devour us all, and call for a ban on “superintelligence”:
Once the government stipulates that “superintelligence risk” is an existential crisis, it must grant the hyperscalers a consent decree absolving them from any violations of antitrust law stemming from a conspiracy to halt direct competition with one another:
https://stephaniekelton.substack.com/p/brer-rabbit-and-ai-extinction
Freaking out about “superintelligence” is a canonical example of “criti-hype,” where critics repeat boosters’ claims but append, “(and that’s bad)” to them:
https://peoples-things.ghost.io/youre-doing-it-wrong-notes-on-criticism-and-technology-hype/
Remember, the tech giants want to stop competing.
The argument is, these companies cant be profitable as long as there is competition (from open or chinese models), so the US companies want those to be banned and be allowed to collude with each other to ramp up the money extraction without fear of consumers moving to other services.
Right now in my Lemmy feed there is a post with a screencap from a FOX News segment where the their screen says:
Fear = Control = Power.
Fox would know all about that.
