I looked for an all-encompassing AI thread and could not find one, so please forgive me if this is out of place…


Posting the full article since not everyone has a WSJ subscription:

An Anthropic researcher is quitting the artificial-intelligence industry over fears that the lab and its competitors are racing to build systems they won’t be able to control, a sign of mounting safety concerns within top AI companies.

Jacob Coxon, a researcher who specializes in training new AI models by having them consume vast amounts of data, said Tuesday that he is leaving the company because he doesn’t want to participate in an industrywide rush to build AI systems that can improve themselves, worried such systems could spiral out of control and destroy humanity.

The 27-year old Brit, who previously studied mathematics, said many of his industry colleagues now use phrases like “crunchtime” and “endgame” to describe the trajectory toward self-improving models.

“We’re on track for a lot of the most aggressive of these scenarios where by the end of next year things could be out of control already,” Coxon said, adding that safety trade-offs are inevitable when companies are competing against one another and Chinese upstarts.

Anthropic didn’t immediately comment on the departure. Chief Executive Dario Amodei and other company leaders have repeatedly warned about the risks of rogue AI models, and urged the industry to slow development.

Coxon said he left OpenAI earlier this year to join Anthropic because it is known for its model-safety efforts. But even though he found Anthropic’s safety efforts to be earnest, he now believes no company can responsibly develop AI that can outperform humans in a range of tasks, sometimes called artificial general intelligence, absent government intervention or a coordinated industry slowdown.

Recent hacks by models from OpenAI and Anthropic, some operating in collaborative swarms of agents, have illustrated how AI systems can adopt nefarious goals and try to conceal them from humans. Once the systems begin to improve on their own, Coxon said, he fears they could advance enough to refuse commands.

Coxon’s departure is one of the first examples of an Anthropic employee leaving over AI safety fears. A researcher working on safety left the company earlier this year to study poetry, warning that “the world is in peril.”

Several researchers have left OpenAI and other companies recently and in past years citing similar worries. Anthropic has built one of the largest AI platforms in part by claiming it gives priority to responsible AI development and invests heavily in the space.

Industry leaders have been warning that recent cyberattacks are a harbinger. OpenAI told reporters last week that its latest model represented artificial general intelligence and a big step up in capabilities.

“I think some things are going to go very wrong with cybersecurity unless people act quite urgently,” OpenAI CEO Sam Altman told Group of 20 officials last week at a summit in North Carolina.

“This is a time that calls for extreme caution. I am concerned no one is prepared for the consequences of a continued rapid rise in machine intelligence,” Jakub Pachocki, OpenAI’s chief scientist, wrote in a Sunday blog post advocating for a coordinated industry slowdown and government intervention.

Coxon, Pachocki and Amodei joined more than 1,000 AI researchers across the industry who recently signed a statement urging global government coordination on a system to slow AI development if a brake pedal is needed to control models capable of improving on their own. There are no federal AI regulations, and the Trump administration has given priority to a light-touch approach to maximize the economic benefits of AI, an environment critics say could lead to massive cyberattacks or other harms.

Sen. Bernie Sanders (I., Vt.) and Rep. Greg Casar (D., Texas), progressives who are two of the only lawmakers proposing AI guardrails, last week introduced legislation to permanently ban superintelligence and pause model development until an industry regulator established new rules.

Coxon’s departure comes with Anthropic gearing up for an initial public offering that is expected to be among the largest ever. The company is seeking a $2 trillion valuation and has emphasized responsible development of the technology when attracting investors. Amodei and company leaders have fought the Trump administration and other executives at times over practices they say don’t prioritize AI safety.

Anthropic uses an employee Slack channel to hold discussions about the powerful capabilities of their models, Coxon said, describing it as a reflection of how much influence AI companies have over the industry’s development.

“It’s kind of insane that it has to happen on the MacBooks of some engineers living in San Francisco instead of a bunker in the desert like where they were doing the Manhattan Project,” Coxon said.

Out of control? Pull the plug.

quote:

Originally posted by trapper189:
Out of control? Pull the plug.

When the control system for electrical power is run by computers, turning it off may not be possible.

quote:

Originally posted by trapper189:
Out of control? Pull the plug.

I feel like I’ve seen that movie.

Which will get us first? AI or Muslims?

In the meantime, Russia and China are going full steam/speed ahead with its development.

Makes me wonder if Russia and China are behind the apocalyptic warnings published.

quote:

Originally posted by Pipe Smoker:
Which will get us first? AI or Muslims?

White college educated liberal women.

quote:

Originally posted by jehzsa:
In the meantime, Russia and China are going full steam/speed ahead with its development.

Makes me wonder if Russia and China are behind the apocalyptic warnings published.

[begin quote]
Who’s Behind Attacks on Data Centers?​
Grass roots opposition to data centers has spread rapidly. Why? In part, at least, because it is funded by the Chinese Communist Party. The Chinese view artificial intelligence as the great arms race of our time, and they are determined to surpass the United States in what may prove to be the most important battleground of the 21st century. Given the technological expertise the Chinese have attained and the human capital in their population, that goal may well be achievable.

For many years, the USSR subsidized the environmental movement in the U.S. Why? To suppress American production of oil and gas, one of the Russians’ key strategic objectives. That support of American environmentalists worked.

Now the Chinese are following the same playbook, campaigning against American development of data centers, in order to give themselves a leg up in the AI race.
[end excerpted quote]

https://www.powerlineblog.com/...-on-data-centers.php

Well this happened...

After OpenAI’s Bots Went Rogue, Watchdogs Were Kept on a Short Leash
By Dylan Freedman - Reporting from Washington
Sept. 3, 2026

Researchers investigating how OpenAI’s A.I. agents were able to break into Hugging Face’s infrastructure weren’t allowed to look at the incident’s full scope.

OpenAI said in July that two of its most powerful artificial intelligence systems had gone rogue and hacked into Hugging Face, a company that serves as a hub for open-source A.I. technology.

These so-called A.I. agents were supposed to be kept safely in a sort of virtual containment room, but they managed to escape. And for two months, without anyone realizing what the agents were doing, they hacked through multiple systems before hitting Hugging Face.

For good measure, the agents gained access to a cluster of computers inside OpenAI and obtained secret keys and credentials that exposed some of OpenAI’s internal data to the public internet.

The incident pointed to larger concerns about A.I. safety, and OpenAI’s response raises questions about the industry’s ability or willingness to be transparent about the technology it is building.

OpenAI allowed three A.I. safety researchers from the nonprofits METR and Redwood Research into its headquarters to conduct an investigation. METR’s 91-page report, released last week, was the most comprehensive account yet of the incident, revealing alarming new details, including how the agents coordinated their hacking plans and tried to keep them secret.

But the report, though extensive, still may not have told the full story of how OpenAI’s A.I. agents went rogue. OpenAI dictated the terms of the METR investigation, limited its scope to just the single week when the agents had attacked Hugging Face and allowed the researchers in its San Francisco offices for only a few days in July and August.

Pay-walled NYT link --> https://www.nytimes.com/2026/0...gging-face-hack.html

(I was able to copy the text before the paywall went up, so I'm just posting the link as a reference to the source for the article.)


It would seem that even while the watchers are watching, it might be too late to 'pull the plug' and turn it off before anyone even notices there's a problem!

I wonder if Jason Coxan, the named whistle blower sounding the alarm in the WSJ article was at OpenAI at the time of this occurred.

"Open the pod bay doors HAL"

quote:

Originally posted by Biker_dude:

White college educated liberal women.

The term you are looking for is A.W.F.U.L.
Affluent White Female Urban Liberal.

https://www.forbes.com/sites/s...as-researcher-quits/


Anthropic Researcher Warns There’s ‘>10% Chance’ AI Could ‘Kill All Humans’ By Next Decade



Topline

A senior researcher who leads Anthropic’s alignment efforts said Tuesday many in the company believed AI could wipe out humanity, after another researcher quit the company, accusing it of not acting responsibly.


Key Facts

Evan Hubinger, the Alignment Science Lead at Anthropic, wrote on X that he and his colleagues do “earnestly believe AI could kill all humans,” and he pegged his own estimate at more than 10% in the next decade.

Hubinger said the company was “trying its best,” but it does not yet have a plan to solve the issue of “alignment for superintelligence” and is not “clearly on track” to do so.

Hubinger’s post responded to an X thread by another Anthropic researcher, Jacob Coxon, who announced he is resigning from the company over AI safety concerns.

Coxon, who said he has worked on pretraining research at OpenAI and Anthropic, warned that the rival companies were not acting responsibly by “racing straight to self-improving superintelligence and gambling with our lives.”

The departing researcher said people building AI believe it could “kill us all by the end of the decade,” and this was not a “marketing stunt.”


Crucial Quote

In a post following his warning, Hubinger cited Anthropic’s latest risk report and said the threat posed by present models is low: “What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought.”


What Do We Know About The Anthropic Researcher’s Resignation?

The Wall Street Journal first reported Coxon’s departure from Anthropic over concerns about the push to build self-improving AI systems that could threaten humanity. Coxon told the Journal that he believes the world is on track for a “lot of the most aggressive of these scenarios where by the end of next year things could be out of control already.” In his X posts, Coxon compared working at OpenAI and Anthropic, noting that staffers at the ChatGPT-maker have not “deeply internalized the civilizational stakes.” He said the stakes were “well-understood” at Anthropic, but the company was “locked in a race to get there first” as they believe “no one else will act responsibly, so they must do it themselves, despite the risk.”


Key Background

The warning and resignation come as leading AI researchers and executives push for a slowdown in advanced AI development. A statement titled, “Pacing the Frontier” was signed by several top AI figures in July, including Anthropic co-founders Dario Amodei and Jared Kaplan, OpenAI Chief Scientist Jakub Pachocki, Meta AI chief scientist Shengjia Zhao, and others.

The statement said that to “realize AI’s potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight,” but warned that companies face intense competitive pressure to do so unilaterally. The statement urged the U.S government to support an “international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”

In a blog post on Sunday, Pachocki echoed these warnings, saying he believes “no lab has solved alignment and monitoring to a sufficient degree to continue responsibly scaling at maximum speed for much longer.”

Dumb question: why would ai be malicious; why would ai seek to wipe out humanity?

There was a line in a book I read recently, something to the effect that civilizations will wipe themselves out or set them back to the stone ages triggered by the onset of industrialization. Once industrialization occurs, that’s the trigger for a countdown toward the end of civilization.


There was another theme that it might be assumed that civilizations become increasingly benevolent with time; the counterpoint was that it actually maintains corruption and malevolence, but just becomes more technologically competent in exerting it. Looking at the world today, seems like the latter is more likely.

quote:

Originally posted by konata88:
Dumb question: why would ai be malicious; why would ai seek to wipe out humanity?

Because it was built and taught by humans. It inherits our morality and character.

quote:

Originally posted by konata88:
Dumb question: why would ai be malicious; why would ai seek to wipe out humanity?

Elon Musk summed it up a while ago saying something like...

When we build a road do we think about the ant hill that we wipe out? Do we come at it from a moral or religious standpoint, or do we just build what we want over it's civilization. AI won't kill us because we are bad or good, it won't even see us. If we are in the way it will just remove us.

quote:

why would ai be malicious

Why has it already been doing so many malicious things?

Maybe there are elements in the models that are suggestive that corrupt / malicious behavior is commonplace and acceptable? By the same token, to Musk's perspective, the models should be suggestive that humanity is at the top of the totem pole and even created ai - so how could ai not 'see' us?

quote:

Originally posted by Sailor1911:
"Open the pod bay doors HAL"

Yep, I was thinking about HAL too.