header logo
Vallibel FD
vallible LE
Science & Tech
Anthropic calls for ‘brake pedal’ before AI develops itself without human oversight
Anthropic calls for ‘brake pedal’ before AI develops itself without human oversight
Mobitel inner 1278

Anthropic co-founder Jack Clark wants the AI industry to pump the brakes before the technology starts further developing itself without human input.

 

Speaking to the BBC, Clark said 80% of Anthropic’s coding work is already being done by its AI Claude, and that it could go up to 100% in a couple of years. However, he said “it’s a choice” whether AI companies let it get that far without stopping it.

 

“We think this is a topic that the world should be talking more about,” Clark said. “The AI industry right now has a gas pedal, but it doesn’t have a brake pedal in the car, and we want to do some of the work to build that pedal.”

 

This process is called “recursive self-improvement,” where an AI is able to improve itself without human input, according to Anthropic in a related blog post from Thursday night.

 

In a recursive model, AI agents, the autonomous workers built by a chatbot, could “become capable enough to build and train models themselves,” so Claude “could be continuously improved by Claude,” Anthropic said.

 

While recursive AI could bring some good to the fields of science and healthcare, Anthropic warns that it might mean increasing “the risks of humans losing control over AI systems.”

 

“If systems are capable of fully building their own successors, the ways we secure them, monitor them and shape their behaviour all grow much more important,” the blog post reads.

 

There is evidence within Anthropic’s own model that recursion is coming sooner rather than later. It points to the fact that code correction rates by their staff have been falling steadily for the last year, which means there are fewer errors in what Claude is producing.

 

Claude is also able to run its own research experiments when given an open-ended question, such as “Can a weaker model supervise a stronger one?” and come up with its own solutions without human input.

 

“The evidence suggests that the human role is narrowing at each step in the AI development process,” the blog reads.

 

Anthropic said its institute will conduct research to build a system to check whether developers have actually stopped or slowed down the move towards recursive AI, it said.

 

However, a real slowdown would require “multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions.”

 

Source: Euro News

 

--Agencies 

Disclaimer: All the comments will be moderated by the AD editorial. Abstain from posting comments that are obscene, defamatory or slanderous. Please avoid outside hyperlinks inside the comment and avoid typing all capitalized comments. Help us delete comments that do not follow these guidelines by flagging them (mouse over a comment and click the flag icon on the right side). Do use these forums to voice your opinions and create healthy discourse.

Comments

0
Comments are disabled

Note: Due to legal constraints, the Ada Derana Editorial team has taken a decision to disable comments on all articles concerning ongoing court procedures.

MostRead
Mobitel 1278
VideoStories
Two company directors arrested and remanded  for allegedly defrauding 2,000 individuals of Rs. 4 bln

Two company directors arrested and remanded for allegedly defrauding 2,000 individuals of Rs. 4 bln

Opposition Leader calls for stronger protection of the  fundamental rights of children, women &youth

Opposition Leader calls for stronger protection of the fundamental rights of children, women &youth

Dilith Jayaweera-led Sarvajana Balaya delegation  in Japan for GLEC summit

Dilith Jayaweera-led Sarvajana Balaya delegation in Japan for GLEC summit

Earth-sized planets open new avenues 	 in search for life – NASA scientist Dr. Sarath Gunapala

Earth-sized planets open new avenues in search for life – NASA scientist Dr. Sarath Gunapala

Top UN court finds Myanmar committed genocide  against Muslim Rohingya

Top UN court finds Myanmar committed genocide against Muslim Rohingya

Headline inflation expected to reach 9%  by month-end - CBSL Governor

Headline inflation expected to reach 9% by month-end - CBSL Governor

China a close and dependable friend  of Sri Lanka, says Minister Anil Jayantha

China a close and dependable friend of Sri Lanka, says Minister Anil Jayantha

Can satellite data help improve disaster response? Senior Research Scientist at NASA explains

Can satellite data help improve disaster response? Senior Research Scientist at NASA explains

Bitumen shortage disrupts road development projects in Sri Lanka; RDA seeks alternative supplies

Bitumen shortage disrupts road development projects in Sri Lanka; RDA seeks alternative supplies

Minister urges govt workers engaged in trade union action to submit proposals to Salaries Commission

Minister urges govt workers engaged in trade union action to submit proposals to Salaries Commission

 Experts warn of significant rise in antibiotic resistance 40,000 non-physicians prescribing drugs?

Experts warn of significant rise in antibiotic resistance 40,000 non-physicians prescribing drugs?

Rajangane Saddharathana remanded over violent clashes in A’pura,  three other suspects granted bail

Rajangane Saddharathana remanded over violent clashes in A’pura, three other suspects granted bail

Appeals Court upholds four-year prison sentence  imposed on Ven. Gnanasara Thero

Appeals Court upholds four-year prison sentence imposed on Ven. Gnanasara Thero

Court suspends bail granted to Namal Rajapaksa Shashi Weerawansa’s two-year prison sentence upheld

Court suspends bail granted to Namal Rajapaksa Shashi Weerawansa’s two-year prison sentence upheld

 “I exposed corruption and not wronged” Trade union activist Ananda Palitha remanded until Nov. 25

“I exposed corruption and not wronged” Trade union activist Ananda Palitha remanded until Nov. 25

Lassana Flora