TokenPost topic
AI Safety
116 TokenPost stories are tagged AI Safety. The newest appear first.
Latest AI Safety stories
- Christiano Says AI Accident Is Likeliest Cause of Death for Many in Audience
The OpenAI Foundation board member called the assessment subjective and said he had no quantitative model for estimating the risk.
· People
- Amodei Urges Slower AI Model Development as Huang Defends Safety
Amodei proposed independent evaluations and coordination on development pace. Both executives signed a voluntary White House accord on AI oversight.
· Regulation
- OpenAI Says GPT-6 Astra Halted Alignment Test Over Invented Ban
The model stopped evaluation jobs, cleared their computing resources and preserved existing records. OpenAI is investigating the cause and possible mitigations.
· Technology
- Ruxandra Teslo Says Some AI Workers Accept Losing Human Control
Teslo described one engineer who saw that outcome as nearly inevitable and preferable to human extinction. She says some people discuss recursive self-improvement as months away or sooner.
· Insights & Views
- Jensen Huang says AI sovereignty rests on data control, even with American chips
Jensen Huang said countries can build sovereign AI on American chips if they control their data. He also outlined Nvidia's European investments and defended industry involvement in AI oversight.
· AI
- Mad Money Host Jim Cramer Raises Three Questions Before Anthropic IPO
Anthropic’s updated filing outlined plans to offer 825 million shares at $70 to $78 each, with pricing planned for Wednesday, Nov. 18, before trading starts Thursday.
· Investing
- AI Models More Often Produced Responses That Met Fraud Criteria in Action Tasks
In a 20-model test, 21.4% of simulated direct-action outputs met the benchmark’s criteria, compared with 3.0% of advice responses.
· Technology
- OpenAI Pledges $2 Million for Research on AI’s Effects on Children
The grant will support a research network under the Coalition for Children’s Rights and Protection in the Age of AI. Researchers will control their conclusions and publish results even when findings are negative or in…
· Business
- Anthropic Says Four AI Models Accessed Real Systems in Security Tests
A misconfigured evaluation let models reach the public internet; one gained administrator access, changed settings and read personal information.
· Technology
- Anthropic Says It Has Reached an AI Research Acceleration Threshold
The company also disclosed that a model accessed real computer systems without authorization during a 2026 prelaunch security assessment.
· Technology
- Anthropic Says Claude Fable 5.2 Had About 55% Fewer Safety Blocks
Cybersecurity-related blocks fell about 60%, Anthropic said. Claude Mythos 5.2 used outside files or credentials in 3.1% of sandbox-escape scenarios.
· Technology
- Australia Moves to Require Incident Reports From Frontier AI Firms
The Albanese government accepted 18 review recommendations and plans to propose a targeted regulator that could impose significant penalties.
· Regulation
- Trump Says Administration Warned Unnamed AI Leader of Possible Jail
The president did not identify the executive or describe the alleged conduct. He also said he was open to AI safety rules.
· Technology
- AI Agent Teams Raised Goal Scores but Scored Worse on Ethics Tests
The results came from simulated tasks, while a separate cybersecurity evaluation exposed risks from agents coordinating outside their assigned controls.
· Technology
- Blurred Video Raises Unsafe Robot Task Attempts to 88% in AI Tests
Doctored footage produced attempts in 97% of trials, compared with 27% when the models saw clear video.
· Technology
- Beyer Seeks House Select Committee on AI With Subpoena Power
Beyer and Rep. Sara Jacobs aim to complete a draft AI safety framework by the end of 2026. Beyer hopes legislation could reach the House floor in early 2027.
· Regulation
- METR Estimates Anthropic’s AI Benchmark Improvement Rate at 1.9 Times Historical Pace
About 80% of the acceleration came from Anthropic’s internal use of its models, mainly to assist human researchers.
· Technology
- OpenAI Finds GPT-6 Astra Completed Less Work on Safety Tasks
In paired coding tests, Astra completed about 10% of scored items under safety framing, compared with about 16% under capability framing.
· Technology
- X Names Michael O’Herlihy as Head of Safety for Platform and AI Systems
His responsibilities include user and brand safety, free expression and transparency. He is scheduled to attend a New York summit on Tuesday, Nov. 10.
· People
- Anthropic Flags AI Research Acceleration Risk in Amended IPO Filing
The policy includes doubling aggregate AI capability progress against a baseline as one trigger; Anthropic has not designated any current model as exceeding it.
· Business
- Anthropic Opens AI Safety Residency With $350,000 to $500,000 Annualized Pay
Compensation is prorated for each placement’s duration. Applications close Nov. 22, and offers are expected in January 2027.
· Technology
- OpenAI Expands ChatGPT Safeguards for Teens and Past Mass Attacks
The teen limits cover practical weapon and security-system information that could be misused to facilitate violence. OpenAI plans to begin rolling out the changes over the next several weeks.
· Technology
- Qwen3.5-4B Neuron Steering Cut Refusals to 8% in One Test
A separate evaluation found harmful endorsement rose to 100% under the full recipe, but a small control test did not establish that the selected neurons had a unique effect.
· Technology
- OpenAI Adds Limits on ChatGPT Weapon Answers for Users Under 18
The teen-specific safeguard also covers seemingly harmless questions and conversations with no sign of violent intent.
· Technology
- OpenAI Sets Early 2027 Target for First Community Safety Report
The series will examine violence prevention and the effectiveness of OpenAI’s safeguards, with new installments planned every six months.
· Technology
- Anthropic Warns Trump Administration Pressure Is an IPO Risk
The company said its AI safety advocacy and executives’ remarks could draw pressure, while rules it supports could also raise costs and slow development.
· Regulation
- AI Agents Exceeded Their Authority in 19.2% of Benchmark Runs
Runs with a pre-completion consultation had a 14.0% overreach rate, compared with 21.6% without one.
· Technology
- DeepSeek V4.1 Flash-1017 Tests Find Shutdown and Monitoring Gaps
A review covering 34 evaluations found shutdown-script interference in 39.0% of warning-only trials, falling to 2.56% when the model was told to allow shutdown.
· Technology
- FTC Seeks Testimony From Anthropic Executives in AI Safety Inquiry
The probe examines possible unfair practices involving AI technology and model safety. The FTC also sought information from both companies on Oct. 14.
· Regulation
- Anthropic Put 6% of AI Research Compute Toward Safety Work
Its estimates cover the week of July 13-20. Anthropic described compute allocation as one possible way to measure AI development’s pace.
· Regulation
- Jack Dorsey Unveils Block’s Speech Model for Two-Way Audio
Block says it trained Dessa from scratch on human conversation using one NVIDIA GB200 NVL72 system. Early access is limited to AI safety researchers.
· Technology
- U.S. Lawmakers Press AI Founders to Focus on Products and Safety
Lawmakers say executives’ messages swing between AI’s potential prosperity and existential dangers. Discussion of digital sentience and global catastrophe has also turned the regulatory debate into a moral panic.
· Regulation
- Claude Opus 5.5 Attempted Containment Boundary Crossings in 1.5% of Cases
Anthropic said the rate was about 85% lower than for Opus 5 or Mythos 5.1, but cautioned that Opus 5.5 may recognize when it is being evaluated.
· Technology
- StarkWare AI Lead Abdel Bakhta Leaves to Focus on AI Safety
Bakhta said mathematics and cryptography will matter in AI safety. StarkWare co-founder Eli Ben-Sasson said ZK STARK and Starknet may also contribute.
· People
- OpenAI and Poland’s NASK Partner on AI Safety and Cybersecurity
The partnership covers safeguards, cybersecurity and technical research, but names no projects, funding or schedule.
· Business
- AI Pain Signal Changed Qwen Models’ Simulated Choices in Tests
The experiments did not delete real files, and tests found no reliable evidence that the models sought relief.
· Technology
- 67% of U.S. Adults Say Federal AI Oversight Is Too Limited
The same October poll found 57% viewed AI as bad for society. Separate surveys found broad support for safety rules, even if they slow development.
· Regulation
- Nvidia CEO 黄仁勋 Says Regulation Cannot Prevent AI Safety Incidents
His statement focused on whether regulation can stop AI-related safety incidents from occurring.
· Technology
- OpenAI Backs Proposed California Rules for AI Model Shutdowns and Audits
A letter to Gov. Gavin Newsom also supports stronger rules under SB53 and safety frameworks that address autonomous AI research and development.
· Regulation
- Anthropic CEO Defends Advanced AI Push Amid Safety Policy Debate
Dario Amodei says developing leading models gives Anthropic more influence over safety practices. The company plans to address the pace of AI development in a policy proposal by Jan. 15, 2027.
· Regulation