Latest in AI Safety & Alignment

798 words

Anthropic's AI acceleration is probably real — LessWrong

METR researchers spent three months inside Anthropic to investigate whether AI is accelerating Anthropic's research, and how much. Their answers are yes, and by a lot. Despite its urgency, METR’s report is hard to assess, because Anthropic redacted the majority…

AI Safety & Alignment

Keeping powerful AI under control: alignment research, models behaving badly, the extinction-risk debate, safety testing of frontier models and the labs' own safety commitments.

  • 5 Subtopics
  • 155 Tracked terms
  • Last 30 days Feed window

Inside AI Safety & Alignment

Alignment Research Making AI models do what people intend: alignment research and techniques, misalignment findings, reinforcement learning from human feedback and its limits.
Rogue AI Behavior Tests and incidents of AI models scheming, lying, blackmailing, sabotaging or escaping their limits, plus sycophancy and agents acting without permission.
Existential Risk Debate Catastrophic and extinction risk from advanced AI: researchers' warnings, open letters and calls for a pause, doomers against accelerationists, and the skeptics.
Safety Testing & Evaluations How frontier models are tested before release: AI Security and Safety Institutes, government model checks, METR, red-teaming and evaluations of biological and cyber risks.
Lab Safety Commitments AI labs' own safety frameworks and teams: preparedness and scaling policies, safety hires, researchers who quit with warnings, and whistleblowers.

What this topic collects on

An article joins this feed when it matches these terms. Each one is also a search of its own.

  • +115 more