AI Risk, plainly
The key ideas in AI safety, explained for readers without a technical background, together with what research has found so far.
Every entry covers
- What it isA plain definition of the idea, without jargon.
- Why it mattersThe part it plays in arguments about serious risks from AI.
- The evidence so farWhat research has found so far, for and against.
Key topics in AI safety
6 published, more in preparation
How AI could go wrong
In preparation: Reward hacking
How we test and monitor AI
- Evaluation awarenessAI systems can often tell when they are being tested, which makes test results a less reliable guide to how they behave in real use.
- Chain-of-thought monitorabilityMany AI models write out their reasoning before they act. Reading it is one of the few ways to catch misbehaviour in advance, but this crucial safety tool could disappear.
Catastrophic risk
- Why might AI want to kill us all?Not out of hatred or human-like lust for power. The worry is that almost any goal would give a capable AI reason to take power from us.
- Why might AI be able to kill us all?There is no one predictable pathway, but sufficiently capable AI could use its superior planning, speed, scale and access to the systems we increasingly hand over to AI.
- Automated AI R&DAI systems are doing a growing share of the work of building better AI, which could make technical progress much faster, and decouple it from human oversight and response.
In preparation: P(doom) · Why build AI if it might kill us all? · AI-enabled power concentration
Minds and moral status
In preparation: AI anthropomorphisation · Stochastic parrots