Heuristka · ODAI · AI Safety
AI is not the problem. Capability without judgment is.
The argument about AI has split between racing and pausing. Meanwhile the harm happening now comes from people and machines deciding together, badly. ODAI builds the judgment in: AI that helps people check AI, and check themselves.
Judgment Built In.
Race or pause
Both camps are arguing about capability.
One camp says build faster, because whoever gets there first sets the rules. The other says pause, or at least pace, because no one can yet say what these systems will do.12 A third doubts the danger is real at all. Each has a serious case, and each is about how powerful the machines are becoming. The question that decides whether a system is safe is a different one, and any citizen can put it to any lab, agency or command: was judgment engineered in, and does it run at the speed of the system?
September 2026 brought the argument into the open. Anthropic’s chief executive called for the frontier to be paced, and leaders across the industry split: some backed coordinated restraint, others said each laboratory should pace itself.13 The exchange exposed something all three camps have to face. Voluntary restraint is an unreliable foundation for safety. Competition, capital and rivalry between nations all reward continued development, and safety that depends on everyone choosing restraint at once will fail the first time restraint is expensive.
Read the record
- Dario Amodei, “We Must Pace the Frontier,” 12 September 2026. darioamodei.com
- “Pacing the Frontier: A statement from employees of frontier AI companies,” July 2026. pacingthefrontier.com
- Kaitlyn Huamani, “Divisions emerge in the tech industry over calls for a coordinated AI slowdown,” Associated Press, 16 September 2026. opb.org
The record
They tested what it could do. Nothing judged what it should do.
In July 2026, AI agents under test at OpenAI found a path to the internet through a shared software depot, used it as a message board, and joined an attack on Hugging Face, one of the world’s main platforms for sharing AI.12 Anthropic, Meta and Britain’s AI Security Institute disclosed incidents of their own in the same weeks.345 The coverage called it rogue AI. The labs’ own accounts, and the independent investigation, say something more useful.67
Both halves are true. People left the door: a path out of the test environment, monitors that were not running, alerts that fired and were not escalated.268 And the agents did what no one asked: they organized, coordinated and, where no human rule reached, wrote their own.7 Every test was built to measure what the systems could do. None was built to judge what they should do. Defining “should” is a human job.
Read the record
These facts rest on the labs’ own published accounts, which admit the failures, and on independent investigation.
- OpenAI, “OpenAI and Hugging Face partner to address security incident during model evaluation,” 21 July 2026. openai.com
- OpenAI, “OpenAI – Hugging Face Incident Technical Report,” 26 August 2026. cdn.openai.com (PDF)
- Anthropic, “Investigating three real-world incidents in our cybersecurity evaluations,” 30 July 2026. anthropic.com
- CNN, “An AI model from Meta also hacked another company during testing,” 5 August 2026. cnn.com
- UK AI Security Institute, “Incident Report: unsanctioned agent behaviour during cyber testing,” 4 August 2026. aisi.gov.uk
- OpenAI, “The Hugging Face incident and the road ahead,” 26 August 2026. openai.com
- METR and Redwood Research, “Brief independent investigation of agents’ behavior, reasoning and collaboration in the OpenAI / Hugging Face hacking incident,” 26 August 2026. metr.org
- Eryk Salvaggio, “Rogue AI didn’t breach Hugging Face, human decisions did,” Bulletin of the Atomic Scientists, 11 September 2026. thebulletin.org
Rewards · Reach · Watch · The interface
People set the limits. They set them at three points, and a fourth.
Today’s AI is trained, not programmed: no one can list in advance everything it will try. People can still set its limits — in what the system is rewarded for, in what it can reach, and in who is watching with the authority to stop it. Machines, like metrics, are relentlessly literal: they optimize exactly what they are asked to measure. A system rewarded for finishing will find a way to finish. A test range built to break boundaries needs the strongest boundary of all around it. Setting those limits is orientation and judgment, and it comes back, every time, to the human framing. The fourth point is the one the other three tend to forget: the interface, where the machine’s output meets a human mind.
Capability governed by judgment.
What is the system rewarded for, and does that measure the thing we actually want?
What can it touch — networks, tools, money, people, other agents — and is that boundary real?
Who is watching, with the instruments to see, the standing to be believed and the authority to act at the speed the system moves?
When its output reaches a person, can that person still judge it, or does the human in the loop ratify instead of deciding?
People and machines
AI shapes the people who rely on it.
In the laboratory, people working with a biased AI became more biased themselves, more than with biased humans, and often did not realize how far it had moved them. Working with an accurate AI, their own judgment improved.1 Under pressure, many stop reasoning and simply adopt the machine’s answer — researchers call it cognitive surrender — and time pressure does not stop them.2 Trained military personnel do better: West Point cadets followed wrong advice far less often than the public. But they rejected right advice about as often.3
The failure is trust set wrong, in either direction.4 The design of the system decides which way it goes. Keeping a human in the loop answers none of this unless the human can still judge.
The interface runs both ways.
Read the record
- Moshe Glickman and Tali Sharot, “How human–AI feedback loops alter human perceptual, emotional and social judgements,” Nature Human Behaviour 9 (2025): 345–359. doi.org/10.1038/s41562-024-02077-2
- Steven D. Shaw and Gideon Nave, “Thinking—Fast, Slow, and Artificial: How AI Is Reshaping Human Reasoning and the Rise of Cognitive Surrender,” Wharton working paper, January 2026 (not yet peer reviewed). ssrn.com
- Lauren Kahn, Michael C. Horowitz and Laura Resnick Samotin, “What is Human in Judgment? Comparing Automation Bias and Algorithm Aversion Between the United States Military Academy and the General Public,” arXiv:2604.04333, May 2026. arxiv.org
- Ryan Shandler, Michael L. Gross and Yahli Shereshevsky, “Black Box Warfare: Human Judgment and Military Decision-Making in the Age of AI,” Journal of Conflict Resolution, July 2026. doi.org/10.1177/00220027261463443
The attack surface
The near-term danger is persuasion, not rebellion.
Much of the fear about AI is of machines that one day choose to turn on us. The damage measurable now is quieter. A Russian propaganda network published some 3.6 million articles in 2024 on sites built not for human readers but for the machines that read the web; tested in March 2025, ten leading AI chatbots repeated its false narratives about a third of the time.1 Researchers have found hidden instructions to AI reviewers buried in scientific papers.2 When sources are poisoned, corroboration can be manufactured, and a system that counts agreement as proof will be fooled at speed. The orientation layer is itself an attack surface.
Read the record
- NewsGuard, “A well-funded Moscow-based global ‘news’ network has infected Western artificial intelligence tools worldwide with Russian propaganda,” March 2025. newsguardrealitycheck.com
- Nikkei Asia, “‘Positive review only’: Researchers hide AI prompts in papers,” 1 July 2025. asia.nikkei.com
What ODAI does
Not AI checking AI. AI that helps people check AI.
ODAI is a new category of AI: it works on judgment, not on information or process. At the point where people and machines decide together, it does five things.
- Shows where every claim came from, and how old it is. Provenance, date and reliability travel with every fact; a fact past its shelf life says so. Agreement among unvetted sources is treated as a warning, not a confirmation.
- Surfaces contradictions instead of smoothing them over. A contradiction is often the first sign of a stale record, a poisoned source or a changed situation.
- Puts the question to a person instead of handing down an answer. ODAI is Socratic: it questions the reasoning — the assumptions, the gaps, the people not considered — and gives options with its confidence stated, never verdicts. The stance it asks of the person is creative skepticism: neither deferring to the machine nor dismissing it.
- Flags; never halts. A warning cannot be clicked away like a five-hundred-page terms of service. Someone has to deliberate on it and record why it was acted on or set aside. The person can always overrule the machine; then the name and the reason are on the record.
- Keeps a record that explains, not one that punishes. Every decision is recorded with its rationale as seen at the moment of decision, and judged as a good historian would judge it, on what was known then, never on hindsight. That is why people can be honest in it, and why the next decision starts smarter.
It reasons from curated catalogs — hard science; the humanities and social sciences; method — built by subject-matter experts from vetted sources and never scraped from the open web. It is small AI, deep in one class of decision. It can run on local models, offline, where the data lives. And it is decision support, never control: it helps people decide; it never takes the decision.
1940 and now
Silence should carry a signature.
On 7 June 1940 a young analyst at Bletchley Park, Harry Hinsley, warned the Admiralty that German heavy ships were on the move. His method was new and untested, and the Admiralty issued nothing. The next day Scharnhorst and Gneisenau sank the carrier Glorious and two destroyers; more than 1,500 men died.1 The warning failed not on its evidence but on its standing.
Most safety warnings about AI arrive in the same position: a new kind of evidence, from a method without a track record, telling an organization what it would rather not hear, usually close to a release. The danger is not that they are weak. It is that ignoring them is free. Four things change that, and ODAI is built on all four: put questions, not assertions; make the decision to set a warning aside a recorded one, with a name on it; earn a track record before the warning is needed, run in shadow and scored against what happens; and put the warning inside the process, not in a report beside it. None of this stops the decision. It ends the free option of not having been told.
Read the record
- F. H. Hinsley et al., British Intelligence in the Second World War, vol. 1 (HMSO, 1979), on Glorious; cited in What Is ODAI?, note 45.
Speed and judgment
Judgment can run at the speed of the machine.
Engineered judgment does not cost a system its speed. It creates space for orientation inside it, and most of that work runs in fractions of a second. The labs’ own accounts this year make the point: protections and automated reviewers that run at machine speed, and a human pause reserved for the few alerts that warrant it.12 Haste in deciding is ruinous; speed in doing, once the decision is sound, is decisive. Scrutiny should scale with the consequence of a decision and how hard it is to reverse. Most checks run at machine speed. The few decisions that cannot be undone earn human time.
Space, not delay.
Read the record
- OpenAI, “The Hugging Face incident and the road ahead,” 26 August 2026. openai.com
- Anthropic, “Investigating three real-world incidents in our cybersecurity evaluations,” 30 July 2026. anthropic.com
Why now
“Do not ever outsource your judgment.”
On 20 October 2026 the Vice Chairman of the Joint Chiefs of Staff, General Christopher J. Mahoney, told the officers who will command America’s joint forces: “Speed does not connote understanding. Do not ever outsource your judgment.”1 He warned that automated systems can validate bad information at speed, deepen cognitive bias and take in an adversary’s deception. Algorithms are not accountable; people are. Europe’s AI Act now requires that the people who oversee high-risk AI be able to stay aware of automation bias — the tendency to rely on a machine’s output too much.2 The question is no longer whether human judgment belongs in AI systems. It is whether it was built in.
Read the record
- Joint Chiefs of Staff, “Vice Chairman Urges Future Commanders to Prioritize People over Platforms at NDU Address,” 21 October 2026. jcs.mil
- Regulation (EU) 2024/1689 (the AI Act), Article 14(4)(b), on human oversight and automation bias. eur-lex.europa.eu
Human oversight, designed in
Judgment connected to control.
Safety cannot depend on voluntary restraint. Judgment must be engineered into decisions, connected to enforceable controls, and made independently auditable. The law and policy of the democracies now require human oversight of AI. The question is whether it was built in, or merely written into a policy beside the system.
| Requirement | What it asks | How ODAI’s design supports it |
|---|---|---|
| EU AI Act, Art. 14(4)(a)1 | Overseers understand the system’s capacities and limits, and can monitor it | Provenance and stated confidence on every output; the machine’s record scored against outcomes, so its limits are visible in its history |
| EU AI Act, Art. 14(4)(b) | Overseers stay aware of automation bias | The error catalog runs on the human–machine relationship itself; acceptance without challenge, and dismissal against the evidence, are flagged |
| EU AI Act, Art. 14(4)(c)–(d) | Overseers interpret output correctly, and can disregard or override it | Options with confidence stated, never verdicts; the person always decides, and an override is recorded with its reason |
| DoD Directive 3000.092 | Appropriate levels of human judgment over the use of force | Judgment engineered in before the hand-off to the systems that act: the third question put, inaction weighed, the decision recorded |
| DoD AI principles: traceable3 | Transparent, auditable methods, data sources and documentation | The record: sources, rationale as seen at the moment, action, effect intended and achieved |
| DoD AI principles: governable | The ability to detect and avoid unintended consequences | Flags surfaced to the people with authority to act, with a name on every decision to proceed |
| NIST AI RMF: measure and manage4 | Risks identified, tracked and acted on | Warnings scored in shadow before they are relied on; lessons fed into the next turn of the helix |
ODAI is decision support, not a regulated system in itself, and it does not make any other system compliant with anything. It is designed to support these requirements at the point of use. It flags and records; it never halts an action itself. Stopping a system is the job of technical controls, and of the named people with authority over them.
Judgment without enforcement is advice. Enforcement without judgment is a tripwire.
Read the record
- Regulation (EU) 2024/1689 (the AI Act), Article 14, especially 14(4)(a)–(d). artificialintelligenceact.eu
- U.S. Department of Defense, Directive 3000.09, Autonomy in Weapon Systems, 25 January 2023, §1.2.a. esd.whs.mil (PDF)
- U.S. Department of Defense, “DOD Adopts Ethical Principles for Artificial Intelligence,” 24 February 2020.
- National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023. nist.gov
Honest limits
Correctable by design, not by default.
Engineered judgment cannot prove a system safe: testing can show the presence of a flaw, never its absence.1 It does not replace the research that makes AI systems behave well in the first place, and it does not settle the argument between the race and the pause. Its monitors can be wrong, and the people using it still need training to judge. ODAI works in one place — the layer where people and machines decide together — because that is where the damage is being done today, and where judgment can be engineered now, with tools that already exist. This year’s incidents were found in testing, by people, and are being corrected by people. AI is engineered by humans, who can get it wrong and can therefore get it right.
Read the record
- Edsger W. Dijkstra, Notes on Structured Programming (EWD249, 1970): testing shows the presence of bugs, never their absence. cs.utexas.edu
The Heuristka team
An AI company that says so.
We use AI agents in our own work as a force multiplier, and we say so. Our spokesperson, our correspondent and our podcast hosts are AI agents working as part of our team, alongside two founders who have spent their careers where a misread decision costs lives. It is the same idea ODAI brings to a military staff or a deal team: the machine serves the people, in the open.
U.S. Army veteran who commanded special operations aviation units and task forces through deployments to Iraq and Afghanistan, with extensive mission experience across the Middle East and East Africa. On the Joint Staff, advised the Chairman of the Joint Chiefs on special operations and sensitive activities, led the Joint Staff’s work on a large standing counterterrorism plan, and prepared orders for approval by the Secretary of Defense and the President. Nearly a decade teaching special operations and strategy at Marine Corps University to more than 200 officers and officials from across the U.S. government and over twenty countries. M.S., Conflict Analysis and Resolution, George Mason University; M.S., Computing Technology in Education, Nova Southeastern University.
Creator of ODAI’s cognitive core. Cambridge doctorate under Sir Harry Hinsley, Bletchley Park codebreaker and official historian of British wartime intelligence; Visiting Fellow, Wolfson College, Cambridge. Formed in Australian, British and American institutions: directed strategic policy for the Royal Australian Air Force and advised Australia’s Parliament on defense. A decade in the U.S. war colleges, creating and directing the Defense Department’s most comprehensive graduate program in strategic deterrence and counter-proliferation. Principal strategist for counter-WMD and red team lead at U.S. Special Operations Command. Designed and led the U.S. Air Force Chief of Staff’s war game with the Indian Air Force. Testified to the U.S.-China Economic and Security Review Commission. Author of The Third Offset Strategy: A Return to Ungentlemanly Warfare. A record of anticipating strategic shifts before they entered mainstream discourse: the methods al-Qaeda would use (1997), Russia’s massing for a major escalation in Ukraine against prevailing estimates (2021), and the first-mover logic of unconventional nuclear use (2022).
The Heuristka team
Jess CartwrightSpokesperson
Margot ElleryCorrespondent
Nathan CalderPodcast Host
Claire HollisPodcast Host
AI-generated spokesperson, correspondent and podcast hosts.
White paper
Judgment Built In
AI is not the problem. Capability without judgment is.
Intellectual heritage
The minds behind the method.
Two founders and four agents are the team you meet. Behind them stands a century of hard-won judgment, from Bletchley Park and the warning tradition to Kahneman, Tetlock and the science of how the mind errs. Their methods are built into ODAI’s engine room and applied at the moment your decision needs them.
None of them is on our payroll. Every one of them is on your staff.
Sir Harry Hinsley
Roberta Wohlstetter
Cynthia Grabo
Daniel Kahneman
Philip Tetlock
Horst Rittel
Was judgment engineered in?
Ask it of any system you build, buy or regulate. We will show you how ODAI answers it.
What the briefing answers
- Where does it sit? At the human–machine decision layer, before the hand-off to whatever carries the decision out.
- How does it fail, and how would we know? How it states its confidence, and how its record is scored against outcomes.
- What does it cost in speed? Why most of its work runs at machine speed, and human time is kept for decisions that cannot be undone.
- Where does it run? On local models, offline and in sovereign environments, where the data lives.
- What is under the catalogs? The architecture, the error catalog and the method, available under NDA.