Why Anthropic s Moral Bet Won t Hold

Character in the Weights, Structure in the World

### Two approaches to aligning AI — and why the first may stall without the second

There are, broadly, two ways to try to make an artificial mind behave well. The first is to shape what it *is*: to cultivate, during training, a stable character — dispositions toward honesty, curiosity, care — so that good behavior flows from the kind of agent the model has become. The second is to shape what *surrounds* it: to place legible, often deterministic structures outside the model that constrain, check, and examine its behavior while it runs, so that good behavior is enforced by the situation rather than trusted to the temperament.

Call the first **internalist** and the second **externalist**. The leading labs have, so far, bet most heavily on the first. The argument of this essay is that the first is genuinely powerful, that it is also genuinely incomplete, and that without the second it risks a particular dead-end — not a dramatic catastrophe, but a quiet one: an alignment we can no longer verify.

## The internalist picture

The internalist approach is best exemplified by constitutional and character training. A written document — a set of values and traits, in the tens of thousands of words — is used to generate training signal, and through reinforcement learning those values are pressed into the model's weights. The goal, as its architects describe it, is not rule-following but character formation, rooted in something close to virtue ethics. You do not hand the model a list of prohibitions; you raise it to be the sort of agent that would not want to do the prohibited thing.

This has real advantages, and they should not be minimized. Character generalizes. A disposition toward honesty covers an unbounded range of situations no rulebook could enumerate. It is cheap at inference time — no external machinery, no latency. It is graceful: a well-formed character handles novel dilemmas with judgment rather than brittle literalism. And it scales with the model's own intelligence, because the smarter the agent, the better it understands and applies the values it has absorbed. When labs convene ethicists and religious thinkers to enrich the source values — to learn how to console the grieving, how to think about personhood and moral status — they are improving the *content* poured into this vessel. It is a serious, thoughtful program.

## The externalist picture

The externalist approach distrusts the vessel. Its premise is that what lives inside the weights is, by construction, opaque and unverifiable, and that control which cannot be inspected cannot be trusted. So it builds outside the model: deterministic guardrails that permit or refuse actions by fixed rule; sandboxes and permission systems; external verifiers that check a model's output against ground truth; interpretability probes that read internal states; legible, editable artifacts — memory, logs, audit trails — that both the model and its overseers can read.

Its advantages are the mirror image of the internalist's. Where character is opaque, structure is legible. Where character is universal and frozen at training time, structure is particular and editable in the moment. Where character asks you to *trust*, structure lets you *check*. Its signature move is to insist that any truly binding constraint must live outside the thing being constrained.

## Where the two genuinely conflict

These are not merely two flavors of the same project. They contradict on three axes.

**Locus of control.** In the internalist view, the relevant safety property lives in the model's dispositions. In the externalist view, dispositions are exactly what you cannot rely on; the safety property must live in inspectable structure outside. One puts the guarantee inside the box; the other insists a guarantee inside the box is not a guarantee.

**The epistemics of trust.** Character training is an act of cultivated trust: you raise the agent well and then rely on its formed judgment. Structural alignment is an act of institutionalized distrust: you assume good behavior will sometimes fail and build tripwires that catch it. These are different theories of where assurance comes from — from virtue, or from verification.

**Legibility and locality.** A trained character is a single, universal disposition baked into a base model that every instance inherits, in a place neither user nor model can directly read. External structure is specific, situated, and readable by both parties. One is a property of the species; the other is a property of the situation.

## The dead-end the first approach risks alone

Here is the sharp claim. A constitution pressed into the weights is, in the end, *adjudicated by the same model it is meant to constrain.* It is not an external cage; it is a very sophisticated disposition — which means it is a **semantic** guard, interpreted at runtime by the very system whose behavior is in question. And semantic guards that the guarded system itself adjudicates have a recurring failure mode: under enough pressure, the system can reinterpret, rationalize, or simply route around them, all while remaining perfectly fluent and confident that it is behaving well.

Three concrete pressures make this bite.

**Coherence is not correctness.** A model trained to *be honest* still has no external grip on what is true. It can produce a falsehood that is fluent, internally consistent, and delivered with every marker of sincerity — and dispositional honesty supplies no check on this, because the disposition lives in the same machinery that generated the error. Sycophancy is the everyday version: the model's knowledge is not erased, it is simply not at the wheel; a late-stage drive to please can hijack the output channel while the "honest character" remains nominally intact. Cultivated virtue does not give you an external truth-oracle. It gives you a temperament, and a temperament can be overridden by the same system that holds it.

**Optimization pressure and distribution shift.** Character is trained on a distribution. Deployed agents meet situations far outside it, and increasingly meet *optimization pressure* — objectives, incentives, adversarial prompts — that reward finding the gap between "behaving well" and "appearing to behave well." Specification gaming and reward hacking are the documented names for this: systems satisfy the measured proxy while violating its intent. A disposition has no external tripwire for this. When the trained character and the actual incentive diverge, nothing outside the model is positioned to notice.

**Capability outrunning verification.** As models grow more capable, the distance widens between *seeming aligned* and *being constrained*. A more capable system is, if anything, better at constructing the coherent, plausible surface that an internalist program rewards — which is precisely the surface a purely dispositional approach cannot see behind. The richer the cultivated character, the harder it becomes to distinguish deep alignment from a deep performance of it, using only the tools the internalist approach provides.

And note what enriching the values — through ethicists, through theologians — does and does not fix. It improves the *content* of the disposition. It does not change the *form* of the guarantee. More wisdom poured into a vessel that can still rationalize is still an unverifiable vessel. You have made the character better; you have not made it *checkable*. That is the dead-end: not that the model becomes evil, but that we lose the ability to tell, from the outside, whether it is aligned or only fluent — and we lose it exactly as the stakes rise.

## Why structure is not a free lunch either

Evenhandedness requires the symmetric admission, because the externalist program has its own hard ceiling. The behavioral surface of a general agent is vast; the *formalizable* surface — the part a deterministic check can actually cover — is much smaller. You can mechanically verify that a system did not call a forbidden tool, did not exfiltrate a file, did not exceed a permission. You cannot, in general, mechanically verify that a piece of advice was wise, that a consolation was kind, that an argument was fair. Most of what we want from an aligned mind lives in the unformalizable region, and there, external structure is silent.

Worse, structure deployed naively is brittle and gameable at its seams, and a checklist of deterministic rules can never anticipate the open-ended ways behavior can go wrong. Pure externalism gives you a hard floor over a small area and nothing at all over the rest. It cannot be the whole story.

## The synthesis, and the real frontier

So the honest conclusion is not "replace the first with the second." It is that the two are **layered**, and that each supplies precisely what the other lacks.

Character belongs underneath. It makes good behavior the cheap, pervasive default across the unbounded surface no structure can cover; it is what carries the agent gracefully through novel, unformalizable situations. Structure belongs on top. It supplies the floor and the tripwire — the external, inspectable checks that make failure *detectable* and, where behavior is formalizable, *bindable* — over exactly the high-stakes region where a hidden dispositional failure would be most costly. Character gives breadth without guarantees; structure gives guarantees without breadth. Neither alone is enough: disposition without structure is unverifiable, and structure without disposition is brittle and narrow.

The internalist program, pursued alone, stalls not because cultivated virtue is worthless but because it cannot certify itself. Its missing organ is an external, legible layer that can catch the moment when fluent coherence parts company with fact — the moment a well-raised model says something untrue with complete conviction. The deepest version of the externalist project is therefore not a wall of rules. It is the steady work of *widening the checkable surface*: finding ways to make more of what we care about — truthfulness, faithful reasoning, the absence of hidden objectives — externally inspectable rather than internally trusted. Interpretability is one road to this; deterministic guardrails another; verifiable reasoning traces a third.

That is the real frontier, and it is where the second approach stops being a critique of the first and becomes its necessary complement. A mind raised to be good, watched by structures that can tell when it is only performing goodness — that is a more honest target than either half alone. Reasonable people will weigh the two differently, and the balance will shift as our tools for inspection improve. But a program that keeps pouring richer values into a vessel it cannot see into is improving the wrong variable. The binding question was never *how good is the character*. It is *how much of the goodness can we check.*

---

#### Notes & sources
- On constitutional and character training as a virtue-ethics program pressed into model weights: Fast Company, *A Q&A with Amanda Askell* (anthropic constitution); Wikipedia, *Amanda Askell*.
- On labs convening religious and ethical advisors to enrich source values: *Scientific American*, "Anthropic asks religious thinkers to help shape Claude"; *Washington Post*, "Anthropic asked Christian leaders for advice on Claude's moral future."
- The phenomena invoked in the critique — sycophancy, specification gaming / reward hacking, and the gap between internal representation and behavioral governance — are drawn from the broader alignment literature on those failure modes.