The Admiralty Code for Public Discourse

The Trouble With "Is This a Good Source?"

Most advice about evaluating information is mood dressed as method. Trust this outlet. Be wary of that one. Check whether it sounds biased. The advice collapses two entirely different questions into a single feeling about a brand, and the collapse is the error — because the two questions are independent, and reasoning begins only when you refuse to merge them.

The first question is: how reliable is this source? The second is: how credible is this specific claim? These do not move together. A completely reliable source can carry an improbable claim — a peer-reviewed journal can publish a result that later fails to replicate. An unreliable outlet can happen to report something true — a tabloid can get a story right. If your evaluation produces one verdict, "good source" or "bad source," you have already lost the information that does the work. The instrument intelligence analysts use does not collapse them. It scores them on two axes, separately, and forces you to justify each in writing. It is teachable in an afternoon, and once you have it you cannot go back to the feeling.

Two Axes, Never One

The first axis grades the source, on a scale from completely reliable to unable-to-be-judged. At the top sits material whose authenticity is not in question: primary documents, official records, verified recordings, peer-reviewed research. Below that, sources usually reliable with a strong track record — established reporting under a named correspondent, subject-matter experts inside their specialty. Then sources only fairly reliable, with mixed records; then sources not usually reliable — partisan outlets, known bias patterns, anonymous sourcing without a verified history; then the actively unreliable; and finally a distinct grade for cannot be judged: a new source, unknown provenance, a leak with no verification infrastructure. That last grade matters because it is not a low score. It is the honest refusal to issue one, and the discipline depends on keeping "bad" and "unknown" apart.

The second axis grades the claim, independent of who carried it: confirmed, meaning independently corroborated by a separate reliable source; probably true, consistent with what else is known; possibly true; doubtful; improbable, contradicted by other information; and again a separate grade for cannot be judged. A source rated highly and a claim rated highly are different facts about the world. You write them as a pair — a letter and a digit — and you never let one stand in for the other.

The pairing is where the instrument earns its keep, because the same source carries different scores for different claims. A court filing is completely reliable evidence of what was filed and a far weaker source for what actually happened outside the courtroom, because it is one party's account. A news report is reliable for the government announced X and weaker for X is true. A leaked memo, once authenticated, is strong evidence that the memo exists and says this and says nothing on its own about whether what it says is correct. The score is not a property of the outlet. It is a property of the outlet with respect to the exact thing it is being asked to support — and you re-score every time the thing changes.

The Floor

An instrument that only describes is not yet a discipline. The discipline is the rule you attach to it: a line below which a claim is not allowed to stand alone.

The rule used here is explicit. No factual claim is published if its only support is a weak source carrying a weakly credible claim. A marginal claim survives only if a stronger independent source corroborates it. A claim that cannot clear the line does not get softened with a confident verb and waved through. It is reclassified as interpretation, removed, or held until it can be verified — and the work stops until that decision is made, rather than after publication when stopping is no longer possible. A floor that bends under the weight of a conclusion you want is not a floor. The entire value is that it does not move.

The Mistakes the Instrument Exists to Catch

The two axes are simple. The reason they have to be enforced is that the mind, unaided, makes the same handful of errors every time, and each error feels like rigour while you are committing it.

It anchors to brand. A famous outlet's individual article is treated as authoritative because of the masthead, when the masthead is not what you are scoring — the specific piece is. It conflates expertise with reliability: an expert speaking outside their specialty is given the authority they earned inside it. It launders aggregators — a source citing a source citing a source is scored as though the last link were the evidence, when the only honest move is to trace to the original and score that. And the most corrosive one: silent score inflation under pressure. When a conclusion you favour rests on weak sourcing, the temptation is to grade the source generously, because the alternative is giving up the conclusion. The instrument's most important function is to make that move visible — to you first. The score reflects the source, not the result you were hoping for. The moment it reflects the result, you are no longer analysing; you are decorating a decision already made.

Why This Is Being Handed Over

This is one of the instruments this publication uses on itself, published here so it can be used on this publication. That is the point of putting it in your hands rather than describing that it exists.

It does not tell you what to believe. It refuses to. It gives you a procedure for holding two questions apart that everything in the information environment is built to merge for you, and a floor that makes "I don't yet know" a precise position instead of a discomfort. Run it on the next thing you are asked to accept — and then run it on this essay. If a claim here rests on a source you would not grade above the floor, that is not a betrayal of the instrument. It is the instrument working, in exactly the hands it was meant for.