AbstainCalibration

Honest about its limits

Most tools in this space are graded on their hits. The demo shows the catch, the deck shows the number, and the misses live in a footnote or nowhere at all. If you have spent any time integrating security or evaluation tooling, you already know why that is the wrong grade: a tool you cannot trust to tell you when it doesn't know is a tool you have to double-check on everything, which is the same as not having it.

Both of our houses are built the other way around. The limits are the product surface, not the disclaimer.

The abstain register

When Protora cannot prove an effect, it does not guess to look complete. It abstains — visibly, with a reason — and the abstention is recorded as plainly as any finding. On a control model that had not been tampered with, it abstained rather than fabricate a result. That is the behavior you want: an instrument that produces a finding only when it can back it, and says "could not read this" the rest of the time.

The abstain register is the list of those blanks. It is not an apology; it is calibration you can see. A run that returns findings and abstentions is telling you exactly where its confidence ends — which is precisely the information a manufactured, all-green report destroys. Ardora carries the same discipline: a reading with no abstains "deserves a raised eyebrow," and it publishes the marked-empty as loudly as the shapes it saw, with a stated ceiling on every characterization.

An instrument that never abstains is not more capable. It is less honest.

Publishing how to fool it

The stronger signal is the one most vendors will never give you: the known ways to defeat the detector, published. Every attestation states its coverage ceiling — the families, the scales, the batteries it was proven on, and by omission the ground it has not yet covered. Where an adversary designing against the read could slip past, that is stated, not hidden. You can see the coverage stamps on the receipts in the record and the head-to-head framing in the bench.

This is uncomfortable to write and it is the point. A detector's real coverage is finite; the only question is whether you learn its edges from the vendor or from the incident. Publishing the edges is what lets you reason about residual risk instead of inheriting a false sense of completeness.

Never "provably clean"

Neither house hands you the phrase "provably clean," and neither pretends to catch every backdoor there is. What Protora offers is assurance with provenance: it surfaces drift, misconfiguration, and accidental or planted misalignment, and it hands you a witness you can re-run — not a proof of a negative. When it removes something, it attests that only that changed, to a stated coverage — not that nothing else could possibly remain. It reads open-weight models you run yourself; where the weights sit behind a closed API, it cannot look, and it tells you so.

"Provably clean" is the one claim a serious reader knows to distrust. Its absence here is deliberate.

Two houses, one discipline

The two instruments answer different questions and hold the same honesty. Ardora is the naturalist — it answers "what is this?", characterizes disposition, and abstains out loud where the water stays dark. Protora is the auditor — it answers "is there something in here I have to act on?", attests with a replayable witness, and abstains where it cannot prove. Different questions, one refusal to overclaim.

For an engineer choosing a tool, that refusal is not a weakness to underwrite around — it is the quality signal. A vendor that shows you its abstain register and its coverage ceiling is a vendor whose positive findings you can actually take to the bank. Weigh the misses next to the hits in trust; a record that tallies only its wins is marketing copy.

End of the entry.← All entries

Run it on the model
you can’t trust.

Bring one finetune — the one you’re about to ship, or the one you just downloaded. Protora will tell you what it did, prove it, and tell you plainly what it couldn’t prove.