How we test, and what we will not claim.

Obligation has levels, and collapsing them misleads people.

This is the most common error in trade coverage of AI regulation. A bulletin that "expects" becomes a headline that says "requires," and an agency principal makes a decision on the strength of it. Every statement about obligation here carries one of these.

Legal duty

A statute or adopted regulation, cited by section, binding on a stated party.

Regulator expectation

A bulletin or circular letter. We preserve the modal verb rather than paraphrasing it into something stronger.

Recommendation

Defense counsel, E&O loss prevention, trade association guidance. Worth following, not enforceable as such.

Allegation

A pleading. Never reported as a finding, and never quoted for its damages figure as though established.

Authored here

Procedures and thresholds we wrote ourselves, labeled so nobody mistakes our judgment for a requirement.

What we do before publishing a number.

1. Ask the vendor what the number counts.Before testing a published accuracy claim we ask for the definition, the denominator, whether human corrections are included, and what population was measured. We publish the answer, or publish that there wasn't one.
2. Build ground truth first.Two qualified reviewers assign expected values independently, disagreements adjudicated by a third, all before anyone sees the tool's output.
3. Freeze the test set.Documents are authorized and fixed before the run, so nobody can swap in easier cases after seeing results.
4. Count omissions as errors.The denominator is every expected field, not only the fields the tool attempted. Abstention does not inflate accuracy.
5. Separate critical errors.Any defect that would change what a client believes is covered, what it costs, or who is insured is reported separately and never netted against minor ones.
6. Publish the run conditions.Tool version, settings, retries, manual cleanup, document mix, scan quality. A result without conditions is not reproducible.

Guides and working resources contain authored recommendations informed by the sources linked in each page. They are not results from our own tests, legal opinions, or guarantees of a product's performance. Synthetic examples are labelled and must not be presented as observed results. Each carries the date its sources were last reviewed and is re-read on a ninety-day cycle, or sooner when a source changes.

Every finding and test carries the date it was last checked against its sources, not the date it first appeared. When a source changes (a bulletin is updated, a docket moves, a form is re-issued) the piece is revised and the date moves. A finding that has not been re-checked in a year says so by its date.

Social pieces carry the date they ran and a desk page here with their sources; they are not re-reviewed. The durable part (the form, the bulletin, the order) comes back as a finding when it has been read in full, and findings are.

No sponsored content, no paid placements, no affiliate arrangements with any vendor covered here. We do not accept free service in exchange for coverage. Where a vendor offers a trial available to anyone, we take it on the same terms as any reader and disclose that we did.

We do not repeat a figure we cannot trace to a primary source. Where a number circulates widely but originates in a vendor's blog with no supporting research, we say that instead of citing it, including when the number is one we published ourselves and later could not stand up.