XLNCXLNC Book a 20-minute scoping call
Menu

Concepts

Reference definitions for the measurement model behind AIM. Each construct has a stable anchor; link to the anchor, not the page.

Rasch conjoint measurement#

Anchor: `#rasch-pcm-conjoint-measurement`

AIM scales are Rasch-family measurement models: the dichotomous Rasch model where items are scored right or wrong, and the form built for answers that can earn part credit, where items have ordered performance categories. Conjoint measurement means performer locations and item difficulties are estimated on one shared logit scale, so a performer's location means the same thing regardless of which subset of items they answered.

Properties that follow, and that benchmarks do not have:

  • Interval scale. A one-logit difference is a constant odds ratio everywhere on the ruler, so distances and averages are meaningful.
  • Specific objectivity. Comparisons between performers do not depend on which items were administered, and comparisons between items do not depend on which performers answered them.
  • Stated uncertainty. Every estimated location carries a standard error computed at that location, not a scale-wide average.

The method is published: Chapter 3, Models, Measurement, and Metrology Extending the SI (Barney and Barney 2024, De Gruyter), open access at https://library.oapen.org/bitstream/handle/20.500.12657/94377/1/9783111036496.pdf#page=114 ; the unobtrusive measurement approach is documented in Chapter 18, Cambridge Handbook of Technology and Employee Behavior (Barney 2019); and the 2016 Annual Review paper covers adaptive measurement. See Scientific references.

Stated standard error per band#

Anchor: `#stated-standard-error-per-band`

Standard errors are reported per band, never as one scale-wide average. Averages hide the floor: an instrument can be tight in the middle of its range and loose at the decision point, and a single mean SE would report it as fine.

Two worked examples from the current instrument set:

  • Canonical MHC ruler: set to an error bar of 0.20, delivered at 0.21 once judge harshness is corrected at each level, in a simulation of 15,000 decisions on judge data from our own runs, seed 20260908. With harshness left in, the same run came in at 1.16. Error bars in logits. The 2026-08-22 gate on one judge's harshness is kept as a historical record on the Evidence page. For the newer scales the design target is SE 0.10 per band (band 1 SE 0.15); its unit and vintage are declared with the first calibration run.
  • Hallucination scale (two-layer cumulative construct, in calibration): interior bands carry a stated design precision of SE 0.10 per band; the first band carries SE 0.15. Both figures are design values, not yet certified, and are labeled that way everywhere they render.

Honest-uncertainty display rule, binding on every AIM surface: rows that fail the floor or lack evidence are shown and flagged, never removed.

Certification tiers#

Anchor: `#certification-tiers`

Every scale and every claim carries one of three tiers, rendered from a shared lexicon so the marketing site and this docs surface cannot drift:

  • Canonical: frozen bank, audit-ready.
  • In calibration: real numbers, evidence accumulating; item locations are candidates until a calibration run passes its gate.
  • Watch: provisional; decision-use prohibited for high-stakes calls.

A tier is stated as of a vintage date. Tier changes are events: promotion and demotion are recorded with the evidence that drove them.

The two-layer certification doctrine#

Anchor: `#two-layer-certification-doctrine`

Two-layer cumulative scales (the hallucination scale is the current instance) are gated in two layers, because the floor band and the interior bands carry different precision:

  • Interior bands (band 2 and above): stated design precision SE 0.10 per band.
  • Floor band (band 1): stated design precision SE 0.15.

Both figures are stated design precision, not yet certified. The doctrine exists so that a looser floor band is declared up front rather than averaged away. Where these figures render on the site, they carry the words "not yet certified."

Renderer honesty#

Anchor: `#renderer-honesty`

Wherever a scale or model is listed, failing states render. Below-floor rows render with muted ink on a warm wash; no-evidence rows render flagged. Dropping a failing row is treated as a defect class, not a style choice. This rule is enforced in the site generator and checked on every rebuild.