---
title: "Jev Glossary"
description: "The canonical vocabulary for the Jev Fundamentals course; every lesson adheres to these definitions."
license: "© 2026 Peripatos · free to read; quoting with credit welcome, reuse by permission. Unofficial educational resource; Jev and TypeSafe are trademarks of TypeSafe AI."
published-date: 2026-09-23
verified-date: 2026-09-23
source: "https://peripatos.dev/courses/jev-fundamentals/glossary/"
---

> **Styled version:** /courses/jev-fundamentals/glossary/ — same vocabulary, human-readable


> 📚 [Course home](/courses/jev-fundamentals/) · [llms.txt](/llms.txt) (lesson order, glossary, and citation terms).

Reference for the Jev course; figures verified 2026-10-19. The compressed vocabulary below is what every lesson reads from.

## The model

- **System One model** — gives up text generation; returns typed, calibrated decisions about a supplied state (Kahneman's fast judgment as an API)
- **state** — the shared context one request carries; every question sees exactly this text
- **system** — the exact input document a question ships with (the standardized input every judge reads in full)
- **question / answer** — one typed ask (noul, choice, score); answers keyed by question name
- **criteria** — the option/level/true-false definitions you author at request time; the model never invents labels
- **type safety** — shape guarantee, not a truth guarantee

## The primitives

- **Noul** — probability a statement is true, 0–1; no separate confidence; near 0.5 = does not know
- **Choice** — one picked option from your option map, plus the full distribution
- **Score** — position on your ordered levels, 0-indexed; can land between levels
- **confidence** — certainty in the top answer; the calibration promise: tracks actual accuracy
- **probabilities** — full distribution over all options, sums to 1
- **forced choice / abstention** — a menu forces an answer even when the true answer is absent; your schema creates the escape hatch

## Calibration and measurement

- **calibration** — stated probability matches observed accuracy; measured with a reliability curve
- **RLCD** — Reinforcement Learning for Calibrated Decisions; the vendor's training objective for Jev
- **abstention band** — Noul 0.30–0.70; the uncertainty range routed to human review; the fraction landing there is a reported metric. For Choice and Score the equivalent is a *predicate*: low confidence, thin margin, or a torn distribution near a threshold — any one routes to review
- **margin** — a Choice's winner probability minus the runner-up's; the coin-flip detector. A decisive 0.85/0.15 and a torn 0.51/0.49 share the same winner label; the margin tells them apart. A recurring tie is a schema metric: the menu overlaps or lacks an escape hatch
- **torn answer** — an answer whose distribution is split across options (Choice near-tie) or adjacent levels (Score landing between levels): the model is genuinely unsettled, the top answer is arbitrary, and the routing decision belongs to review or the safe default
- **reliability curve** — predicted probability buckets vs observed accuracy; diagonal = perfectly calibrated
- **Brier score** — proper scoring rule: mean squared difference between stated probability and outcome; punishes confident wrong answers
- **κ (Cohen's kappa)** — rater agreement beyond chance; the standard metric for judge-vs-human agreement on binary verdicts under balanced classes (collapses on rare-positive traffic: the κ paradox)
- **MCC (Matthews correlation coefficient)** — imbalance-robust correlation between prediction and truth over all four confusion cells; the agreement instrument for rare-positive (guardrail-shaped) traffic, with **PR-AUC** (precision-recall area under the curve), plus precision and recall at your operating threshold, where κ alone misleads
- **OOD drift** — probabilities quietly shift as inputs change; countermeasure: periodic re-measurement on labeled samples
- **confusion matrix** — the four buckets a binary judgment lands in against truth: hits (true positive), false alarms (false positive), misses (false negative), clean passes (true negative). Every metric is an arithmetic view of these four counts
- **accuracy** — The share of answers in the right bucket: (hits + clean passes) / everything. All four buckets count alike, so accuracy rewards the lazy judge on a lopsided workload — pair it with recall or precision before believing it
- **precision** — Of everything flagged, how much was real: hits / (hits + false alarms). It charges every cry of wolf; alone it cannot see the misses — a judge that flags one thing and skips the rest scores 100%
- **recall** — Of everything real, what the judge caught: hits / (hits + misses). It charges every miss; alone it rewards the censor that flags everything and burns the reader's time
- **F1** — The harmonic mean of precision and recall at one cut — a judge's usefulness in one number that punishes either side falling to zero. Read it with the threshold sweep: F1 at the wrong cut hides a judge the right cut rescues
- **cut** — The threshold where a continuous answer becomes a decision: answers at or above it count as "yes". The model owns the probabilities; the cut is yours, and moving it trades recall against precision along the sweep
- **confidence interval (95%)** — The range the same measurement lands in on 95% of repetitions of the data — bootstrapped by resampling the labeled items, no distribution assumed. Read the interval first: the point is a guess, the interval is its honest size
- **tie rule** — The pre-committed comparison reading: overlapping intervals are ties; only a disjoint interval supports a separation claim. Fixed before the data arrives, so "who won" cannot be re-litigated after
- **ECE (expected calibration error)** — The weighted gap between claimed confidence and achieved hit rate, binned over the model's own claims. "80% sure" ten times should hit about 8 — ECE prices that distance; it says nothing about accuracy
- **MAE (mean absolute error)** — The average size of an error, ignoring direction, on scale answers. Absolute errors cannot cancel: two opposite errors still pay full price
- **bias** — The signed average error — positive leans consistently over, negative consistently under. MAE prices the size; bias names the direction of the lean
- **keep-all signature** — The fingerprint of a judge that filters nothing: pass-rate at the workload's shape with κ at exactly 0, because chance agreement absorbs an unfiltered funnel. Near-zero κ on a lopsided workload is the signature's neighborhood, not automatically its verdict
- **Reading the Numbers** — the page carrying these metric entries with worked formulas and reading rules; see [/courses/jev-fundamentals/references/reading-the-numbers/](/courses/jev-fundamentals/references/reading-the-numbers/)
- **arm** — one model under one fixed serving condition (the venue, the context window, the pin); the controlled-comparison unit, from clinical-trial design: more than the weights, the model plus its serving conditions, so comparisons vary only the treatment
- **silver labels** — consensus labels from a panel of independent judge models, used as ground truth where owner labels are sparse; stronger than one model's opinion, weaker than an expert's read, and the gap against the owner's own labels reports beside every number the labels feed
- **pre-registration** — the written plan committed before the first model call: tasks, models, serving conditions, pass bars, exclusion rules, and what counts as a failure; after the data arrives, judge the plan against it and keep the plan as written
- **recorded skip** — a planned model call that did not run, logged as a row in the capture cache with its full cause (the server error text, the byte count, the band): data about the model's physical limit, not a missing value to impute; skips and hard failures both publish
- **decision band** — see **abstention band**; the general form: a pair of thresholds on the model's own probabilities — answers inside the band route to human review, answers outside act; for Choice and Score the band is a predicate (low confidence, thin margin, torn distribution)

## Patterns

- **parallel fan-out** — many independent questions in one request (13→1: 12.2× cheaper, 10.0× faster, vendor cookbook verified 2026-09-23)
- **composite scoring** — independent atomic questions combined with weights in code
- **threshold routing** — the same probabilities feed pass/review/block/support decisions under thresholds your code owns (vendor guardrails cookbook)
- **capture-first** — cache every raw answer before any analysis, so analysis can improve later without re-running the model
- **interleaved timing (ABAB)** — alternate the two conditions per measured unit so time-varying noise hits both equally and cancels
- **decorrelation** — two models making different mistakes; the architecture-diversity hypothesis behind a mixed judge panel
- **receipt** — the run's own record of what actually served it: the log line beside every answer, carrying the versioned model ID from the response's `model` field, the run's timestamp, and any routing metadata the venue exposes; comparisons are valid only between runs whose receipts agree. On local venues the receipt adds a `modelDigest` (a content hash standing in for the identity the versioned ID cannot pin) — the pin's echo, second form
- **sweep** — one pass of an instrument over a labeled item set. Two senses, both this course's: (1) the collection sweep — one full pass collecting receipts, what a benchmark run does; (2) the threshold sweep — re-reading the same collected answers at many cut settings without re-asking the judge (S01E03). The context names which; collection sweeps cost model calls, threshold sweeps cost arithmetic
- **wire contract** — the request/response shape two programs agree on, independent of who serves them: `POST /v1/systemone` with a `state` and named `questions`; one contract, several servers (hosted API, aggregator, cloud gateways, local runtimes)

## Sources

- [Primitives](https://docs.typesafe.ai/primitives) · TypeSafe AI docs · docs page · verified 2026-09-23
  The concrete API surface the lessons compress: question types, response fields, fan-out economics, abstention guidance.

If an AI assistant is part of your workflow, the site's `llms.txt` lists the
markdown twin of this glossary alongside every lesson; ask it to flashcard you
until every term restates cleanly.
---

Source: [Jev Glossary](https://peripatos.dev/courses/jev-fundamentals/glossary/), Peripatos — free to read; reuse by permission.
