06 Taxonomy
Design

Design 06: M4 (Taxonomy, the Third Domain)

Status: accepted 2026-10-08 (all eight decisions as recommended). Builds on: designs 01 to 05 and M3 as built (commit 0f0d224).


1. Why taxonomy

Organizations and people are the domains Thisness was designed around, and every benchmark so far is synthetic: generated by code written alongside the resolver it tests. A third domain should be unlike the first two and should come with ground truth nobody here made up. Scientific names of plants are both.

  • Real ground truth. World Flora Online (WFO) publishes its taxonomic backbone as public domain (CC0 1.0): every name, its status (accepted or synonym), the accepted name it resolves to (acceptedNameUsageID) and its basionym (originalNameUsageID), as Darwin Core archives, one per family (721 of them) or one for everything. Which names refer to one taxon is curated by taxonomists, not generated.
  • Every part of the engine is exercised. Misspellings and author abbreviations exercise normalizers; names that move between genera exercise the ledger's renames, merges and splits; sister species and genus membership exercise "related, not the same"; vernacular names across languages exercise multi-valued, multi-source observations; herbarium labels and trade lists that carry only a name exercise the name-only plan.
  • It is the case the project exists for. The peacock flower has had more names than one: "Red flowered fence plant", "Barbados Pride", "Flower Fence", "Paradise Flower"; Linnaeus named it Poinciana pulcherrima in 1753; Swartz moved it to Caesalpinia in 1791 as Caesalpinia pulcherrima (L.) Sw. Linnaeus's system kept one name per plant and let the others go. Thisness keeps every observation with its source, holds the canonical label apart from identity, and shows what supports each value. A domain where that design visibly matters is the right test of it.

2. What is an observation, and what is an entity

An observation is a name as one source used it: a herbarium label ("C. pulcherrima"), a trade list ("Barbados Pride"), a checklist row ("Poinciana pulcherrima L."), a field record, or a row of a taxonomic backbone. Like any observation it is a claim from a source, not a truth.

An entity is a taxon: a group of organisms that one taxonomic treatment recognizes as one species (or subspecies, or variety). Two facts of nomenclature make this harder than it sounds:

  • Homotypic synonyms share a type specimen: Poinciana pulcherrima L. and Caesalpinia pulcherrima (L.) Sw. are the same name moved to another genus. The parenthetical "(L.)" records the original author. That they refer to one taxon is a nomenclatural fact.
  • Heterotypic synonyms have different types, and treating them as one taxon is a taxonomic opinion: one authority lumps two described species, another keeps them apart. Two authorities can disagree, and both can be competent.

So "the same entity" depends on whose treatment is followed, which is the authority question the book chapter raises about Linnaeus. Section 3 is how Thisness answers it without hiding it.

3. Authorities: opinions with sources

Assertions are observations. A row of a backbone ("Poinciana pulcherrima L., synonym, accepted name Caesalpinia pulcherrima (L.) Sw., according to WFO 2024-12") enters as an ordinary observation from source wfo, with two extra attributes: the asserted accepted name and the asserted status. A comparison over those attributes ("this observation's asserted accepted name is that observation's name") is evidence like any other: typed, weighted, cited, explainable. No new engine concept is needed, and a second authority (Catalogue of Life) is simply another source whose rows are observations too.

Disagreement stays visible. When WFO treats two names as one taxon and Catalogue of Life keeps them apart, the pair carries supporting evidence from one source and contradicting evidence from the other: a contested edge, a review or a relationship candidate, never a silent choice. The dossier shows which authority said what.

The canonical label is a stated choice (decision 4). Today's canonicalizer picks each attribute's most frequent value, which for taxa would make a popular outdated name canonical. The Taxon profile names a reference authority (WFO by default); an entity's canonical name is the name that authority accepts among its members, and only without one the most frequent. The snapshot records which authority supplied the label, so "accepted according to WFO 2024-12" is part of the record, not an unstated default.

4. The Taxon profile

4.1 Attributes

Attribute Notes
scientific_name The name as given, with or without authorship ("Caesalpinia pulcherrima (L.) Sw.", "C. pulcherrima")
authorship When a source gives it separately
rank species, subspecies, variety, form; when a source gives it
family, kingdom Higher classification when given
accepted_name, taxonomic_status Asserted by a backbone source (section 3)
vernacular_names Multi-valued, each with its language; see section 7
identifiers WFO:wfo-0000…, IPNI:…; compared within a scheme

4.2 The name parser

A pure-Python parser for botanical names under the International Code of Nomenclature for algae, fungi, and plants (decision 5). Features:

  • taxon.canonical: genus, epithet and infraspecific epithets, lowercased, without authorship or rank markers ("caesalpinia pulcherrima");
  • taxon.genus, taxon.epithet, taxon.infra, taxon.rank_marker (var., subsp., f.), taxon.hybrid;
  • taxon.epithet_stem: the epithet with its Latin gender ending removed ("pulcherrim"), since an epithet changes ending when its species moves to a genus of different grammatical gender;
  • taxon.authors: authorship normalized (spacing, periods, "ex" and "in" handled), with the basionym author in parentheses kept apart from the combining author;
  • taxon.abbreviated_genus: set when the genus is given as an initial ("C."), to be compared only against genera starting with that letter;
  • the accepted name's canonical form, for backbone observations, and taxon.vernacular per language.

gnparser (Global Names Architecture, MIT, written in Go) is the standard parser and handles far more edge cases. It is an optional backend for evaluation, so the shipped parser can be measured against it.

4.3 Comparisons

Comparison Levels
name exact canonical, epithet gender variant, fuzzy (a typo), abbreviated genus compatible, same epithet in another genus (a recombination), different
authorship same, basionym author matches (one name's parenthetical author is the other's author), compatible abbreviation, different, missing
rank same, different
assertion one's asserted accepted name is the other's name, both assert the same accepted name, one asserts a different accepted name
vernacular shared name in the same language, else
family same, different
identifier as for organizations, within a scheme

4.4 Constraints, caps and traps

  • Cannot-link: different kingdoms (the same genus name can be a plant and an animal: Oenanthe is a plant genus and a bird genus); identifiers that differ within a scheme.
  • Homonyms: the same binomial with different, incompatible authors can be two taxa (a later homonym). Same name, different author is a strong negative, not a match: the lesson of "Premier Services AG", where identical strings named different entities.
  • Caps: a shared vernacular name is capped low ("bluebell" names different plants in Scotland and England); a shared genus alone is a relationship signal (sister species), never identity.
  • Recombinations are the domain's characteristic positive: same epithet stem, a new genus, and the old author in parentheses.

4.5 What this profile does not settle

  • Pro parte synonyms and misapplied names. A name can refer partly to one taxon and partly to another ("p.p."), or be used by some authors for a different taxon than its type ("auct. non L."). One observation then belongs to two entities, which Thisness does not model: membership is one entity per observation. These go to review as relationships ("misapplied for"), and the limitation is stated (decision 8).
  • The hierarchy. Genus, family and order are relationships between taxa, not identity. M4 records them as relationships where sources give them; it does not resolve higher ranks as entities.

5. Benchmark

  • Truth from WFO. A benchmark is a sample of WFO taxa; ground truth is WFO's accepted-name mapping.
  • Observations as they occur in practice, generated from each taxon's real names: the accepted name and its synonyms, with or without authorship, authorship abbreviated differently ("L.", "Linn."), the genus abbreviated, typos, case, rank markers written variously, gender endings after a recombination, and outdated synonyms used as if current (the commonest real error). The traps are real ones from the data: homonyms, recombinations, sister species sharing a genus, and gender variants.
  • Three evaluations. Unaided (noisy names only); with a reference (Catalogue of Life rows ingested as authority observations, measured against WFO, so disagreement between two real authorities is measured rather than assumed away); and name-only (the M3 plan on names without authorship).
  • Data handling (decision 6). A small fixture derived from a few WFO families ships with Thisness for tests (CC0 permits it); thisness benchmark fetch wfo downloads a family archive or the full backbone on demand. Catalogue of Life (CC BY 4.0) is fetched, attributed, and never shipped.

6. Engine changes

  • Canonicalizer per profile (section 3). Snapshots are built by one engine-wide canonicalizer; profiles gain their own, and the Taxon profile's prefers the reference authority's accepted name.
  • Prompt hints per profile. The M3 grouping and adjudication prompts speak of organizations and people. Profiles supply domain guidance (for taxa: recombinations are the same taxon, homonyms with different authors are not, a shared genus is a relationship) as a versioned part of the prompt, so the prompt's ref still changes when the guidance does.
  • Relationships between taxa use the existing relationship records; nothing new.

7. Names that belong to people

Vernacular names include names held by Indigenous and local communities: the Sámi, Cherokee and Surinamese names the chapter writes about. The profile accepts them as vernacular_names with language and source, and the project commits to three rules (decision 7):

  1. Only from published sources whose licence and terms permit it, with attribution. Nothing is scraped.
  2. Following the CARE principles for Indigenous data governance (collective benefit, authority to control, responsibility, ethics), including honoring a source's restrictions on use.
  3. A gap is stated, not hidden. Thisness applies privacy policy per attribute; knowledge sovereignty can require policy per source (a community's names may be shown but not sent to a model, or used only with consent). Per-source policy is designed in a later milestone; until then such names are given the strictest attribute policy the profile allows.

The Person profile has no race attribute, and the project keeps it that way on purpose.

8. Order

  1. Name parser and the Taxon profile; per-profile canonicalizer.
  2. WFO benchmark: fetch command, shipped fixture, noise generator, unaided evaluation; starter model.
  3. Authority observations: the reference evaluation with Catalogue of Life; dossier and Workbench showing which authority said what.
  4. Per-profile prompt hints; name-only and adjudicator evaluation on taxa.

9. Decisions

# Question Recommendation
1 Scope Plants only (WFO, the botanical Code); animals under the zoological Code (authorship with year, different conventions) later
2 What "the same entity" means A taxon as circumscribed by the profile's reference authority, with other authorities' views as evidence from their sources (section 3)
3 How authorities enter As observations from their source, carrying asserted accepted names; no new engine concept
4 Canonical label The name the reference authority accepts, recorded with the authority; most frequent only as a fallback
5 Name parser Our own pure-Python botanical parser; gnparser as an optional backend to measure it against
6 Data Ship a small CC0 fixture from a few WFO families; fetch the rest on demand; Catalogue of Life fetched with attribution, never shipped
7 Indigenous and local names Section 7's three rules; per-source policy as a later milestone
8 Pro parte and misapplied names Out of scope for identity; review and relationships; stated as a limitation

10. Implementation notes

10.1 Step 1: the name parser, the Taxon profile, per-profile canonicalizers

  • Parser (normalize/taxon.py): genus, epithet, rank marker and infraspecific epithet; the combining author and the basionym author in parentheses, normalized to keys ("H. Karst." is hkarst, "Linn." and "Linnaeus" are l); "ex" (the author after it counts) and "in" (the one before it); hybrids written with × or x; an abbreviated genus; cultivar epithets in quotes dropped; annotations ("auct. non", "s.l.", "p.p.", "nom. illeg.") removed and kept as flags; "sp." and "spp." read as a genus alone; zoological years ignored. Letter case tells an epithet from an author only when the genus is written normally; when the genus is in capitals or lowercase, position decides. The first version decided from the whole string, and "CAESALPINIA PULCHERRIMA Sw." lost its epithet. Epithet stems drop Latin gender endings (pulcherrima, pulcherrimus and pulcherrimum are one stem; niger, nigra and nigrum are another).
  • Comparisons (comparisons/taxon.py) as section 4.3, with three corrections found while testing:
    1. Authorship is compared only within one genus, or as basionym evidence. A name moved to another genus has another combining author by nature, so "Poinciana pulcherrima L." against "Caesalpinia pulcherrima Sw." is not a disagreement. Treating it as one held the peacock flower's names apart.
    2. A missing basionym author is not a difference: "Sw." and "(L.) Sw." are the same authorship.
    3. Assertions are compared with each other, not with names. When both records' sources say where the record belongs, the comparison is between those two accepted names. A checklist listing "Pinus picea L." as accepted and WFO listing it as a synonym of Abies alba disagree, although the name is identical; the first version read the identical name as agreement and merged the dissent silently.
  • The reference authority decides the entity (decision 2, made concrete). Assertions from the reference (WFO by default, taxon(authority=...)) are their own comparison level, and a rule, AcceptedByReference, makes them decisive, as a shared registration number is for organizations. A heterotypic synonym ("Pinus picea L." for Abies alba Mill.) shares nothing with its accepted name but WFO's word, so a weighted assertion could never outweigh "a different name". Other sources' assertions stay weighted evidence, and a source that disagrees with the reference produces a contradicting edge: the dissenting record stays apart and goes to review with the reference's taxon, which a test checks. A record its source lists as accepted asserts itself.
  • Name identifiers name names. WFO and IPNI identifiers belong to names, and synonyms have different ones, so a shared identifier is strong and different identifiers prove nothing; there is no identifier cannot-link as there is for organizations.
  • Canonical label. Profiles gained a canonicalizer; the engine's default is now ByProfile, which builds each entity's snapshot with its own profile's canonicalizer (organizations and people keep most-frequent-value, so their snapshots are unchanged), and the store records the canonicalizer that actually built each snapshot. The Taxon profile's AcceptedByAuthority("wfo") labels an entity with the name WFO accepts among its members, or the accepted name WFO asserts for a synonym member, and sets accepted_according_to. An entity whose only WFO record is a synonym is labelled with the accepted name WFO gives it.
  • Community names (section 7) are their own attribute, community_names, with the local_only model policy, so they are matched locally but never sent to a hosted model. They are normalized together with vernacular names into one feature. This closes part of the per-source policy gap within attribute-level policy; the per-source part remains for a later milestone.
  • CLI. The Taxon profile is registered (thisness ingest --type taxon); demo and eval stay limited to organizations and people until step 2 adds the taxon benchmark.
  • Not measured yet. The profile resolves the peacock flower's names correctly under a fixed hand-set model in the tests; real weights come from fitting on the WFO benchmark and the starter model in step 2.
  • Tests: test_taxon.py (13 parse cases and the edge cases; stems and author keys; every name, authorship and assertion level; kingdom conflicts; the authority canonicalizer; the peacock flower's four names resolving to one taxon labelled by WFO, its sister species apart, and Pinus picea joining Abies alba; disagreeing authorities going to review). 534 tests.

10.2 Step 2: the WFO benchmark and the taxon starter

  • Source. WFO publishes its Plant List on ChecklistBank (dataset 2004, "World Flora Online Plant List 2026-06", CC0); Zenodo, the planned source, was down when this was built, and ChecklistBank serves one family at a time, which suits a benchmark better than the whole backbone. benchmark/taxa.py fetches species-level names per family (accepted and synonyms, with each synonym's accepted taxon) and caches them under ~/.cache/thisness/wfo. The shipped fixture is three small families (Pinaceae, Betulaceae, Juglandaceae: 6,705 names, 115 KB) with a CC0 note; python -m thisness.benchmark wfo-fixture rebuilds it. The evaluation benchmark uses five other families (Fagaceae, Sapindaceae, Oleaceae, Salicaceae, Cupressaceae) and the starter two more (Ranunculaceae, Ericaceae), so no family is both trained on and measured.

  • Records follow section 5: whole genera sampled so sister species meet; each record is the accepted name or a synonym, with authorship 55% of the time (sometimes spelled out, sometimes with the basionym author dropped), the genus sometimes abbreviated, a typo, capitals, rank markers written variously. Kinds are measured, not planted: a taxon is a homonym if one of its names also names another taxon in WFO, infraspecific, synonymized, or plain.

  • Reference mode (--reference). WFO's own rows for the sampled taxa enter as observations from source wfo, as they would in a store that holds the backbone. This is not the Catalogue of Life evaluation (step 3), which measures two authorities disagreeing; it measures what the engine does when the reference is present, an upper bound for the unaided case.

  • Every name generator blocks on the name, so EM could not learn the name comparison anywhere, nor (as first written) authorship and rank, whose attributes wrongly included name. Three fixes: authorship and rank declare their own attributes; the asserted-name generator blocks on accepted_name alone, so in reference mode the name comparison is learned where a synonym meets its accepted name; and fitting borrows m from the starter for any comparison no session could learn, when the levels match, recording borrowed_from_prior in the model's diagnostics. Unaided runs borrow only name.

  • Homonyms needed their own level and a stated weight. The first reference run merged later homonyms (Quercus ilex Lour. with Quercus ilex L.) because "different authors" also describes two synonyms in one genus, which match by definition, so EM learned it near zero. Authorship gained a homonym level: one canonical name, two authors. EM then learned it at +9.6 bits, the wrong sign: u comes from random pairs, which almost never share a canonical name, so u for a level that exists only on an exact name is near zero. The profile caps it at -8 bits; the cap is the Code's rule (a later homonym is another name) stated, not a statistic, and a test checks that the uncapped weight is still positive, so the cap is not silently redundant if that ever changes. Fixing it also found a generator bug: author spellings replaced the "L." inside "S.L.Tung".

  • Results (800 taxa, seed 20261008, T_high 12):

    records B3 P B3 R false-merge clusters reviews
    Unaided, no starter 1,854 1.000 .524 0 347
    Unaided, starter 1,854 .999 .517 1 (2 records) 321
    Reference (WFO rows) 6,409 .998 .786 7 (9 records) 237

    Unaided recall is bounded by what names can show: a heterotypic synonym shares nothing with its accepted name, and connectivity for synonymized taxa is .72. Most unaided records also have no authorship, and an exact binomial alone is about 8.3 bits; lowering the strong-edge floor to 8 lifts unaided recall to .73 but makes 19 false-merge clusters (P .983), so the floor stays at 12 and those pairs go to review. In reference mode the nine wrongly placed records are all homonyms: a record written "Quercus ilex Lour." is joined to Q. ilex L. through the many records that give no author, which nothing in the records can separate. The largest false-merge cluster (115) is one such record in an otherwise correct cluster.

  • Starter (profiles/data/taxon_starter.json, 3 KB): 2,500 taxa from the starter families in reference mode, nothing unlearned. profile.starter_model is set.

  • Seen in the Workbench (thisness demo --type taxon --reference): the Workbench needed only the plural "taxa" and an icon, but looking at a real taxon found two faults the tests had not. The entity's fields were a majority over its members, and synonyms outnumber their accepted name, so Castanea dentata Borkh. read as a variety authored by Marshall with the status "synonym". AcceptedByAuthority (version 2) now takes the fields of the authority's accepted record when one is a member, and leaves out fields that describe a single name (authorship, rank, status) when only the authority's assertions are. And the dossier counted support for the most frequent value rather than for the value shown ("species 8/11", where 8 were varieties); AttributeSupport.shown is the group holding the snapshot's value, which affects organizations and people wherever a snapshot is not the majority value.

  • Boundary reviews are mostly one weak point. About one entity in five has an open boundary review. Most come from the recombination level (one epithet stem, another genus) at about 9 bits: common epithets (dentata, japonica, alba) pair unrelated plants such as Cnesmocarpon dentatum and Castanea dentata, and records without a family cannot tell them apart. They go to review rather than merging, so precision holds; term frequency on the epithet is the fix, left for a later step.

  • Tests: the fixture's integrity, generator determinism and WFO truth, reference records carrying identifiers and accepted names, whole-author spelling, the shipped starter matching the profile and trained on families the fixture and benchmark do not use, the homonym cap, a regression floor on the fixture in both modes (unaided P .9986 R .501; reference P .9978 R .827, 3 false-merge clusters), the canonical fields of an accepted name and of a synonym-only taxon, the support count for the value shown, and demo --reference labelling by WFO. 544 tests.

10.3 Step 3: a second authority, and which authority said what

  • Catalogue of Life, latest release on ChecklistBank (3LR, here 2026-09-11, CC BY 4.0), fetched per family and cached under ~/.cache/thisness/col/<release> with its citation in ATTRIBUTION.txt; never shipped. For these families CoL's plants come from World Plants (Hassler's Synonymic Checklists), not from WFO, so the two are independent and their disagreements are real. taxa.align matches CoL's names to WFO's on the canonical name and, where WFO has homonyms, the combining author; a unique name matches when the authors share one ("Hook.fil." and "Hook.f." are the same author: the parser now writes "fil." and "filius" as "f.").

  • Dissent, measured. Each CoL row's truth is the WFO taxon of its name. A row whose own placement (itself if CoL accepts it, else the accepted name CoL gives) is another WFO taxon is dissent: rank when it is the same species at another rank (CoL files synonyms under Castanopsis lamontii, WFO under its variety lamontii), placement otherwise (CoL sinks Fagus longipetiolata into F. sinensis; WFO accepts both). Over the benchmark's 800 taxa CoL contributes 3,930 rows: 576 rank-level dissent, 215 placement, 183 whose accepted name has no single WFO match. A taxon with dissent is of kind disputed (141 of 800).

  • A fix found by the second authority. The assertion comparison scored "both records assert Y, one of them WFO's" as the reference's word, so a CoL record for name X that CoL sinks in Y was matched by the decisive rule as if WFO had said so. The reference vouches for a record only through a name it lists: both records its own, or its assertion about one naming the other (taxon_assertion version 3). The starter was rebuilt.

  • Results (800 taxa, measured against WFO):

    records B3 P B3 R false-merge clusters
    WFO rows 6,409 .998 .786 7
    WFO and CoL rows 10,433 .993 .712 47
    CoL as the reference (taxon(authority="col")), no WFO rows 5,878 .889 .697 126

    With both authorities, 646 of the 791 dissenting CoL records stay apart in review, 120 join WFO's taxon, and 25 join the taxon CoL gives them, mostly rank-level (the species Quercus pubescens and its autonym subspecies, which WFO keeps as two accepted taxa and CoL treats as one); those 25 and 22 agreeing CoL rows misplaced the same way are most of the extra false merges. With CoL as the reference, the low precision against WFO is the two checklists disagreeing, not the engine: scored against CoL's own placement (each record's truth mapped through CoL), the same run is P .993 R .740 with 20 false-merge clusters. The engine follows the reference it is given; which reference to choose is a decision the profile records, and the label says which authority supplied it.

  • Which authority said what. Profiles declare assertions (AssertionFields: the status and accepted-name attributes, the attribute completing the name, the reference source). The dossier groups members by source and by the accepted name each source gives them, normalized as names are ("Castanea dentata Borkh." and "Castanea dentata (Marshall) Borkh." agree), marks the reference and whether each view agrees with the label, and renders an Authorities section in its context. The Workbench shows the section on entity pages and panels, a disagreeing view in red; review cards show each record's assertion ("synonym of …") in place of the raw fields. Support for a shown value is now found by the dossier's grouping, so a label that joins name and authorship counts the records naming it.

  • Not settled here. Dissent placed with CoL's taxon at rank level (autonyms) is the largest remaining error with two authorities; a rule for autonyms (a species and its autonym subspecies or variety under one reference) is a domain decision for later. Term frequency on epithets (section 10.2's boundary reviews) is still open.

  • CLI: thisness demo --type taxon --reference --col; python -m thisness.benchmark run --type taxon --reference --col [--authority col].

  • Tests: alignment (author spellings, homonyms, ambiguous names left out), dissent classification and WFO truth on the fixture, the reference not vouching for names it does not list (comparison and end to end: Fagus longipetiolata), the dossier's authorities and their context section, the Workbench's Authorities section and review cards, --col limited to taxa.

10.4 Step 3b, tried and withdrawn: term frequency on epithets

Section 10.2 said boundary reviews mostly come from common epithets (dentata, japonica) pairing unrelated plants across genera. Measured, that is wrong: one demo review was taken as typical. Of the cross-entity edges in reviews (800 taxa, WFO rows), most are true matches stuck under the strong-edge floor: an exact name without authorship (about 8 bits; 613 pairs) and an abbreviated genus (about 5 bits; 398 pairs). The recombination level is about 550 pairs, half true. Two changes were tried and withdrawn, nothing committed: term frequency on the epithet stem (several adjustments per profile, so the engine could carry one on the name and one on the epithet) changed nothing measurable; a fuzzy level of at most one edit instead of Jaro-Winkler 0.92 (which scores crispa and crispata, carolinensis and carolineana as one name, since epithets share prefixes by construction) left precision and recall unchanged and added 34 reviews. The weight of an exact name grows with the store, as log2(m · taxa): 8.3 bits at 800 taxa, 9.8 at 3,000, about 17 against all of WFO; and precision without a reference falls as the store grows (3,000 taxa: P .994, 32 false-merge clusters), since homonyms grow with it.

10.5 Step 4: prompt guidance per profile, and the model layer on taxa

  • Prompts version 2, guidance from profiles. The three M3 prompts (adjudicate pairs, group names, recommend a review) spoke of organizations and people. Version 2 speaks of no domain; profiles carry prompt_hints, appended as "Guidance for records" (Prompt.with_hints, for_profile). The guidance is part of the system text, so it is part of the prompt's ref: invocations, decisions and recommendations cite the prompt with the guidance actually sent, and changing a profile's guidance is a new prompt. The organization and person rules moved into those profiles unchanged. The taxon guidance gives the Code's rules (gender agreement, recombination with the basionym author, a later homonym is another name, a species and its own variety are different entities) and one rule of conduct: the rules of nomenclature may be used, but which names are synonyms is an authority's opinion, taken from the records and cited, never from the model's memory. Version 1 stays shipped, so earlier invocations remain reproducible.

  • The adjudicator takes its profile's guidance when the engine uses it (Adjudicator.for_profile); the review assistant applies it once it knows the review's type; the name-only plan at the start of its run.

  • Name-only collapse per profile. collapse_key is a profile field (None keeps the organization key). For taxa it is the canonical name and the combining author: "Quercus ilex L." and "QUERCUS ILEX L." are one node, while "Quercus ilex", "Quercus ilex L." and "Quercus ilex Lour." stay three, since collapsing cannot be undone within a run.

  • A negative threshold per profile (max_negative_weight, -4 bits for taxa). The edge policy's T_low of 0 made any pair weighing zero or less a negative edge. Two synonyms in one genus differ in name by nature and weigh about zero, so they were negative edges inside their own taxon, and the whole taxon failed validation: with the adjudicator joining more of it, all 204 records of Quercus robur were held as singletons in one review. At -4 bits only real evidence against identity (a homonym, another family) is negative. With WFO rows this halves reviews (237 to 118) at one more false-merge cluster; without, 321 to 286. The Workbench's model page shows the profile's threshold.

  • Results (800 taxa; the oracle provider answers from ground truth, so these are ceilings; thisness eval --type taxon [--reference] --adjudicator oracle --mode decide [--oracle-error 0.05], --plan name-only):

    B3 P B3 R false-merge clusters reviews
    Names only, no model .999 .517 1 286
    Names only, adjudicator (decide) .999 .747 1 157
    Names only, adjudicator wrong 5% .998 .698 2 166
    Name-only plan .998 .997 2 0
    Name-only plan, wrong 5% .997 .884 4 101
    WFO rows, no model .997 .789 8 118
    WFO rows, adjudicator (decide) .999 .891 1 105
    WFO rows, adjudicator wrong 5% .999 .764 1 207

    The adjudicator finds the false merges FS made (8 to 1 with WFO rows) as well as resolving reviews. The name-only plan is the strongest result: grouping plant names is what a model does well, and its two false-merge clusters are authorless homonyms that the collapse key keeps as one node.

  • What a wrong "different" costs. In decide mode a "different" answer is a negative edge, and a negative edge inside a component holds it for review: a contradiction is surfaced, never settled silently (design 05). With WFO rows a taxon's component is large, so a model wrong 5% of the time holds whole taxa and recall falls below the run with no model (.764 against .789), though precision holds. Until a real model's error rate on taxa is measured, shadow mode is the safe default with a reference loaded; a policy that weighs a single model "different" against a strongly validated component is a decision for later.

  • Cost. With WFO rows and the lower negative threshold there are 14,535 contested pairs at 800 taxa; most are sister species (the oracle answered "related" to 44%). max_pairs (by default 500, the 5,000 used here) takes the heaviest first. Simulated spend at Sonnet prices: about $3 for the names-only adjudication, $7 for 5,000 pairs with WFO rows, $3 for the name-only plan. A real model's numbers need an API key.

  • Tests (test_prompt_hints.py): guidance in the prompt and its ref for all three prompts; version 2 free of domain words, each profile's guidance where it belongs; the taxon collapse key; the per-profile negative threshold; the adjudicator's invocations citing the taxon-guided prompt and resolving more on the fixture with no more than one extra false merge; the name-only plan on plant names.

10.6 A real model on taxa: gpt-6-luna

The first real-model runs, on OpenAI's gpt-6-luna ($0.10 and $0.50 per million input and output tokens), 800 taxa, seed 20261008, 16 requests in flight. Every exchange was recorded; thisness eval --replay <cassette> --save <store> answers a recorded run again, free and offline, and keeps it in a store the Workbench opens.

  • Fixes the run needed. OpenAI's API takes the output limit as max_completion_tokens (its reasoning models reject max_tokens); the provider picks the field by host, and an answer cut off at the limit is reported as such. thisness eval now scores the model's answers against the benchmark's truth (vs_truth): accuracy, wrongly apart (one entity answered "different" or "related"), wrongly different (the answer that sets a negative edge), wrongly same, and the confusion counts.

  • The guidance was wrong, and the model followed it. With step 4's taxon guidance, luna was wrongly apart on 14% of true matches, "different" on 8%. Its rationales were sound: it applied "a species and its own variety are different entities" and "the same epithet in two genera without basionym evidence usually names two plants", both too strong, since WFO sinks many varieties into their species and accepts many recombinations without the author in the record. The revised guidance (in profiles/taxon.py) answers "related" or "unsure" from names alone and "different" only on contradicting evidence (another family or kingdom, one name with two authors, an authority placing the names apart). Wrongly different fell to 0, accuracy rose from .910 to .919, unsure answers from 10 to 145, and wrongly same stayed at 2.8% (sister species).

  • Results (B3 against WFO):

    P R false-merge clusters reviews records held cost
    Names only, no model .999 .517 1 286 0
    Names only, luna decide .995 .726 6 164 7 $0.23
    Names only, oracle decide .999 .747 1 157 1
    WFO rows, no model .997 .789 8 118 6
    WFO rows, luna decide (5,000 of 14,535 pairs) .999 .708 3 128 1,646 $0.65
    WFO rows, oracle decide .999 .891 1 105 87
  • What luna's wrong "same" costs. With WFO rows its wrongly-same rate is 5% (109 pairs, 87 applied as strong edges). A wrong "same" joins two taxa; luna's own correct "different" answers between them then contradict the component, and consensus holds it whole: 9 of the 10 largest held reviews contain an applied wrong "same". Precision holds (false-merge clusters 8 to 3); recall falls below the run with no model. Most of those answers cannot be told from names alone: an authorless record of a homonym ("Quercus hybrida"), a typo landing on another name ("Quercus andrewsii" for Q. andresii), a recombination WFO places elsewhere. One is a real slip: Cyclobalanopsis delavayi and Sapindus delavayi (Franch.) Radlk., two families, joined on the basionym author.

  • A stronger model: gpt-6.1-sol ($2 and $10 per million tokens, about 15 times luna). Names only, decide: P .995, R .728, 6 false-merge clusters, $3.63, the same outcome as luna; it judges better (accuracy .946 against .919, wrongly same .022 against .028) mostly by answering "unsure" more. With WFO rows: P .999, R .771, 2 false-merge clusters, 1,231 records held, wrongly same .038, $8.91, no failed requests. Better than luna, still below the run with no model on recall: a 3.8% wrong "same" is enough to hold whole taxa. Model quality was not what bound the result; the policy was (section 10.7). 16 of luna's 625 requests in its WFO-rows run failed and were not recorded, so their cause is unknown.

10.7 A model's "same" yields to a contradiction

Section 10.6 found that with an authority's rows loaded, each wrong "same" a model applied held a whole taxon: it joined two taxa, the model's own correct "different" answers between them then put a negative edge inside the component, and consensus held it. The adjudicator now applies a model's "same" only where it adds no contradiction (same_yields, the default; adjudicator ref version 2, while the old policy keeps version 1). Before consensus, every component over strong edges that holds a negative or cannot-link pair has the strong edges the model set inside it put back to contested, so that component is settled on the evidence alone, and those answers are recorded as not applied. A model's "same" cannot override a "different", from the model or from the evidence.

Measured on the recorded runs, replayed (800 taxa, decide mode):

P R false-merge clusters records held
WFO rows, no model .997 .789 8 6
WFO rows, luna, old policy .999 .708 3 1,646
WFO rows, luna, yielding .997 .888 9 66
WFO rows, sol, old policy .999 .771 2 1,231
WFO rows, sol, yielding .998 .890 8 67
WFO rows, oracle, yielding .999 .894 2 60
WFO rows, oracle wrong 5%, old policy / yielding .999 / .998 .764 / .847 1 / 5 1,137 / 286

With yielding, both real models reach the oracle's recall, and luna does what sol does at a fourteenth of the cost. Precision returns to the level of the run with no model (8 or 9 false-merge clusters against 8), not below it: the old policy's fewer false merges came from holding whole taxa, which also held the evidence's own false merges out of sight. Against no model, a cheap model in decide mode with a reference loaded is now recall +.10 at equal precision, for $0.65 at 800 taxa. Names-only runs barely change (luna R .726 to .726, records held 7 to 3): without authority records there is little to contradict. Decide mode with yielding is the recommended setting for taxa, with or without a reference; shadow mode remains the default.

Correction (narrower yielding). As first written, a model's "same" yielded to any negative pair in its component, including the evidence's own. On organizations that discarded a right model's work: a company's members often disagree on an old address or phone, and a correct "same" across that is the model doing its job (oracle recall .983 with no yielding, .964 with it; 617 of 1,790 right answers applied). A model's "same" now yields only to its own "different" answers and to cannot-link constraints. Replayed:

no yielding yielding to any negative yielding to the model's "different"
805 organizations, oracle: R .983 .964 .983
805 organizations, luna: R / false-merge clusters .972 / 3 .962 / 4 .981 / 4
1,000 people, oracle: R .951 .952 .951
800 taxa, WFO rows, luna: R .708 .888 .887
800 taxa, WFO rows, sol: R .771 .890 .890

Against no yielding, each run gains recall and gives back about one false-merge cluster: a component no longer held whole is settled on its evidence, and that evidence's own false merge appears.

10.8 gpt-6-luna on organizations and people

The same model and settings on the M2 benchmarks (seed 11, decide mode, yielding to the model's "different"):

P R false-merge clusters reviews cost
805 organizations, no model .999 .888 1 135
organizations, luna .997 .981 4 35 $0.20
organizations, oracle .999 .983 1 31
1,000 people, no model .998 .723 4 350
people, luna .995 .868 9 155 $0.25
people, oracle .999 .951 2 57

On organizations luna reaches the oracle's recall and cuts reviews by three quarters; it calls 10% of distinct-company pairs "same" (26 of 253), the folding error design 05 anticipated, for three more false-merge clusters. On people it is both too cautious and too credulous: "unsure" on 46% of pairs (408 of them true matches) and "same" on a quarter of the pairs of different people (51 of 208), so false-merge clusters rise from 4 to 9. Section 10.9 tries revising people's guidance.

10.9 People's guidance: two revisions tried and withdrawn

On people, luna's wrong "same" answers (51 of 208 pairs of different people) were mostly the benchmark's hardest traps: a father and son with one name (18), similar names of different people (11), members of one household (8), initials taken as a specific person ("F. Qureshi"), and a name with a city and nothing in conflict. Its 408 "unsure" answers on true matches were the same evidence: an exact name and a city, one record nearly empty. It answered one ambiguity two ways, and from the records alone it cannot tell them apart.

Two revisions, both measured on 1,000 people and both withdrawn (patch kept outside the repository):

P R false-merge clusters reviews
No model .998 .723 4 350
Section 10.8's guidance (kept) .995 .868 9 155
"Same" only with a shared date of birth, identifier, email, phone or street address .998 .731 4 338
"Same" for a full name at one place when the name is rare, with a name-rarity result added to each pair .993 .832 13 215

The strict rule did what it said: wrongly same fell from 25% to 1.4%. It also left luna "unsure" on 88% of pairs, since most contested people pairs share nothing but a name and a place, and recall fell back to the no-model level. The middle rule depended on telling luna how common a name is. The term-frequency adjustment cannot say that: it is relative to the exact-name u from random pairs, which is tiny, so nearly every real name comes out more common than that baseline (1,692 of 1,868 pairs "common", none "rare"); luna ignored the signal, and both measures got worse. An absolute rarity measure would need a different quantity, and the father-and-son trap would remain: same name, same city, often the same address.

For people, the model suggests rather than decides. The evidence and the review queue settle merges; the review assistant (thisness recommend) answers open reviews as suggestions a person accepts or rejects. A model whose "same" is right about three times in four on the hardest people pairs is useful there and risky as an automatic merge. The recommended settings by domain: organizations and taxa, decide mode with yielding; people, the review assistant.

10.10 The review assistant on people

thisness recommend asked gpt-6-luna for a suggestion on each of the 350 open reviews of the 1,000-people demo store (resolved with no model): 350 answered, $0.14. Scored against the benchmark's truth (the store's records are the benchmark's), 225 of the 261 decided suggestions would be right if accepted (86%): "same" 199 of 204, a regrouping 18 of 46, "different" 6 of 9, "related" 2 of 2; 89 were "unsure". Accepting every suggestion would merge different people in 13 reviews (5%) and leave a match apart in 23. A review shows the model every record of the question at once, where the adjudicator shows it one pair, and its "same" is far more reliable there (97.5% against luna's 75% on the hardest pairs in decide mode). Suggestions are shown, never applied: a person accepts or rejects each one.

recommend now asks --concurrency reviews at a time (default 4): requests are prepared from the store in order, the model calls run in parallel, and answers are checked and recorded in order, so the recommendations are those a one-at-a-time run records.

10.11 What the model does, per profile

Profiles state the model's role (model_role): organizations and taxa decide, people suggest; none means shadow. thisness resolve --provider ... follows it: with decide, contested pairs go to the adjudicator in decide mode (its "same" yielding to its own "different"); with suggest, the run resolves on the evidence and the review assistant then suggests answers for the reviews it opened. --model-role overrides the profile for a run (decide, shadow, suggest, off), and --concurrency now applies to resolve, recommend and ui alike.

The Workbench shows the labels of the assistant's request. Cards carry R1, R2, ... and comparisons C1, C2, ..., from one function the assistant also uses (review_labels), so a rationale such as "R11, R13 and R14 share an email" reads against the board; the review queue has a Suggestion column.

View the source of this page