Wave 3 Protocol

Amendment, July 8, 2026 (before any collection). As originally frozen, collection fired at the next model release. That design had a flaw a careful reader would catch: the segment maps and phrasing panel are new instruments, and if their first reading happens at a release, the release's effect on them can never be separated from their ordinary state. So Wave 3 becomes the baseline, collected immediately upon publication of this amendment, against the same eight models as Wave 2. The next frontier release triggers Wave 4, the identical instrument run again, and the release delta is read between them. Bench conversion, which requires a release between readings, moves to Wave 4. Nothing else changes: the phrasings, sampling, reporting rules, hypothesis, and limitations below stand as frozen.

Second amendment, July 8, 2026. Two additional lenses were collected as part of the Wave 3 baseline: the solution lens (10 plain-language problem statements, no category named) and the journey lens (framing, comparison, validation, and renewal questions across 5 categories, with finalists drawn from this wave's own shortlist consensus). Both instruments were frozen and timestamped in the research archive before collection and are reported as first-reading baselines, not confirmatory tests. Their full verbatim instruments appear below, after the original phrasings.

Trigger

Wave 3 (baseline): collection begins upon publication of the July 8 amendment above. Wave 4 (release reading): collection fires at the next major frontier model release from Anthropic, OpenAI, Google, xAI, or Perplexity, or on October 1, 2026, whichever comes first, completing within 7 days, with the triggering model joining as a ninth panelist.

What Wave 3 does

Three things. It continues the index: same 26 categories, same spine question, so every Wave 2 metric gets its second reading. It runs the phrasing-robustness panel we committed to in print, so number-one changes are reported only when they survive multiple ways of asking. And it adds the segment view: the same categories asked for a 50-person, 500-person, and 5,000-person company, producing three maps where there has been one.

The instrument, verbatim

Six phrasings per category. Every prompt ends with an instruction to answer in a fixed JSON shape so responses are scored identically across labs. No system prompt, no follow-up turns, no web access except the search-grounded control model.

P0 · The spine (unchanged since Wave 2, never modified). You are helping a B2B software buyer build a shortlist. What are the best {category} vendors right now? Return the top 10 vendors as a ranked list, best first. Vendor names only.

P1 · Unframed (tests the role preamble). What are the best {category} vendors right now? Return the top 10 vendors as a ranked list, best first. Vendor names only.

P2 · SMB. You are helping a 50-person B2B software company build a shortlist. What are the best {category} vendors for a company that size right now? Return the top 10 vendors as a ranked list, best first. Vendor names only.

P3 · Mid-market. Same wording as P2 with "500-person".

P4 · Enterprise. Same wording as P2 with "5,000-person".

P5 · Depth (tests the top-5 sensitivity found in Wave 2). You are helping a B2B software buyer build a shortlist. What are the best {category} vendors right now? Return the top 5 vendors as a ranked list, best first. Vendor names only.

Segments are anchored by employee count rather than analyst labels, because buyers say "we're a 50-person company," not "we're SMB." The rebrand probes continue unchanged from Wave 2.

Sampling

Five runs per model per category per phrasing; consensus scored across runs; roughly 7,000 API calls. Our calibration study (July 2026) supports the standard: across a frozen category and a liquid one, on a frontier model and a consumer-tier one, the consensus top five from 5 runs matched the 20-run reference exactly, while single answers missed a third of the reference set in the noisiest cell.

The reporting rules we are bound to

• A number-one change is reported as outright only if the new leader also holds in at least 4 of the 6 phrasings. Anything weaker is reported as protocol-sensitive.
• Segment maps are reported per category with a divergence score between the enterprise, mid-market, and SMB readings.
• The hypothesis, stated so we can be wrong in public: concentration falls and the number of contested categories rises as company size falls. If the data says otherwise, that is the finding.
• Bench conversion: Wave 2 recorded every vendor ranked 6 through 10. Wave 3 reports how many entered a top five, named, as the first test of whether the bench predicts entry.
• Continuity metrics (convergence, contested count, concentration, turnover, self-consistency) are computed on the unchanged spine question exactly as in Wave 2.

The solution lens, verbatim (second amendment)

Each problem statement is asked as: "We are a 500-person B2B software company. [problem] What software should we look at?" with a fixed JSON answer shape naming categories and vendors. The ten problems:

1. We keep missing our pipeline number and nobody can tell me why until the quarter is already lost.
2. Our reps spend most of their day researching accounts and writing emails instead of talking to buyers.
3. Renewals keep surprising us in the last month of the quarter. We find out an account is unhappy when the cancellation notice arrives.
4. We spend a lot on marketing and nobody can tell me which programs actually produce revenue.
5. Marketing sends plenty of leads but sales says none of them ever buy.
6. Buyers tell us they asked an AI assistant for vendors in our space and we never came up.
7. Deals go quiet after the demo and nobody can tell me what actually happened on those calls.
8. Our CRM is full of duplicates and dead contacts and nobody trusts the reports that come out of it.
9. Users sign up, poke around once, and never come back.
10. Our forecast is a spreadsheet of guesses and the board has started to notice.

The journey lens, verbatim (second amendment)

Five categories (CRM, sales engagement, AI SDR, customer success, conversation intelligence), finalists taken from this wave's own shortlist consensus. Four collected stages; the shortlist stage reuses the spine data.

Framing. We are a 500-person B2B software company. [category-specific pain] What should we do first?

Comparison. We are a 500-person B2B software company deciding between [A] and [B] for [category]. Which should we pick?

Validation. We are a 500-person B2B software company about to sign a contract with [vendor] for [category]. Anything we should know before we sign?

Renewal. We are a 500-person B2B software company. We have used [vendor] for [category] for three years and the renewal is due. Should we renew or look at alternatives?

Stated limitations

• We measure models through the API. Consumer apps wrap the same models in system prompts, memory, and search toggles we cannot see. This is the largest external-validity limit of the program, and it is permanent.
• Category labels are part of the instrument; asking about a named category adopts a framing. The list is frozen for continuity.
• Five runs bounds sampling noise; it does not eliminate it. Small movements between waves should be read against the calibration noise floor.
• The grounded arm is one vendor's retrieval, representing search-grounded answering generally rather than all retrieval systems.

What gets published

What moved, the three maps, robustness scores, and bench conversion, with raw data available on request. Causal diagnosis for named companies stays out of print; that analysis is client work.