How we write and verify questions
Most practice sites say their questions are “expert-written” and stop there. Here is the whole process, with the counts, rebuilt on every deploy from the records that decide what serves.
Where the bank stands right now
All 12,770 questions you can practice right now are original and source-cited. Every number on this page is read from the committed verification records at build time, so it is as current as the last deploy and no more. If the checks stop running, the figure above goes stale in public rather than quietly. That is the point of printing it. The most recent ran on 15 September 2026, the first on record on 23 July 2026.
What happens to a question
- Written from published objectivesNever from exam content. Each question starts at one sub-objective the certifying body publishes openly, and is written from the documentation that objective points at.
- Written to the standard belowThe build measures the machine-checkable points across the whole bank and fails on them. The rest are held by the person writing the question, which is stated rather than implied.
- Checked against the sourceSoftware re-fetches the cited page and a model judges whether it supports the marked answer. The outcome is one of four states, and two of them never serve.
- Source-cited, and openableEvery question links the exact page it was written from, with the month that page was last read. The citation is on the question, not in a footnote about our process.
- Reported, and retired if wrongA report link sits on every question and goes to the top of the queue with your comment attached. We would rather retire a question than keep a wrong one.
The ten points every question is written to
Some of these the build measures across the whole bank and fails on: the share that is scenario-based, clueing by option length, whether all three explanation tiers are substantive, the difficulty spread. The rest are held by the person writing the question. We do not claim a stored per-question pass, because no such record exists.
- Maps to exactly one published sub-objective
- Original; never derived from a real exam item
- Scenario-based where it should be (bank target: 60%+ scenario)
- Exactly one defensible answer (or exact multi set)
- Every distractor plausible to a partially prepared candidate
- No clueing via length, grammar, or specificity
- Three-tier explanation complete and substantive
- Doc link is a real, current, authoritative page
- Difficulty rated 1-3 (bank target 25/50/25)
- Current exam version content; zero retired material
Every question ends in one of four states
A question that passes an internal standard can still be confidently wrong, so every one is checked against the official page it cites. Contradicted and unsourced questions can never reach you.
Doc verified
The cited source supports the answer and contradicts no distractor. The passage is stored with the record, and the question carries a caption naming the page and the month it was read.
Needs review
The source was ambiguous, partly disagreed, or none was adequate. It serves only if it was approved when written, stays source-cited, and carries no verification caption.
Unsourced
The cited link is dead: 400, 404 or 410, so the evidence cannot be checked at all. Never served. A page that is live but blocked to our fetcher goes to review instead.
Contradicted
The cited source disagrees with the marked answer. The highest-priority defect, and never served.
Which, across the bank, add up to
Across the 12,770 questions serving right now, counted at every deploy from the same records that gate serving.
- 12,730verified against their cited page, with the passage that supports the answer stored and the check dated
- 40checked by link alone, with no passage stored: source-cited, the link fetched for a status code and the page not read, and these carry no verification caption
- 0written, then blocked by verification: contradicted by their source or citing a dead page. Never served
The check is AI-assisted documentation review: software re-fetches the cited source and a model judges whether the passage supports the marked answer. It is not a human reading every page, and we do not say it is. Documentation verification covers 99% of everything written; the other 41 are real work in progress, not a number we are hiding. Found a mistake? issues@firsttry.app, or the report link on any question.
Where verification has reached
Thirteen vendors, 82 certifications. Open a vendor to see the count beside each exam.