Source Check evidence
Methodology: verifying a PDF comparison result
A public record of the adjacent supply-side audit cited in our verification guide.
Published and updated ยท Read the guide
Evidence label and scope
Evidence label: adjacent supply-side pilot. The frozen confirmatory snapshot ID is week1-confirmatory-20260817T075951Z: query compare pdf files, collected August 17, 2026 in the United States, English, on desktop.
We selected the first 10 eligible non-forum publisher results and the first five useful YouTube transcripts with at least 100 words. Video ranks 2 and 4 were skipped because rank 2 had no available transcript and rank 4 was irrelevant; the useful video ranks were 1, 3, 5, 6, and 7. The result is 15 selected sources: 13 known and two unknown publisher pages.
Public decision ledger
Download the sanitized 15-source CSV decision ledger. It includes the selected source metadata and final yes/no/unknown decisions, but no copied page or transcript text, reviewer identities, local paths, or private data.
| Criterion | Organic | Video | All known |
|---|---|---|---|
| Source verification | 0/8 | 0/5 | 0/13 |
| Accuracy limitation | 1/8 | 0/5 | 1/13 |
| Zero-result response | 1/8 | 0/5 | 1/13 |
| Evidence linkage | 2/8 | 2/5 | 4/13 |
| All four | 0/8 | 0/5 | 0/13 |
The DocDiffAI-defined composite requires all four safeguards. Its predeclared threshold was fewer than 3 of 15 sources satisfying all four. Even if both unknown pages satisfied all four, the conservative worst case is 2 of 15.
Frozen operational definitions
- Source verification
- An explicit post-comparison instruction to independently inspect or verify the relevant passages in both original and changed source files. Navigating or reviewing a generated report alone did not qualify.
- Accuracy limitation
- A statement that automated comparison can miss, misclassify, or falsely flag a change, or that a clean result does not establish identity.
- Zero-result response
- A concrete action for a no-differences result, such as checking selected files or settings, or checking text, images, or formatting.
- Evidence linkage
- A finding-level link to a page or passage in both files, including synchronized evidence that supports checking each result item.
A status was yes only when the saved source met the definition, no when reviewed material did not meet it, and unknown when the publisher page was inaccessible or insufficiently extracted. Unknown pages were never recoded as no; they are unknown for all four criteria.
Coding and adjudication
Two independent coders reviewed saved full text and transcripts. They agreed on 281 of 300 exploratory decisions (93.7%) and 53 of 60 confirmatory decisions (88.3%) before adjudication; disagreements received a final documented decision.
The post-collection protocol replaced unreliable candidate automation as a decision layer with independent full-text coding and adjudication. That stronger two-coder correction was not independently preregistered. Automated patterns were used only to help navigate candidate material.
Limits of this audit
This is one dated US desktop query and a non-random sample, not a demand study or a population estimate. It does not demonstrate demand for the verification article, output accuracy, competitive uniqueness, causality, ranking effects, or AI-citation effects.
Product and utility pages were eligible because they competed for the query, and their brevity can reflect product-page intent rather than editorial negligence. The two unknown publisher pages remain unknown. These results describe observed coverage for this corpus, not how any product performs.
Corpus integrity identifiers
Confirmatory Google task ID: 08170759-9020-0139-0000-072da158c8bf. YouTube task ID: 08170759-9020-0139-0000-6f8a2dd108c2. Google raw SHA-256: 8e3363444009cb7242e2487d3822cdb9c9baf8107893a427a64cca67edd5aeb6. YouTube raw SHA-256: daf30bbe3bbd55b85f2ef489187af7e5b9e3d5b957444435dff808ead2f96c01. Frozen corpus ledger SHA-256: 1bf203af73ff4a91eab32c9d69ab9159507f35eb0cb3fad6db72de514d792d4e.