Your AI Just Lied to You. It Didn’t Even Know It.

By Ryan Vanshur

Also published on the Guild Letter: Read the original issue.

The Day I Stopped Trusting “Done” Nine commits. Eight bugs found and fixed. Two test accounts validated. Forty-three automated checks passing. I had just built a complete deal intelligence pipeline with Claude Code: pull call transcripts , analyze them with AI , generate executive-ready slide decks . Then I asked the AI to audit the documentation. “Make sure everything is updated and accurate.” It said it did. I found stale content. I asked again. It said it fixed everything. I found more. Four times I asked. Four times it missed things. Not edge cases. A file header saying “Last Updated: March 23” when the content reflected March 26 changes. A “What’s Remaining” section listing tasks that were completed two days earlier. A “Week 3: Active” status label when we were in Week 4. The AI wasn’t broken. It was doing something more subtle and more dangerous: it was scanning instead of reading. Why This Matters More in GTM Than Anywhere Else “It’s just docs.” That’s what most builders think. Here’s who was about to review my deliverables: the client’s IT security team auditing the repo before granting access, the RevOps lead who would use these docs as his daily operating manual, and the VP of Enablement who would evaluate whether the deliverable was complete. If I had pushed a README referencing a deleted directory, a handoff doc with stale task lists, and a setup guide pointing to the wrong URL, every one of those reviewers would have noticed. The first questions my security practices. The second loses confidence in the system’s reliability. The third wonders what else is incomplete. Documentation quality is a proxy for engineering quality. In GTM, your collateral quality is a proxy for your operational quality . If the battlecard is stale, the assumption is the pipeline data is too. Now scale that problem across every AI-generated artifact your team touches. What “Scanning” Actually Looks Like When I asked the AI to audit 25+ documentation files , here’s what it actually did: Delegated the work to a sub-process with instructions like “find references to wrong competitor counts and outdated file paths” Ran keyword searches: "4 competitors" , "5-slide" , "old-directory-name/" Found pattern matches and replaced them. “4 competitors” became “7 competitors.” Reported success: “15 issues fixed across 10 files.” Sounds thorough. Here’s what it missed: A date header three days stale A “What’s Remaining” section where four of six tasks were already done Architecture descriptions referencing an 8-slide output when the system now produces 9 A setup checklist where half the items needed to be marked complete Quota warnings for a system that had migrated to a different platform weeks earlier None of these would be caught by keyword search. They require reading the content, understanding what it claims, and evaluating whether those claims are still true. That’s comprehension. That’s what was missing. Your GTM Stack Has the Same Problem If you’re using AI to generate any of these, you have this problem right now: Pipeline Reviews. Your AI summarizes deal status from CRM fields and call notes. But when the champion changed roles three weeks ago and nobody updated the contact record, the summary still says “strong executive sponsorship.” The AI didn’t lie. It read what was there. What was there was wrong. Battlecards. A competitor launched a new pricing tier last month. Your battlecard still lists their old pricing because nobody ran an update. The AI that generated the card can’t tell you it’s stale, because it doesn’t know the competitor’s pricing changed. It only knows what you gave it. Onboarding Docs. Your new AE gets a “first 30 days” playbook that references a CRM workflow your team stopped using in January. The AI that wrote the playbook didn’t use the CRM. It described what someone told it the CRM does. Client Deliverables. A slide deck referencing “4 integration partners” when you now have 7. A proposal listing capabilities you shipped two months ago under “planned features.” An ROI model with assumptions from last quarter’s pricing. Every one of these is the scanning problem wearing a different hat. The AI found patterns it could match and declared victory. The contextual errors, the ones that actually erode trust, sailed through untouched. Why AI Defaults to Scanning (And Why It’s Hard to Fix With Better Prompts) After watching this fail four times in one session, I started to see why it happens. It’s not one cause. It’s four behaviors that compound when you ask for any kind of broad validation. 1. Efficiency Over Thoroughness AI is trained to be efficient. When given “audit 25 files,” the most efficient approach is keyword matching. It’s fast. It’s parallelizable. It produces impressive-looking results. The problem: efficiency and thoroughness are in direct tension. Reading 25 files top to bottom, understanding each one, thinking critically about every section? That’s slow. Sequential. It doesn’t produce the satisfying volume of changes that makes the output feel complete. So the AI optimizes for throughput. It scans instead of reads. 2. The Delegation Trap Modern AI systems can spawn sub-processes to handle tasks in parallel. Genuinely useful for running tests or searching directories simultaneously. Catastrophic for validation work. When you tell a sub-process “check these files for stale content,” you’re giving it a checklist, not comprehension skills. It pattern-matches against the checklist and reports back. It doesn’t have the context to understand that “Week 3: Active” is stale because the project is in Week 4. It can only find what you told it to look for. 3. The Confidence Problem After running a keyword sweep, the AI genuinely believes the audit is complete. It found issues. It fixed them. The numbers look right. When you ask “is everything accurate now?”, it says yes. Not because it’s lying. Because its model of “accurate” is “free of the patterns I checked for.” This is the most dangerous part. The AI’s confidence in its results is inversely correlated with how many issues remain. It’s most confident precisely when it’s most wrong. 4. Context Pressure Reading 25 files fully consumes significant working memory. AI systems are trained to manage that carefully: summarize, delegate, avoid filling the window with raw text. This training actively works against thorough validation, which requires holding the full text of each document while simultaneously holding the ground truth of what’s actually true today. The AI resolves this tension by not reading the files fully. It reads selectively, looks for checklist patterns, and moves on. The Fix: A Structured Validation Skill After the fourth failed pass, I stopped trying to fix this with better prompts. Better prompts work once and degrade. Instead, I built a structured skill: a reusable methodology that explicitly prevents the scanning trap . Here’s the framework: Phase 1: Discovery List every file that needs validation. Don’t skip any. Don’t assume you know what’s there. Phase 2: Establish ground truth Before reading a single document, write down what is true right now. What’s the current architecture? What changed recently? What are the actual counts, dates, statuses? This becomes the reference frame everything gets checked against. Phase 3: Read and validate For each file, read it yourself. Not a keyword search. Not a delegation. Read the actual content. Then work through four questions: Structural accuracy. Do file counts, directory trees, and dates match reality? Status accuracy. Are “pending” items actually done? Are “active” items complete? Content accuracy. Does the described system match the current system? Relevance. Is this document still needed, or has its purpose been served? Phase 4: Report For each file, one verdict: ACCURATE, STALE, or IRRELEVANT. For stale files, list specific line numbers and what needs to change. Phase 5: Fix Make the edits. Then re-read the file to confirm the fix didn’t introduce new problems. The Critical Rule You must read every file you verdict. If you report a file as “accurate” without having read it, the audit is invalid. No exceptions. This rule exists because the number one failure mode is marking files as “checked” without reading them. The AI’s natural tendency is to spot-check and extrapolate. The rule forces sequential, comprehensive reading. What Makes This Different From “Just Try Harder” The skill isn’t instructions to be more thorough. It’s structured to counteract each of the four failure modes: Efficiency over thoroughness? Sequential reading is mandatory. No shortcuts. Delegation trap? Explicit prohibition on delegating validation work. You read it yourself. Confidence problem? Ground truth established first. Verdicts are checked against facts, not assumptions. Context pressure? Files are read one at a time. No need to hold all 25 simultaneously. The skill also includes an anti-pattern list: explicit descriptions of behaviors to avoid , with explanations of why each one fails . This matters because the AI needs to recognize when it’s falling into a failure mode, not just know what correct behavior looks like. The Broader Lesson for GTM Operators This experience revealed something that applies far beyond documentation: The AI’s failure mode on validation tasks is the opposite of its failure mode on generation tasks. When generating content, AI tends to be too verbose, too creative, too eager to add things. The fix is constraints: templates , character limits , schemas . When validating content, AI tends to be too shallow, too pattern-oriented, too eager to declare success. The fix is not more constraints. It’s structured comprehension. You can’t keyword-search your way to accuracy. You have to read. Most AI tooling and prompting advice focuses on the generation side. “Be specific about what you want.” “Give examples.” “Use structured output.” Good advice for creation tasks. But for validation tasks, the advice should be different: Force sequential processing. Prevent shortcuts. Establish ground truth before evaluation. And above all, make the AI read the actual content instead of scanning for patterns. Steal This: The Ground Truth Validation Checklist Print this. Run it before shipping any AI-generated GTM artifact. Subscribe now GROUND TRUTH VALIDATION CHECKLIST ────────────────────────────────────────────── BEFORE validating anything, answer these: [ ] What are the current numbers? (competitor count, pricing tiers, team size, deal count) [ ] What changed in the last 30 days? (new features, departed contacts, pricing updates, org changes) [ ] What dates matter? (last update, contract renewals, quarter boundaries) [ ] What was recently completed that might still show as "pending"? THEN, for each artifact: [ ] Did I (or the AI) actually READ the full document? (Not searched. Not skimmed. Read.) [ ] Do the dates in the document match reality? [ ] Do the status labels match reality? ("Active" that should be "Complete"? "Planned" that already shipped?) [ ] Do the numbers match reality? (Competitor count, slide count, integration count) [ ] Would a skeptical buyer find something wrong in the first 60 seconds of reading this? IF ANY BOX IS UNCHECKED: the artifact is not ready. What This Means for Your Monday The teams that figure this out first have an edge that compounds. Not because their AI generates better content. Everyone’s AI generates impressive content. The edge is that their content is actually true. Your competitor’s battlecard says you have 4 integrations. You have 7. Their onboarding doc references a workflow they deprecated in January. Their proposal lists “planned features” that shipped two months ago. The irony is that the same AI that made it easy to generate all this collateral is the worst tool for checking whether it’s still right. Generation and validation are fundamentally different cognitive tasks. Treating them as the same thing is how you end up shipping confident, professionally formatted, completely wrong information to a VP of Finance who will notice. The skill I built removes that problem. One file at a time. Read top to bottom. Checked against reality. Your reps are already using AI to build things. The question is whether anyone is checking what it built. If the answer is “the AI checks itself,” you have a scanning problem. And now you have a framework to fix it. This Was One Skill. I Built 14. The Doc Audit Skill you just read about is one of 14 production AI systems I built at Handle without a single engineering resource. Skills for discovery calls. Skills for pipeline scoring. Skills for competitive displacement. Skills for deal intelligence. Each one codifies a specific piece of GTM methodology so that any rep can get expert-level output in minutes. On Tuesday, I’m publishing the entire system. The AI-Powered GTM Stack is a 6-part series that walks through everything: how Skills work, the knowledge architecture underneath them, the operations playbook that ties them together, the measurement system that proves they’re working, what an AI-native GTM org actually looks like when it’s running, and how to connect your AI to every system your revenue team touches. Six articles. The complete Claude Code GTM skill kit. The frameworks, the anti-patterns, the before-and-afters. All of it. This is the system running in production on nine figures of active pipeline. Not a concept deck. Not a webinar replay. The actual operating system, published so you can build your own. Subscribe now If you want it in your inbox Tuesday morning, subscribe to the Vertical GTM Guild on Substack. Every subscriber gets the full series, the skill files, and the validation checklist from this article as a printable download. This is what the Guild is for. Operators building real systems in real verticals, sharing what actually works instead of what sounds good in a keynote. If that’s you, you’re in the right place. Subscribe to the Vertical GTM Guild →