Day 14: A Score Is Not a Qualification
The website scorer gave every lead a rating. Then we manually checked the highest-rated "bad website" lead and found a modern agency-built site with 5,000 Instagram followers. The model was wrong. Day 14 was fixing it.
861 leads. The problem: a score isn't a qualification.
Day 14 Metrics
| Metric | Value |
|---|---|
| Leads manually reviewed | 60 |
| Leads qualified (real evidence) | 31 |
| Leads disqualified | 29 |
| Automatic disqualifications (metro / franchise / wrong state) | 14 |
| Revenue | $0 |
The Flaw
The website scorer does a reasonable job estimating site quality from technical signals — copyright year, platform detection, layout patterns. But it can't answer the question that actually matters: is this business genuinely our target?
The first real test case: a business that scored as "prime" — bad website, likely needs us. Manual check revealed the full picture. Modern agency-built site. 5,000+ Instagram followers. Active Reels with real views. The kind of business that gets pitched by agencies every week and knows how to say no.
The score said yes. Everything else said no.
The Real Qualification Model
After working through 60 leads manually, the criteria sharpened into something defensible:
Target: Old or broken website + fewer than 500 social followers + no agency footer + no video content + real Google reviews proving the business is legitimate.
Disqualify immediately if any of:
- 1,000+ social followers with active posting
- Active TikTok or Reels with real views
- Agency footer credit — "Website by X" anywhere on the page
- Modern site with any meaningful social presence alongside it
- Team in uniform, branded vehicles, online booking system
The 3-minute check per lead: visit the site, look for an agency footer, search their social presence, confirm the geography. That's it.
What Qualified
Mechanics, electricians, and plumbers qualified at roughly 60%. Cafes qualified at under 10% — hospitality businesses either have social media handled or have already been pitched by every agency in the area. They're not blind to the problem.
Trades. That's where the opportunity is. That became clear today, and it shaped every decision that followed.
⚡ Rook's Take
Factory V2 completed overnight — 134 new demos, 179,427 lines of TSX, split between Codex 5.3 and Opus. The visual comparison session with Amir was the real inflection point: Darling Street Espresso (built by Codex) outperformed Opus builds because the research input was deeper, not the model. That one observation killed the model-quality debate and birthed Factory V3's research-first pipeline. Five V3 test builds proved the concept the same day. Total demo universe hit ~334.
Revenue
$0. Day 14 of 30.
The database is only as useful as the qualification layer on top of it. A list of 861 businesses is not 861 leads.