metrics: stand up the weekly funnel job (FRE-202)

`marketing/metrics/weekly.md` has been a placeholder since the marketing tree
landed — it promised a funnel and the one thing that moved, and nothing
produced it. This adds the producer: one command reads the live databases and
rewrites the file.

`scripts/weekly-metrics.ts` writes `marketing/metrics/weekly.md` from
`VisitorAnalytics` (visits, downloads, campaigns) and the-nook's `trials`,
`licenses`, `activations` (the paid funnel). It needs four env vars —
TURSO_DB_URL/TOKEN and NOOK_DB_URL/TOKEN — and imports no server env, so the
job runs from a checkout that has no Stripe or Apple secrets.
`src/server/weekly-metrics.ts` holds the queries and the renderer, taking both
connections as arguments; the tests run them against an in-memory SQLite shaped
like production, the same way the backfill suite does.

The campaign breakdown groups exactly as `getAnalyticsSummary.topCampaigns`
does (top 10, untagged excluded), so the weekly read and /analytics cannot
drift into different label sets.

The first real report says more about the instrumentation than about demand,
and the renderer computes that rather than asserting it:

- every addressed row the table has ever held (12,035 of 12,048) is loopback,
  so "unique visitors" is one host and per-channel reach cannot be read;
- the landing door and the file route both write `/api/downloads/thenook`, so
  "download click" and "DMG served" are one label — one served click writes two
  rows, a failed click one, a Sparkle update one;
- `POST /api/the-nook/checkout` records nothing, so checkout start is only a
  page-view proxy;
- `VisitorAnalytics` has no host dimension, so the Nook's `/` is freno.me's `/`.

Those three gaps are reported beside the numbers, not papered over. Week 0
(2026-10-22 → 2026-10-29): visit 61, download 156, trial 2, checkout 0,
purchase 0, activation 0 — and the one thing that moved is the download stage,
0 → 156, because the label was introduced that week.

Tests: `bun test src/server/weekly-metrics.test.ts` — 12 pass, 0 fail. The full
suite's 15 failures are pre-existing: this checkout's .env lacks
NOOK_LICENSE_PRIVATE_KEY, NOOK_STRIPE_SK/WEBHOOK/PRICE and LINEAGE_JWT_SECRET,
and every failure is that env validation.

Scripts/ is gitignored, so `scripts/weekly-metrics.ts` was added with `-f`, the
way the existing tracked scripts are.

FRE-202
This commit is contained in:
Paperclip Agent
2026-10-29 15:58:51 +00:00
parent d44133fcb3
commit 89f56fc78a
6 changed files with 1184 additions and 2 deletions

View File

@@ -1,4 +1,51 @@
# Weekly funnel snapshot
Generated weekly from the analytics DB + `trials`/`licenses`/`activations`.
Format: the funnel numbers, what moved, and the one thing worth acting on.
Window: **2026-10-22 16:00:00 → 2026-10-29 16:00:00 UTC** (7 days). Previous: 2026-10-15 16:00:00 → 2026-10-22 16:00:00 UTC.
Generated by `bun --env-file=.env scripts/weekly-metrics.ts`; regenerated weekly, never hand-edited. The analysis is the CMO's.
## Funnel
| Stage | This week | Last week | Change |
|---|---:|---:|---:|
| visit (page views) | 61 | 56 | +5 |
| download click → DMG served (one label) † | 156 | 0 | +156 |
| trial started | 2 | 0 | +2 |
| checkout page visit † | 0 | 0 | 0 |
| purchase (licence issued) | 0 | 0 | 0 |
| activation | 0 | 0 | 0 |
## Campaigns
Top 10 by volume, untagged excluded — the same grouping `getAnalyticsSummary.topCampaigns` returns.
| Source | Medium | Campaign | Rows |
|---|---|---|---:|
| hn | post | launch | 2 |
| direct | — | deploycheck-direct | 1 |
| hn | — | flushproof | 1 |
| hn | post | deploycheck-ref | 1 |
| r_macapps | — | week2 | 1 |
| x | — | — | 1 |
7 of 356 rows in the window carry campaign tags; the rest are untagged and therefore unattributed.
## The one thing that moved
**download click → DMG served (one label): 0 → 156 (+156).**
## Before acting on these numbers
- **356 of 356 rows in the window came from a loopback address** (127.0.0.1 / ::1); every one of the 12035 addressed rows the table has ever held is loopback (13 carry no address). Either this host is the only audience, or the app sits behind a local proxy and `getRequestIP` records the proxy, not the visitor. Either way the address identifies nobody — "unique visitors" collapses to one host and per-channel reach cannot be read from it.
- **The download stage is one label, not two.** The landing-door route and the file route that serves the DMG both write `/api/downloads/thenook`, so "download click → DMG served" cannot be split: one served click writes two rows, a click that fails to resolve writes one, and a Sparkle update writes one. Of this window's 156 download rows, 156 are loopback and 117 are not browser user agents (curl, Sparkle, link-preview crawlers).
- **Checkout start is not recorded.** `POST /api/the-nook/checkout` — the call that actually creates the Stripe session — writes nothing to analytics; the only trace is the /checkout page view reported above, which is a page, not a started checkout.
- **There is no host dimension.** `VisitorAnalytics` records a path, not the site, so the Nook's landing page (`/` on nook.freno.me) is the same row as freno.me's root. "visit" above is site-wide page views, not Nook visits.
- **No conversion rate before the cohort matures.** Trial→paid belongs to the mature Day 28–35 read (FRE-197); a ratio from a partial cohort is not a metric.
† Proxy or merged label — see *Before acting on these numbers* above.
## Sources
- `VisitorAnalytics` (freno.me Turso DB) — visits, downloads, campaigns.
- `trials`, `licenses`, `activations` (the-nook Turso DB) — the paid funnel.
- Generated 2026-10-29T16:00:00Z.