forked from Mike/freno-dev
metrics: stand up the weekly funnel job (FRE-202)
`marketing/metrics/weekly.md` has been a placeholder since the marketing tree landed — it promised a funnel and the one thing that moved, and nothing produced it. This adds the producer: one command reads the live databases and rewrites the file. `scripts/weekly-metrics.ts` writes `marketing/metrics/weekly.md` from `VisitorAnalytics` (visits, downloads, campaigns) and the-nook's `trials`, `licenses`, `activations` (the paid funnel). It needs four env vars — TURSO_DB_URL/TOKEN and NOOK_DB_URL/TOKEN — and imports no server env, so the job runs from a checkout that has no Stripe or Apple secrets. `src/server/weekly-metrics.ts` holds the queries and the renderer, taking both connections as arguments; the tests run them against an in-memory SQLite shaped like production, the same way the backfill suite does. The campaign breakdown groups exactly as `getAnalyticsSummary.topCampaigns` does (top 10, untagged excluded), so the weekly read and /analytics cannot drift into different label sets. The first real report says more about the instrumentation than about demand, and the renderer computes that rather than asserting it: - every addressed row the table has ever held (12,035 of 12,048) is loopback, so "unique visitors" is one host and per-channel reach cannot be read; - the landing door and the file route both write `/api/downloads/thenook`, so "download click" and "DMG served" are one label — one served click writes two rows, a failed click one, a Sparkle update one; - `POST /api/the-nook/checkout` records nothing, so checkout start is only a page-view proxy; - `VisitorAnalytics` has no host dimension, so the Nook's `/` is freno.me's `/`. Those three gaps are reported beside the numbers, not papered over. Week 0 (2026-10-22 → 2026-10-29): visit 61, download 156, trial 2, checkout 0, purchase 0, activation 0 — and the one thing that moved is the download stage, 0 → 156, because the label was introduced that week. Tests: `bun test src/server/weekly-metrics.test.ts` — 12 pass, 0 fail. The full suite's 15 failures are pre-existing: this checkout's .env lacks NOOK_LICENSE_PRIVATE_KEY, NOOK_STRIPE_SK/WEBHOOK/PRICE and LINEAGE_JWT_SECRET, and every failure is that env validation. Scripts/ is gitignored, so `scripts/weekly-metrics.ts` was added with `-f`, the way the existing tracked scripts are. FRE-202
This commit is contained in:
Paperclip Agent
381
src/server/weekly-metrics.test.ts
Normal file
381
src/server/weekly-metrics.test.ts
Normal file
@@ -0,0 +1,381 @@
|
||||
import { beforeEach, describe, expect, it } from "bun:test";
|
||||
import { Database } from "bun:sqlite";
|
||||
import {
|
||||
VISITOR_ANALYTICS_COLUMNS,
|
||||
VISITOR_ANALYTICS_DDL,
|
||||
VISITOR_ANALYTICS_INSERT
|
||||
} from "./analytics-schema";
|
||||
import {
|
||||
NOOK_DOWNLOAD_PATH,
|
||||
getWeeklyReport,
|
||||
pickMover,
|
||||
readCampaigns,
|
||||
readFunnel,
|
||||
readTraffic,
|
||||
renderWeeklyMarkdown,
|
||||
windowsFor,
|
||||
type Connections,
|
||||
type FunnelCounts,
|
||||
type MetricsConn
|
||||
} from "./weekly-metrics";
|
||||
|
||||
// Same reasoning as the backfill suite: the queries run real SQL, so an
|
||||
// in-memory SQLite shaped like production is the only honest stand-in. A stub
|
||||
// that returned canned rows would pass with a column or a label the live table
|
||||
// does not have.
|
||||
|
||||
const NOOK_DDL = [
|
||||
`CREATE TABLE trials (
|
||||
fingerprint TEXT PRIMARY KEY,
|
||||
started_at TEXT NOT NULL,
|
||||
email TEXT
|
||||
)`,
|
||||
`CREATE TABLE licenses (
|
||||
id TEXT PRIMARY KEY,
|
||||
key TEXT UNIQUE NOT NULL,
|
||||
email TEXT NOT NULL,
|
||||
stripe_session_id TEXT UNIQUE NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
revoked INTEGER NOT NULL DEFAULT 0
|
||||
)`,
|
||||
`CREATE TABLE activations (
|
||||
id TEXT PRIMARY KEY,
|
||||
license_id TEXT NOT NULL,
|
||||
device_fingerprint TEXT NOT NULL,
|
||||
device_name TEXT NOT NULL,
|
||||
activated_at TEXT NOT NULL,
|
||||
deactivated_at TEXT
|
||||
)`
|
||||
];
|
||||
|
||||
function adapter(db: Database): MetricsConn {
|
||||
return {
|
||||
async execute(query) {
|
||||
const args = (query.args ?? []) as never[];
|
||||
const statement = db.query(query.sql);
|
||||
if (/^\s*(select|pragma|with)\b/i.test(query.sql)) {
|
||||
return { rows: statement.all(...args) as unknown[], rowsAffected: 0 };
|
||||
}
|
||||
return {
|
||||
rows: [] as unknown[],
|
||||
rowsAffected: statement.run(...args).changes
|
||||
};
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
let analyticsDb: Database;
|
||||
let nookDb: Database;
|
||||
let conns: Connections;
|
||||
|
||||
const NOW = new Date("2026-10-29T16:00:00Z");
|
||||
|
||||
const row = (
|
||||
overrides: Record<string, unknown>
|
||||
): Record<string, unknown> => ({
|
||||
id: crypto.randomUUID(),
|
||||
user_id: null,
|
||||
path: "/",
|
||||
method: "GET",
|
||||
referrer: null,
|
||||
user_agent: "Mozilla/5.0 (Macintosh)",
|
||||
ip_address: "203.0.113.9",
|
||||
country: "GB",
|
||||
device_type: "desktop",
|
||||
browser: "safari",
|
||||
os: "macos",
|
||||
utm_source: null,
|
||||
utm_medium: null,
|
||||
utm_campaign: null,
|
||||
duration_ms: null,
|
||||
fcp: null,
|
||||
lcp: null,
|
||||
cls: null,
|
||||
fid: null,
|
||||
inp: null,
|
||||
ttfb: null,
|
||||
dom_load: null,
|
||||
load_complete: null,
|
||||
...overrides
|
||||
});
|
||||
|
||||
function insertAnalytics(overrides: Record<string, unknown>): void {
|
||||
const values = row(overrides);
|
||||
analyticsDb
|
||||
.prepare(VISITOR_ANALYTICS_INSERT)
|
||||
.run(...(VISITOR_ANALYTICS_COLUMNS.map((column) => values[column]) as never[]));
|
||||
|
||||
// `created_at` is not in VISITOR_ANALYTICS_COLUMNS (it defaults to now), so
|
||||
// the window each row belongs to has to be set after the insert.
|
||||
if (values.created_at) {
|
||||
analyticsDb
|
||||
.prepare(`UPDATE VisitorAnalytics SET created_at = ? WHERE id = ?`)
|
||||
.run(values.created_at as string, values.id as string);
|
||||
}
|
||||
}
|
||||
|
||||
// Inside the current window for NOW - 7 days: 2026-10-22 16:00 → 2026-10-29 16:00.
|
||||
const THIS_WEEK = "2026-10-27 10:00:00";
|
||||
// Inside the previous window: 2026-10-15 16:00 → 2026-10-22 16:00.
|
||||
const LAST_WEEK = "2026-10-18 10:00:00";
|
||||
|
||||
beforeEach(() => {
|
||||
analyticsDb = new Database(":memory:");
|
||||
analyticsDb.run(VISITOR_ANALYTICS_DDL);
|
||||
nookDb = new Database(":memory:");
|
||||
for (const ddl of NOOK_DDL) nookDb.run(ddl);
|
||||
conns = { analytics: adapter(analyticsDb), nook: adapter(nookDb) };
|
||||
});
|
||||
|
||||
describe("windowsFor", () => {
|
||||
it("puts the previous window directly before the current one", () => {
|
||||
const windows = windowsFor(NOW, 7);
|
||||
expect(windows.current).toEqual({
|
||||
start: "2026-10-22 16:00:00",
|
||||
end: "2026-10-29 16:00:00"
|
||||
});
|
||||
expect(windows.previous).toEqual({
|
||||
start: "2026-10-15 16:00:00",
|
||||
end: "2026-10-22 16:00:00"
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("readFunnel", () => {
|
||||
it("counts each stage inside its window and nothing outside it", async () => {
|
||||
// Page views count; API calls do not.
|
||||
insertAnalytics({ path: "/", created_at: THIS_WEEK });
|
||||
insertAnalytics({ path: "/downloads", created_at: THIS_WEEK });
|
||||
insertAnalytics({ path: "/api/trpc/anything", created_at: THIS_WEEK });
|
||||
insertAnalytics({ path: "/", created_at: LAST_WEEK });
|
||||
|
||||
// The merged download label, click and served alike.
|
||||
insertAnalytics({ path: NOOK_DOWNLOAD_PATH, created_at: THIS_WEEK });
|
||||
insertAnalytics({ path: NOOK_DOWNLOAD_PATH, created_at: THIS_WEEK });
|
||||
|
||||
// Checkout page doors, including the legacy /nook/checkout path.
|
||||
insertAnalytics({ path: "/checkout", created_at: THIS_WEEK });
|
||||
insertAnalytics({ path: "/nook/checkout", created_at: THIS_WEEK });
|
||||
|
||||
// The licence DB stores ISO-8601 with milliseconds.
|
||||
nookDb
|
||||
.prepare(`INSERT INTO trials (fingerprint, started_at) VALUES (?, ?)`)
|
||||
.run("fp-1", "2026-10-28T20:28:46.806Z");
|
||||
nookDb
|
||||
.prepare(`INSERT INTO trials (fingerprint, started_at) VALUES (?, ?)`)
|
||||
.run("fp-old", "2026-08-26T19:26:12.791Z");
|
||||
nookDb
|
||||
.prepare(
|
||||
`INSERT INTO licenses (id, key, email, stripe_session_id, created_at)
|
||||
VALUES (?, ?, ?, ?, ?)`
|
||||
)
|
||||
.run("lic-1", "NOOK-1", "buyer@example.com", "cs_1", "2026-10-25T12:00:00.000Z");
|
||||
nookDb
|
||||
.prepare(
|
||||
`INSERT INTO activations (id, license_id, device_fingerprint, device_name, activated_at)
|
||||
VALUES (?, ?, ?, ?, ?)`
|
||||
)
|
||||
.run("act-1", "lic-1", "device-1", "Hermes", "2026-10-25T12:05:00.000Z");
|
||||
|
||||
const window = windowsFor(NOW, 7).current;
|
||||
const funnel = await readFunnel(conns, window);
|
||||
|
||||
// "/checkout" and "/nook/checkout" are page views too, so they land in
|
||||
// both the visit count and the checkout count.
|
||||
expect(funnel).toEqual({
|
||||
visit: 4,
|
||||
download: 2,
|
||||
trial: 1,
|
||||
checkout: 2,
|
||||
purchase: 1,
|
||||
activation: 1
|
||||
});
|
||||
});
|
||||
|
||||
it("reports zeros against an empty database", async () => {
|
||||
const window = windowsFor(NOW, 7).current;
|
||||
expect(await readFunnel(conns, window)).toEqual({
|
||||
visit: 0,
|
||||
download: 0,
|
||||
trial: 0,
|
||||
checkout: 0,
|
||||
purchase: 0,
|
||||
activation: 0
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("readTraffic", () => {
|
||||
it("separates loopback and automation from browser traffic", async () => {
|
||||
insertAnalytics({
|
||||
path: NOOK_DOWNLOAD_PATH,
|
||||
ip_address: "127.0.0.1",
|
||||
user_agent: "curl/8.7.1",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({
|
||||
path: NOOK_DOWNLOAD_PATH,
|
||||
ip_address: "::1",
|
||||
user_agent: "The Nook/0.13.4 Sparkle/2.9.6",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({
|
||||
path: NOOK_DOWNLOAD_PATH,
|
||||
ip_address: "203.0.113.9",
|
||||
user_agent: "Mozilla/5.0 (Macintosh) Safari/605.1.15",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({ path: "/", ip_address: "203.0.113.9", created_at: THIS_WEEK });
|
||||
|
||||
const traffic = await readTraffic(conns, windowsFor(NOW, 7).current);
|
||||
|
||||
expect(traffic).toEqual({
|
||||
analyticsRows: 4,
|
||||
loopbackRows: 2,
|
||||
allTimeRows: 4,
|
||||
allTimeAddressedRows: 4,
|
||||
allTimeLoopbackRows: 2,
|
||||
downloadRows: 3,
|
||||
downloadLoopbackRows: 2,
|
||||
downloadNonBrowserRows: 2
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe("readCampaigns", () => {
|
||||
it("groups tagged rows, excludes untagged, and orders by volume", async () => {
|
||||
insertAnalytics({
|
||||
path: "/",
|
||||
utm_source: "hn",
|
||||
utm_medium: "post",
|
||||
utm_campaign: "launch",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({
|
||||
path: "/",
|
||||
utm_source: "hn",
|
||||
utm_medium: "post",
|
||||
utm_campaign: "launch",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({
|
||||
path: "/",
|
||||
utm_source: "r_macapps",
|
||||
utm_campaign: "week2",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({ path: "/", created_at: THIS_WEEK });
|
||||
|
||||
const campaigns = await readCampaigns(conns.analytics, windowsFor(NOW, 7).current);
|
||||
|
||||
expect(campaigns).toEqual([
|
||||
{ source: "hn", medium: "post", campaign: "launch", count: 2 },
|
||||
{ source: "r_macapps", medium: null, campaign: "week2", count: 1 }
|
||||
]);
|
||||
});
|
||||
|
||||
it("caps the list at ten rows", async () => {
|
||||
for (let i = 0; i < 12; i += 1) {
|
||||
insertAnalytics({
|
||||
path: "/",
|
||||
utm_source: `source-${i}`,
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
}
|
||||
|
||||
const campaigns = await readCampaigns(conns.analytics, windowsFor(NOW, 7).current);
|
||||
expect(campaigns.length).toBe(10);
|
||||
});
|
||||
});
|
||||
|
||||
describe("pickMover", () => {
|
||||
const base: FunnelCounts = {
|
||||
visit: 0,
|
||||
download: 0,
|
||||
trial: 0,
|
||||
checkout: 0,
|
||||
purchase: 0,
|
||||
activation: 0
|
||||
};
|
||||
|
||||
it("picks the stage with the largest absolute change", () => {
|
||||
const mover = pickMover(
|
||||
{ ...base, visit: 61, download: 156, trial: 2 },
|
||||
{ ...base, visit: 56 }
|
||||
);
|
||||
expect(mover).toEqual({
|
||||
key: "download",
|
||||
label: "download click → DMG served (one label)",
|
||||
now: 156,
|
||||
before: 0,
|
||||
delta: 156
|
||||
});
|
||||
});
|
||||
|
||||
it("breaks ties toward the earlier stage", () => {
|
||||
const mover = pickMover(
|
||||
{ ...base, visit: 5, download: 5 },
|
||||
{ ...base, visit: 0, download: 0 }
|
||||
);
|
||||
expect(mover?.key).toBe("visit");
|
||||
});
|
||||
|
||||
it("returns null when nothing moved", () => {
|
||||
expect(pickMover({ ...base, visit: 3 }, { ...base, visit: 3 })).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("getWeeklyReport / renderWeeklyMarkdown", () => {
|
||||
it("reports a week in the agreed stage order with the computed caveats", async () => {
|
||||
insertAnalytics({ path: "/", created_at: THIS_WEEK });
|
||||
insertAnalytics({
|
||||
path: NOOK_DOWNLOAD_PATH,
|
||||
ip_address: "127.0.0.1",
|
||||
user_agent: "curl/8.7.1",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
insertAnalytics({
|
||||
path: NOOK_DOWNLOAD_PATH,
|
||||
ip_address: "127.0.0.1",
|
||||
user_agent: "curl/8.7.1",
|
||||
created_at: THIS_WEEK
|
||||
});
|
||||
nookDb
|
||||
.prepare(`INSERT INTO trials (fingerprint, started_at) VALUES (?, ?)`)
|
||||
.run("fp-1", "2026-10-28T20:28:46.806Z");
|
||||
|
||||
const report = await getWeeklyReport(conns, { now: NOW, days: 7 });
|
||||
const markdown = renderWeeklyMarkdown(report);
|
||||
|
||||
expect(report.mover?.key).toBe("download");
|
||||
expect(markdown).toContain(
|
||||
"Window: **2026-10-22 16:00:00 → 2026-10-29 16:00:00 UTC** (7 days)"
|
||||
);
|
||||
expect(markdown).toContain("| visit (page views) | 1 | 0 | +1 |");
|
||||
expect(markdown.indexOf("visit (page views)")).toBeLessThan(
|
||||
markdown.indexOf("download click → DMG served")
|
||||
);
|
||||
expect(markdown.indexOf("download click → DMG served")).toBeLessThan(
|
||||
markdown.indexOf("trial started")
|
||||
);
|
||||
expect(markdown).toContain("**download click → DMG served (one label): 0 → 2 (+2).**");
|
||||
expect(markdown).toContain("2 of 3 rows in the window came from a loopback address");
|
||||
expect(markdown).toContain("Checkout start is not recorded");
|
||||
expect(markdown).toContain("There is no host dimension");
|
||||
});
|
||||
|
||||
it("says nothing moved rather than inventing a winner", async () => {
|
||||
const report = await getWeeklyReport(conns, { now: NOW, days: 7 });
|
||||
const markdown = renderWeeklyMarkdown(report);
|
||||
|
||||
expect(report.mover).toBeNull();
|
||||
expect(markdown).toContain("_Nothing moved: every stage held its previous-week value._");
|
||||
expect(markdown).toContain("_No tagged traffic in the window._");
|
||||
});
|
||||
|
||||
it("rejects a window that is not a positive number of days", async () => {
|
||||
await expect(getWeeklyReport(conns, { now: NOW, days: 0 })).rejects.toThrow(
|
||||
"days must be a positive number"
|
||||
);
|
||||
});
|
||||
});
|
||||
591
src/server/weekly-metrics.ts
Normal file
591
src/server/weekly-metrics.ts
Normal file
@@ -0,0 +1,591 @@
|
||||
/**
|
||||
* The weekly funnel reading: `marketing/metrics/weekly.md`'s data half.
|
||||
*
|
||||
* The funnel the marketing cluster agreed on is
|
||||
*
|
||||
* visit → download click → DMG served → trial started → checkout started
|
||||
* → purchase → activation
|
||||
*
|
||||
* and this module reads it from the two live databases, plus the campaign
|
||||
* dimension the /analytics page already exposes through
|
||||
* `getAnalyticsSummary.topCampaigns`.
|
||||
*
|
||||
* Two properties matter more than the numbers:
|
||||
*
|
||||
* - It takes both connections as arguments. `~/server/analytics` imports the
|
||||
* full server env (Stripe keys, Apple secret, licence key …); the weekly
|
||||
* job only needs the two databases, and must be runnable from a checkout
|
||||
* whose `.env` carries nothing else. The SQL here is the same shape as the
|
||||
* analytics summaries, so the weekly read and the dashboard cannot drift
|
||||
* into different label sets.
|
||||
* - Every stage carries what it actually measured. Two of the seven agreed
|
||||
* stages share one label in production today and a third is only a page-view
|
||||
* proxy; the report states that next to the number instead of inventing a
|
||||
* figure (see `renderWeeklyMarkdown`).
|
||||
*
|
||||
* No `~/env`, no `~/server/database`: the caller owns the connections, the
|
||||
* tests own an in-memory SQLite, and the script owns the libsql clients.
|
||||
*/
|
||||
|
||||
import { DOWNLOAD_PATH_PREFIX } from "~/lib/download-asset";
|
||||
|
||||
/** The one asset key the Nook lands under — `downloadPath("thenook")`. */
|
||||
export const NOOK_DOWNLOAD_PATH = `${DOWNLOAD_PATH_PREFIX}thenook`;
|
||||
|
||||
/** An `execute`-shaped connection: the libsql client and the test adapter both fit. */
|
||||
export interface MetricsConn {
|
||||
execute(query: {
|
||||
sql: string;
|
||||
args?: any[];
|
||||
}): Promise<{ rows: unknown[]; rowsAffected?: number }>;
|
||||
}
|
||||
|
||||
export interface Connections {
|
||||
/** freno.me Turso DB — VisitorAnalytics. */
|
||||
analytics: MetricsConn;
|
||||
/** the-nook Turso DB — trials, licenses, activations. */
|
||||
nook: MetricsConn;
|
||||
}
|
||||
|
||||
export type StageKey =
|
||||
| "visit"
|
||||
| "download"
|
||||
| "trial"
|
||||
| "checkout"
|
||||
| "purchase"
|
||||
| "activation";
|
||||
|
||||
export interface StageDef {
|
||||
key: StageKey;
|
||||
label: string;
|
||||
/** True when the number is a proxy or a merged label, not a clean stage. */
|
||||
approximate: boolean;
|
||||
}
|
||||
|
||||
/**
|
||||
* The seven agreed stages. `download` deliberately carries two of them: the
|
||||
* door and the file route both write `/api/downloads/thenook`, so a click and
|
||||
* the DMG it fetched are one label in the data (proved in the report's caveats).
|
||||
*/
|
||||
export const STAGES: StageDef[] = [
|
||||
{ key: "visit", label: "visit (page views)", approximate: false },
|
||||
{
|
||||
key: "download",
|
||||
label: "download click → DMG served (one label)",
|
||||
approximate: true
|
||||
},
|
||||
{ key: "trial", label: "trial started", approximate: false },
|
||||
{
|
||||
key: "checkout",
|
||||
label: "checkout page visit",
|
||||
approximate: true
|
||||
},
|
||||
{ key: "purchase", label: "purchase (licence issued)", approximate: false },
|
||||
{ key: "activation", label: "activation", approximate: false }
|
||||
];
|
||||
|
||||
export interface FunnelCounts {
|
||||
visit: number;
|
||||
download: number;
|
||||
trial: number;
|
||||
checkout: number;
|
||||
purchase: number;
|
||||
activation: number;
|
||||
}
|
||||
|
||||
export interface CampaignRow {
|
||||
source: string | null;
|
||||
medium: string | null;
|
||||
campaign: string | null;
|
||||
count: number;
|
||||
}
|
||||
|
||||
/** Traffic-shape facts the report has to state next to the funnel. */
|
||||
export interface WindowTraffic {
|
||||
analyticsRows: number;
|
||||
loopbackRows: number;
|
||||
/** All-time row counts, so "every row ever" is a checked claim, not a guess. */
|
||||
allTimeRows: number;
|
||||
allTimeAddressedRows: number;
|
||||
allTimeLoopbackRows: number;
|
||||
downloadRows: number;
|
||||
downloadLoopbackRows: number;
|
||||
downloadNonBrowserRows: number;
|
||||
}
|
||||
|
||||
export interface Mover {
|
||||
key: StageKey;
|
||||
label: string;
|
||||
now: number;
|
||||
before: number;
|
||||
delta: number;
|
||||
}
|
||||
|
||||
export interface WeeklyReport {
|
||||
window: { start: string; end: string; days: number };
|
||||
previous: { start: string; end: string };
|
||||
counts: FunnelCounts;
|
||||
previousCounts: FunnelCounts;
|
||||
campaigns: CampaignRow[];
|
||||
taggedRows: number;
|
||||
traffic: WindowTraffic;
|
||||
mover: Mover | null;
|
||||
generatedAt: string;
|
||||
}
|
||||
|
||||
/** Loopback addresses — every row the live table has ever held. */
|
||||
const LOOPBACK = ["127.0.0.1", "::1", "::ffff:127.0.0.1"];
|
||||
|
||||
/** SQLite's `datetime()` string, UTC: `YYYY-MM-DD HH:MM:SS`. */
|
||||
export function sqlTime(date: Date): string {
|
||||
return date.toISOString().slice(0, 19).replace("T", " ");
|
||||
}
|
||||
|
||||
export interface Windows {
|
||||
current: { start: string; end: string };
|
||||
previous: { start: string; end: string };
|
||||
}
|
||||
|
||||
/**
|
||||
* The current window is `[now - days, now)` and the previous one sits directly
|
||||
* before it, so a weekly read always compares like with like. Boundaries are
|
||||
* passed as SQL parameters rather than `datetime('now', …)` so the file a run
|
||||
* writes is reproducible and the two queries cannot straddle a second.
|
||||
*/
|
||||
export function windowsFor(now: Date, days: number): Windows {
|
||||
const span = days * 24 * 60 * 60 * 1000;
|
||||
const end = now;
|
||||
const start = new Date(end.getTime() - span);
|
||||
const previousEnd = start;
|
||||
const previousStart = new Date(previousEnd.getTime() - span);
|
||||
|
||||
return {
|
||||
current: { start: sqlTime(start), end: sqlTime(end) },
|
||||
previous: { start: sqlTime(previousStart), end: sqlTime(previousEnd) }
|
||||
};
|
||||
}
|
||||
|
||||
function toNumber(value: unknown): number {
|
||||
if (typeof value === "number") return value;
|
||||
if (typeof value === "bigint") return Number(value);
|
||||
if (typeof value === "string") return Number(value) || 0;
|
||||
return 0;
|
||||
}
|
||||
|
||||
async function scalar(
|
||||
conn: MetricsConn,
|
||||
sql: string,
|
||||
args: any[]
|
||||
): Promise<number> {
|
||||
const result = await conn.execute({ sql, args });
|
||||
const row = result.rows[0] as Record<string, unknown> | undefined;
|
||||
if (!row) return 0;
|
||||
return toNumber(Object.values(row)[0]);
|
||||
}
|
||||
|
||||
async function countInWindow(
|
||||
conn: MetricsConn,
|
||||
sql: string,
|
||||
start: string,
|
||||
end: string
|
||||
): Promise<number> {
|
||||
return scalar(conn, sql, [start, end]);
|
||||
}
|
||||
|
||||
/** One window's funnel. Two databases, one call. */
|
||||
export async function readFunnel(
|
||||
conns: Connections,
|
||||
window: { start: string; end: string }
|
||||
): Promise<FunnelCounts> {
|
||||
const { analytics, nook } = conns;
|
||||
const { start, end } = window;
|
||||
|
||||
const visit = await countInWindow(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ?
|
||||
AND path NOT LIKE '/api/%'`,
|
||||
start,
|
||||
end
|
||||
);
|
||||
|
||||
const download = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ?
|
||||
AND path = ?`,
|
||||
[start, end, NOOK_DOWNLOAD_PATH]
|
||||
);
|
||||
|
||||
const checkout = await countInWindow(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ?
|
||||
AND (path LIKE '/checkout%' OR path LIKE '/nook/checkout%')`,
|
||||
start,
|
||||
end
|
||||
);
|
||||
|
||||
// The licence DB stores ISO-8601 (`2026-10-28T20:28:46.806Z`); the analytics
|
||||
// DB stores `YYYY-MM-DD HH:MM:SS`. `datetime()` normalises both sides so a
|
||||
// boundary seconds apart cannot include or drop a row on string comparison.
|
||||
const trial = await scalar(
|
||||
nook,
|
||||
`SELECT COUNT(*) AS n FROM trials
|
||||
WHERE datetime(started_at) >= datetime(?) AND datetime(started_at) < datetime(?)`,
|
||||
[start, end]
|
||||
);
|
||||
|
||||
const purchase = await scalar(
|
||||
nook,
|
||||
`SELECT COUNT(*) AS n FROM licenses
|
||||
WHERE datetime(created_at) >= datetime(?) AND datetime(created_at) < datetime(?)`,
|
||||
[start, end]
|
||||
);
|
||||
|
||||
const activation = await scalar(
|
||||
nook,
|
||||
`SELECT COUNT(*) AS n FROM activations
|
||||
WHERE datetime(activated_at) >= datetime(?) AND datetime(activated_at) < datetime(?)`,
|
||||
[start, end]
|
||||
);
|
||||
|
||||
return {
|
||||
visit,
|
||||
download,
|
||||
trial,
|
||||
checkout,
|
||||
purchase,
|
||||
activation
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Traffic shape for one window: how much of it came from this host, and how
|
||||
* much of the download label was automation. Without these two facts the
|
||||
* numbers read as demand, which they are not.
|
||||
*/
|
||||
export async function readTraffic(
|
||||
conns: Connections,
|
||||
window: { start: string; end: string }
|
||||
): Promise<WindowTraffic> {
|
||||
const { analytics } = conns;
|
||||
const { start, end } = window;
|
||||
const placeholders = LOOPBACK.map(() => "?").join(", ");
|
||||
|
||||
const allTimeRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics`,
|
||||
[]
|
||||
);
|
||||
|
||||
const allTimeAddressedRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics WHERE ip_address IS NOT NULL`,
|
||||
[]
|
||||
);
|
||||
|
||||
const allTimeLoopbackRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE ip_address IN (${placeholders})`,
|
||||
[...LOOPBACK]
|
||||
);
|
||||
|
||||
const analyticsRows = await countInWindow(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ?`,
|
||||
start,
|
||||
end
|
||||
);
|
||||
|
||||
const loopbackRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ?
|
||||
AND ip_address IN (${placeholders})`,
|
||||
[start, end, ...LOOPBACK]
|
||||
);
|
||||
|
||||
const downloadRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ? AND path = ?`,
|
||||
[start, end, NOOK_DOWNLOAD_PATH]
|
||||
);
|
||||
|
||||
const downloadLoopbackRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ? AND path = ?
|
||||
AND ip_address IN (${placeholders})`,
|
||||
[start, end, NOOK_DOWNLOAD_PATH, ...LOOPBACK]
|
||||
);
|
||||
|
||||
// Anything whose UA is not a browser engine: curl, Sparkle's updater,
|
||||
// link-preview crawlers. A real install click arrives as `Mozilla/5.0 …`.
|
||||
const downloadNonBrowserRows = await scalar(
|
||||
analytics,
|
||||
`SELECT COUNT(*) AS n FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ? AND path = ?
|
||||
AND (user_agent IS NULL OR user_agent NOT LIKE 'Mozilla/%')`,
|
||||
[start, end, NOOK_DOWNLOAD_PATH]
|
||||
);
|
||||
|
||||
return {
|
||||
analyticsRows,
|
||||
loopbackRows,
|
||||
allTimeRows,
|
||||
allTimeAddressedRows,
|
||||
allTimeLoopbackRows,
|
||||
downloadRows,
|
||||
downloadLoopbackRows,
|
||||
downloadNonBrowserRows
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The campaign dimension, grouped exactly as `getAnalyticsSummary.topCampaigns`
|
||||
* groups it: top 10 by volume, untagged rows excluded (the untagged bucket is
|
||||
* a different question and would otherwise swallow the list).
|
||||
*/
|
||||
export async function readCampaigns(
|
||||
analytics: MetricsConn,
|
||||
window: { start: string; end: string }
|
||||
): Promise<CampaignRow[]> {
|
||||
const result = await analytics.execute({
|
||||
sql: `SELECT utm_source, utm_medium, utm_campaign, COUNT(*) AS count
|
||||
FROM VisitorAnalytics
|
||||
WHERE created_at >= ? AND created_at < ?
|
||||
AND (utm_source IS NOT NULL OR utm_medium IS NOT NULL OR utm_campaign IS NOT NULL)
|
||||
GROUP BY utm_source, utm_medium, utm_campaign
|
||||
ORDER BY count DESC
|
||||
LIMIT 10`,
|
||||
args: [window.start, window.end]
|
||||
});
|
||||
|
||||
return result.rows.map((raw) => {
|
||||
const row = raw as Record<string, unknown>;
|
||||
return {
|
||||
source: (row.utm_source as string | null) ?? null,
|
||||
medium: (row.utm_medium as string | null) ?? null,
|
||||
campaign: (row.utm_campaign as string | null) ?? null,
|
||||
count: toNumber(row.count)
|
||||
};
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* "The one thing that moved": the stage with the largest absolute change
|
||||
* against the previous window. Ties go to the earlier stage, so a week when
|
||||
* nothing moved returns null rather than a made-up winner.
|
||||
*/
|
||||
export function pickMover(
|
||||
current: FunnelCounts,
|
||||
previous: FunnelCounts
|
||||
): Mover | null {
|
||||
let best: Mover | null = null;
|
||||
|
||||
for (const stage of STAGES) {
|
||||
const delta = current[stage.key] - previous[stage.key];
|
||||
if (delta === 0) continue;
|
||||
if (!best || Math.abs(delta) > Math.abs(best.delta)) {
|
||||
best = {
|
||||
key: stage.key,
|
||||
label: stage.label,
|
||||
now: current[stage.key],
|
||||
before: previous[stage.key],
|
||||
delta
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
return best;
|
||||
}
|
||||
|
||||
export interface ReportOptions {
|
||||
/** End of the current window; defaults to the wall clock. */
|
||||
now?: Date;
|
||||
/** Window length in days; defaults to 7. */
|
||||
days?: number;
|
||||
}
|
||||
|
||||
/** Read everything `.md` needs, in one call. */
|
||||
export async function getWeeklyReport(
|
||||
conns: Connections,
|
||||
options: ReportOptions = {}
|
||||
): Promise<WeeklyReport> {
|
||||
const days = options.days ?? 7;
|
||||
if (!Number.isFinite(days) || days <= 0) {
|
||||
throw new Error(`days must be a positive number, got ${days}`);
|
||||
}
|
||||
|
||||
const now = options.now ?? new Date();
|
||||
const windows = windowsFor(now, days);
|
||||
|
||||
const [counts, previousCounts, traffic, campaigns] = await Promise.all([
|
||||
readFunnel(conns, windows.current),
|
||||
readFunnel(conns, windows.previous),
|
||||
readTraffic(conns, windows.current),
|
||||
readCampaigns(conns.analytics, windows.current)
|
||||
]);
|
||||
|
||||
return {
|
||||
window: { ...windows.current, days },
|
||||
previous: windows.previous,
|
||||
counts,
|
||||
previousCounts,
|
||||
campaigns,
|
||||
taggedRows: campaigns.reduce((sum, row) => sum + row.count, 0),
|
||||
traffic,
|
||||
mover: pickMover(counts, previousCounts),
|
||||
generatedAt: now.toISOString().replace(/\.\d{3}Z$/, "Z")
|
||||
};
|
||||
}
|
||||
|
||||
function cell(value: string | null): string {
|
||||
return value === null || value === "" ? "—" : value;
|
||||
}
|
||||
|
||||
function signed(value: number): string {
|
||||
if (value === 0) return "0";
|
||||
return value > 0 ? `+${value}` : `−${Math.abs(value)}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Render the weekly file. Interpretation lives in the CMO's read; this renderer
|
||||
* only states what the data is, what moved, and what the numbers cannot say.
|
||||
* The Gaps section is not boilerplate: each bullet is computed from the window
|
||||
* it describes, so it empties itself if the underlying hole is ever fixed.
|
||||
*/
|
||||
export function renderWeeklyMarkdown(report: WeeklyReport): string {
|
||||
const { counts, previousCounts, traffic } = report;
|
||||
const lines: string[] = [];
|
||||
|
||||
lines.push("# Weekly funnel snapshot");
|
||||
lines.push("");
|
||||
lines.push(
|
||||
`Window: **${report.window.start} → ${report.window.end} UTC** (${report.window.days} days). ` +
|
||||
`Previous: ${report.previous.start} → ${report.previous.end} UTC.`
|
||||
);
|
||||
lines.push("");
|
||||
lines.push(
|
||||
"Generated by `bun --env-file=.env scripts/weekly-metrics.ts`; regenerated " +
|
||||
"weekly, never hand-edited. The analysis is the CMO's."
|
||||
);
|
||||
lines.push("");
|
||||
|
||||
lines.push("## Funnel");
|
||||
lines.push("");
|
||||
lines.push("| Stage | This week | Last week | Change |");
|
||||
lines.push("|---|---:|---:|---:|");
|
||||
for (const stage of STAGES) {
|
||||
const now = counts[stage.key];
|
||||
const before = previousCounts[stage.key];
|
||||
const label = stage.approximate ? `${stage.label} †` : stage.label;
|
||||
lines.push(`| ${label} | ${now} | ${before} | ${signed(now - before)} |`);
|
||||
}
|
||||
lines.push("");
|
||||
|
||||
lines.push("## Campaigns");
|
||||
lines.push("");
|
||||
if (report.campaigns.length === 0) {
|
||||
lines.push("_No tagged traffic in the window._");
|
||||
} else {
|
||||
lines.push(
|
||||
"Top 10 by volume, untagged excluded — the same grouping " +
|
||||
"`getAnalyticsSummary.topCampaigns` returns."
|
||||
);
|
||||
lines.push("");
|
||||
lines.push("| Source | Medium | Campaign | Rows |");
|
||||
lines.push("|---|---|---|---:|");
|
||||
for (const row of report.campaigns) {
|
||||
lines.push(
|
||||
`| ${cell(row.source)} | ${cell(row.medium)} | ${cell(row.campaign)} | ${row.count} |`
|
||||
);
|
||||
}
|
||||
}
|
||||
lines.push("");
|
||||
lines.push(
|
||||
`${report.taggedRows} of ${traffic.analyticsRows} rows in the window carry ` +
|
||||
`campaign tags; the rest are untagged and therefore unattributed.`
|
||||
);
|
||||
lines.push("");
|
||||
|
||||
lines.push("## The one thing that moved");
|
||||
lines.push("");
|
||||
if (!report.mover) {
|
||||
lines.push("_Nothing moved: every stage held its previous-week value._");
|
||||
} else {
|
||||
const mover = report.mover;
|
||||
lines.push(
|
||||
`**${mover.label}: ${mover.before} → ${mover.now} (${signed(mover.delta)}).**`
|
||||
);
|
||||
}
|
||||
lines.push("");
|
||||
|
||||
lines.push("## Before acting on these numbers");
|
||||
lines.push("");
|
||||
|
||||
if (traffic.analyticsRows > 0 && traffic.loopbackRows > 0) {
|
||||
lines.push(
|
||||
`- **${traffic.loopbackRows} of ${traffic.analyticsRows} rows in the window came from a loopback address** ` +
|
||||
`(127.0.0.1 / ::1)` +
|
||||
(traffic.allTimeAddressedRows > 0 &&
|
||||
traffic.allTimeAddressedRows === traffic.allTimeLoopbackRows
|
||||
? `; every one of the ${traffic.allTimeAddressedRows} addressed rows the table ` +
|
||||
`has ever held is loopback (${traffic.allTimeRows - traffic.allTimeAddressedRows} carry no address)`
|
||||
: `; across the table's history, ${traffic.allTimeLoopbackRows} of ` +
|
||||
`${traffic.allTimeAddressedRows} addressed rows are loopback`) +
|
||||
`. Either this host is the only audience, or the app sits behind a local proxy ` +
|
||||
`and \`getRequestIP\` records the proxy, not the visitor. Either way the address ` +
|
||||
`identifies nobody — "unique visitors" collapses to one host and per-channel ` +
|
||||
`reach cannot be read from it.`
|
||||
);
|
||||
}
|
||||
|
||||
if (traffic.downloadRows > 0) {
|
||||
lines.push(
|
||||
`- **The download stage is one label, not two.** The landing-door route and the file ` +
|
||||
`route that serves the DMG both write \`${NOOK_DOWNLOAD_PATH}\`, so ` +
|
||||
`"download click → DMG served" cannot be split: one served click writes two rows, ` +
|
||||
`a click that fails to resolve writes one, and a Sparkle update writes one. ` +
|
||||
`Of this window's ${traffic.downloadRows} download rows, ` +
|
||||
`${traffic.downloadLoopbackRows} are loopback and ` +
|
||||
`${traffic.downloadNonBrowserRows} are not browser user agents (curl, Sparkle, ` +
|
||||
`link-preview crawlers).`
|
||||
);
|
||||
}
|
||||
|
||||
lines.push(
|
||||
`- **Checkout start is not recorded.** \`POST /api/the-nook/checkout\` — the call that ` +
|
||||
`actually creates the Stripe session — writes nothing to analytics; the only trace is ` +
|
||||
`the /checkout page view reported above, which is a page, not a started checkout.`
|
||||
);
|
||||
|
||||
lines.push(
|
||||
`- **There is no host dimension.** \`VisitorAnalytics\` records a path, not the site, so ` +
|
||||
`the Nook's landing page (\`/\` on nook.freno.me) is the same row as freno.me's root. ` +
|
||||
`"visit" above is site-wide page views, not Nook visits.`
|
||||
);
|
||||
|
||||
lines.push(
|
||||
`- **No conversion rate before the cohort matures.** Trial→paid belongs to the mature ` +
|
||||
`Day 28–35 read (FRE-197); a ratio from a partial cohort is not a metric.`
|
||||
);
|
||||
lines.push("");
|
||||
|
||||
lines.push(
|
||||
"† Proxy or merged label — see *Before acting on these numbers* above."
|
||||
);
|
||||
lines.push("");
|
||||
lines.push("## Sources");
|
||||
lines.push("");
|
||||
lines.push("- `VisitorAnalytics` (freno.me Turso DB) — visits, downloads, campaigns.");
|
||||
lines.push("- `trials`, `licenses`, `activations` (the-nook Turso DB) — the paid funnel.");
|
||||
lines.push(`- Generated ${report.generatedAt}.`);
|
||||
lines.push("");
|
||||
|
||||
return lines.join("\n");
|
||||
}
|
||||
Reference in New Issue
Block a user