Files
freno-dev/src/server/analytics-schema.ts
Paperclip Agent 39d874a7e2 analytics: make each weekly funnel stage separately measurable (FRE-366)
Gap 1 — click and served share one label: trackDownload now takes a
stage ("click" | "served") carried on a new VisitorAnalytics.stage
column; the Nook door and the tRPC proxy record clicks, the DMG file
route records served. summariseDownloads() and the /analytics downloads
block are untouched — the stage is a separate column, so the path
grouping cannot change.

Gap 2 — checkout start records nothing: the checkout route records a
created Stripe session through the shared trackRequest writer (same
referrer, campaign, country dimensions as every door) under its own
path, and STAGES.checkout reads it instead of the /checkout page proxy.

Gap 3 — loopback addresses, no host dimension: every writer now
resolves the client address through resolveClientIp, which trusts
x-forwarded-for only when the direct peer is the loopback production
proxy (trust boundary stated in request-client.ts), and records the
host on each row through the same site-context extractor routing uses.
stage and host reach the table through VISITOR_ANALYTICS_ADDED_COLUMNS;
the weekly job applies the same bootstrap before reading so it renders
seven stages against a database no deploy has migrated yet.

The weekly read splits downloadClick/dmgServed, scopes visits to
nook.freno.me, and computes its caveats from the window: pre-instrumentation
rows are stated and excluded, and the loopback warning now fires only
when rows recorded since the instrumentation still carry it.
2026-10-29 17:02:39 +00:00

140 lines
4.4 KiB
TypeScript

/**
* VisitorAnalytics schema — the single source of truth for the table, its
* indexes, and the column list both writers insert.
*
* Why this file exists: `VisitorAnalytics` had no `CREATE TABLE` anywhere in
* this repo. It existed only in the live Turso database, so a fresh or
* restored environment had nothing to write into — and because
* `flushAnalyticsBuffer` swallows errors by design, every insert would fail
* silently. The definition below mirrors production exactly.
*
* `session_id` is deliberately kept. It is a leftover from the removed
* analytics-page feature (added in d2ee61b, dropped from the code in
* 58d48da) — nothing reads or writes it now, and the Nook macOS app never
* touches this table at all. SQLite would let us `DROP COLUMN`, but the
* column costs nothing and the live data in it may still be wanted, so the
* schema records it rather than removing it.
*
* Pure string constants, no imports — so the parity test can apply them to
* an in-memory SQLite database without a network or environment.
*/
/**
* The table as it exists in production. `IF NOT EXISTS` makes applying this
* a no-op against the live database; it only does work on a database that
* has never had the table.
*/
export const VISITOR_ANALYTICS_DDL = `
CREATE TABLE IF NOT EXISTS VisitorAnalytics (
id TEXT PRIMARY KEY,
user_id TEXT,
path TEXT NOT NULL,
method TEXT NOT NULL,
referrer TEXT,
user_agent TEXT,
ip_address TEXT,
country TEXT,
device_type TEXT,
browser TEXT,
os TEXT,
session_id TEXT,
duration_ms INTEGER,
created_at TEXT NOT NULL DEFAULT (datetime('now')),
fcp REAL,
lcp REAL,
cls REAL,
fid REAL,
inp REAL,
ttfb REAL,
dom_load REAL,
load_complete REAL,
utm_source TEXT,
utm_medium TEXT,
utm_campaign TEXT,
stage TEXT,
host TEXT,
FOREIGN KEY (user_id) REFERENCES User(id) ON DELETE SET NULL
)
`;
/** Indexes the summary, funnel and path queries rely on. */
export const VISITOR_ANALYTICS_INDEXES = [
`CREATE INDEX IF NOT EXISTS idx_analytics_user_id ON VisitorAnalytics(user_id)`,
`CREATE INDEX IF NOT EXISTS idx_analytics_created_at ON VisitorAnalytics(created_at)`,
`CREATE INDEX IF NOT EXISTS idx_analytics_path ON VisitorAnalytics(path)`,
`CREATE INDEX IF NOT EXISTS idx_analytics_ip_address ON VisitorAnalytics(ip_address)`,
`CREATE INDEX IF NOT EXISTS idx_analytics_session_id ON VisitorAnalytics(session_id)`,
`CREATE INDEX IF NOT EXISTS idx_analytics_lcp ON VisitorAnalytics(lcp) WHERE lcp IS NOT NULL`,
`CREATE INDEX IF NOT EXISTS idx_analytics_fcp ON VisitorAnalytics(fcp) WHERE fcp IS NOT NULL`
];
/** Columns that may be missing from a table created before they existed. */
export const VISITOR_ANALYTICS_ADDED_COLUMNS = [
{
name: "utm_source",
ddl: `ALTER TABLE VisitorAnalytics ADD COLUMN utm_source TEXT`
},
{
name: "utm_medium",
ddl: `ALTER TABLE VisitorAnalytics ADD COLUMN utm_medium TEXT`
},
{
name: "utm_campaign",
ddl: `ALTER TABLE VisitorAnalytics ADD COLUMN utm_campaign TEXT`
},
{
// Which funnel stage a download row is: "click" (a door) or "served"
// (the file route). Null on every non-download row (FRE-366).
name: "stage",
ddl: `ALTER TABLE VisitorAnalytics ADD COLUMN stage TEXT`
},
{
// The Host the client addressed, e.g. "nook.freno.me" — the site
// dimension the table never had (FRE-366).
name: "host",
ddl: `ALTER TABLE VisitorAnalytics ADD COLUMN host TEXT`
}
];
/**
* Every column both writers insert, in order. The parity test asserts these
* all exist on the table, which is what stops the DDL and the INSERT from
* drifting apart again.
*/
export const VISITOR_ANALYTICS_COLUMNS = [
"id",
"user_id",
"path",
"method",
"referrer",
"user_agent",
"ip_address",
"country",
"device_type",
"browser",
"os",
"utm_source",
"utm_medium",
"utm_campaign",
"duration_ms",
"fcp",
"lcp",
"cls",
"fid",
"inp",
"ttfb",
"dom_load",
"load_complete",
"stage",
"host"
] as const;
/**
* The one INSERT statement used by both analytics writers — the buffered
* writer in `analytics.ts` and the page-visit path in the tRPC router.
* Shared so the two can never disagree about column order or count.
*/
export const VISITOR_ANALYTICS_INSERT = `INSERT INTO VisitorAnalytics (
${VISITOR_ANALYTICS_COLUMNS.join(", ")}
) VALUES (${VISITOR_ANALYTICS_COLUMNS.map(() => "?").join(", ")})`;