Files
PodTui/tasks/bounded-feed-lifecycle/01-persisted-retention-window.md
Michael Freno deac6081ca feat(feed): bound feed lifecycle to 30-day window with nonblocking refresh
Persisted feeds keep only episodes from the last 30 days (plus completed
downloads); older episodes live in volatile memory and survive refreshes
via union merge, with per-feed in-memory caches capped at 500. Refresh
batches run at FETCH_CONCURRENCY=4 with per-feed incremental apply (no
Promise.all barrier), config.json writes are trailing-edge debounced
(250ms, immediate flushPendingSave for unsubscribes), and cold
fetch-more refetches abort at FETCH_TIMEOUT_MS. A shared activity store
powers a global top-right indicator covering refresh, fetch-more,
subscribe, search, and downloads.

Also includes the in-flight incremental RSS parsing (chunked with
event-loop yields) and refresh spinner work this tree already carried.
2026-08-12 10:13:20 -04:00

7.9 KiB
Raw Blame History

01. Persist only a 30-day episode window, keep downloaded episodes, clean up stale data

meta: id: bounded-feed-lifecycle-01 feature: bounded-feed-lifecycle priority: P1 depends_on: [] tags: [implementation, tests-required]

objective:

  • Bound what the app writes to config.json: each persisted feed keeps only episodes published within the last 30 days, plus any episode whose download is completed — everything older lives in volatile memory only (wired up in task 02). Loading an over-window legacy config must prune it automatically (cleanup on first launch).

background (read this before touching code):

  • Feeds persist through src/utils/feeds-persistence.ts. saveFeedsToFile(feeds) is a fire-and-forget wrapper around updateConfig({ feeds }) in src/utils/config.ts, which read-modify-writes the whole config.json behind a serialized promise chain (writeChain).
  • Today saveFeedsToFile writes every loaded episode, so config.json grows forever (the Feed page's "Fetch More" keeps expanding feed.episodes and saving).
  • Downloads persist separately in downloads.json (same config dir, see src/utils/config-dir.ts getConfigFilePath("downloads.json")). Each record has episodeId, status, feedId, etc. The DownloadStatus enum lives in src/types/episode.ts — read it there for the completed member's string value; do NOT hardcode a guessed string.
  • src/stores/feed.ts calls saveFeedsToFile from a module-scope saveFeeds() helper. Callers must not change in this task.
  • Style: match the file you edit. feeds-persistence.ts and config.ts are tab-indented WITH semicolons (some other repo files aren't — don't "fix" that anywhere).

deliverables:

  • src/utils/feeds-persistence.ts:
    • New exported constant PERSISTED_WINDOW_DAYS = 30.
    • New exported pure function episodeIsPersistable(ep: Episode, downloadedIds: Set<string>, now: Date): boolean — returns true when:
      • ep.pubDate is missing/not a valid Date (fail-safe: never drop an undatable episode), OR
      • ep.pubDate.getTime() >= now.getTime() - PERSISTED_WINDOW_DAYS * 24 * 3600 * 1000, OR
      • downloadedIds.has(ep.id).
    • New (module-private) async helper readDownloadedEpisodeIds(): Promise<Set<string>> — reads getConfigFilePath("downloads.json") with Bun.file, returns the episodeIds of records whose status equals DownloadStatus.COMPLETED; returns an empty set on any error or missing file. Note: an episode whose download is merely in-flight is NOT exempted; it will be re-included by the next save after completion, since the in-memory feed.episodes still holds it — document this in the function comment.
    • saveFeedsToFile(feeds: Feed[]) — before calling updateConfig, map each feed to { ...feed, episodes: feed.episodes.filter(ep => episodeIsPersistable(ep, downloadedIds, new Date())) }. The downloaded-ids lookup is async, so wrap the whole body in a fire-and-forget async IIFE (.catch(() => {})) that preserves the existing sync/fire-and-forget signature; on any lookup failure, save the feeds unpruned (never lose data on an error path).
    • loadFeedsFromFile() — after reviveDates, apply the same prune to the loaded feeds; if the prune removed at least one episode, call saveFeedsToFile(pruned) to rewrite config.json (this is the startup cleanup for legacy configs). await the prune path deterministically (the function is already async).
  • src/utils/config.ts:
    • New exported whenConfigIdle(): Promise<void> returning the module-internal writeChain promise. Tests need a way to await pending serialized writes; today updateConfig hides the chain and tests cannot observe when a write lands.
  • tests/feed-retention.test.ts (new) — see tests section.

steps:

  1. Read src/types/episode.ts to confirm DownloadStatus.COMPLETED's runtime value and the Episode shape (id, pubDate).
  2. Read src/utils/feeds-persistence.ts and src/utils/config.ts fully (they are short).
  3. Add whenConfigIdle() to config.ts next to updateConfig.
  4. In feeds-persistence.ts: add imports (getConfigFilePath from ./config-dir, DownloadStatus and type Episode from ../types/episode), the constant, episodeIsPersistable, readDownloadedEpisodeIds, then rework saveFeedsToFile and loadFeedsFromFile per deliverables. Keep reviveDates untouched.
  5. Ensure saveFeeds in src/stores/feed.ts still compiles unchanged (signature-compatible).
  6. Write tests/feed-retention.test.ts, run it, then run the full suite and lint.

tests:

  • Conventions (copy them): tests/feed-refresh.test.ts shows the harness — mkdtempSync into process.env.XDG_CONFIG_HOME before importing anything under test (module-level init reads the config dir), rmSync in afterAll, tabs/no-semicolon style not required but match repo.
  • New tests/feed-retention.test.ts:
    • Unit (ArrangeActAssert) for episodeIsPersistable:
      • episode 40 days old, not downloaded → false.
      • episode 40 days old, id in downloadedIdstrue.
      • episode 5 days old → true.
      • episode with pubDate: new Date(NaN)true (fail-safe).
    • Save-path integration:
      • Arrange: write a downloads.json in the temp config dir containing one completed record for old-downloaded-id (include all fields the loader reads in src/stores/download.ts's DownloadRecord: at minimum episodeId, feedId, status, filePath: null, downloadedAt: null, fileSize: 0, error: null, audioUrl: "", episodeTitle: "").
      • Act: call saveFeedsToFile([feed]) where the feed has three episodes — recent, old-not-downloaded (id: "old-plain-id"), old-downloaded (id: "old-downloaded-id"). Await whenConfigIdle() (plus one more microtask/await Promise.resolve() round if the async IIFE resolves after the chain call — flush both).
      • Assert: parse config.json raw; the feed's persisted episodes contain the recent and old-downloaded-id episodes and NOT old-plain-id.
    • Load-path cleanup:
      • Arrange: seed config.json (write it directly with Bun.write) with one feed holding only over-window episodes; no downloads.json.
      • Act: await loadFeedsFromFile(), then await whenConfigIdle().
      • Assert: returned feed has zero episodes AND re-reading config.json shows the episodes pruned (cleanup rewrite happened).

acceptance_criteria:

  • saveFeedsToFile never writes an episode older than 30 days unless its id is a completed download in downloads.json.
  • loadFeedsFromFile prunes over-window episodes from legacy configs and rewrites config.json when it pruned anything.
  • Undatable episodes (pubDate missing/invalid) are always persisted.
  • No call site of saveFeedsToFile/loadFeedsFromFile needed to change (compatible signatures).
  • bun test tests/feed-retention.test.ts passes; the existing bun test suite passes; bun run lint is clean.

validation:

  • bun test tests/feed-retention.test.ts
  • bun test (full suite — watch feed-refresh/feed-pagination for regressions)
  • bun run lint
  • Manual smoke (optional): bun start, subscribe to any feed, quit, then cat ~/.config/podtui/config.json | python3 -c "import sys,json; print(max(e['pubDate'] for f in json.load(sys.stdin)['feeds'] for e in f['episodes']))" and confirm no persisted episode is older than 30 days.

notes:

  • updateConfig captures the patched data eagerly at call time (JSON.parse(JSON.stringify(patch))), so pruning in saveFeedsToFile before the updateConfig call is exactly where the filter must live — filtering later would be silently ineffective for already-queued writes.
  • This task intentionally does NOT change in-memory behavior, refresh merging, or cache bounds — that is task 02. If both are worked on in parallel, 02 imports nothing from 01 except the documented window semantics; the module-level contract above is the seam.
  • downloads.json is written by src/stores/download.ts (saveDownloads); reading it directly here avoids a store→module import cycle (download.ts already imports the feed store).