Persisted feeds keep only episodes from the last 30 days (plus completed downloads); older episodes live in volatile memory and survive refreshes via union merge, with per-feed in-memory caches capped at 500. Refresh batches run at FETCH_CONCURRENCY=4 with per-feed incremental apply (no Promise.all barrier), config.json writes are trailing-edge debounced (250ms, immediate flushPendingSave for unsubscribes), and cold fetch-more refetches abort at FETCH_TIMEOUT_MS. A shared activity store powers a global top-right indicator covering refresh, fetch-more, subscribe, search, and downloads. Also includes the in-flight incremental RSS parsing (chunked with event-loop yields) and refresh spinner work this tree already carried.
7.9 KiB
7.9 KiB
01. Persist only a 30-day episode window, keep downloaded episodes, clean up stale data
meta: id: bounded-feed-lifecycle-01 feature: bounded-feed-lifecycle priority: P1 depends_on: [] tags: [implementation, tests-required]
objective:
- Bound what the app writes to
config.json: each persisted feed keeps only episodes published within the last 30 days, plus any episode whose download is completed — everything older lives in volatile memory only (wired up in task 02). Loading an over-window legacy config must prune it automatically (cleanup on first launch).
background (read this before touching code):
- Feeds persist through
src/utils/feeds-persistence.ts.saveFeedsToFile(feeds)is a fire-and-forget wrapper aroundupdateConfig({ feeds })insrc/utils/config.ts, which read-modify-writes the wholeconfig.jsonbehind a serialized promise chain (writeChain). - Today
saveFeedsToFilewrites every loaded episode, soconfig.jsongrows forever (the Feed page's "Fetch More" keeps expandingfeed.episodesand saving). - Downloads persist separately in
downloads.json(same config dir, seesrc/utils/config-dir.tsgetConfigFilePath("downloads.json")). Each record hasepisodeId,status,feedId, etc. TheDownloadStatusenum lives insrc/types/episode.ts— read it there for the completed member's string value; do NOT hardcode a guessed string. src/stores/feed.tscallssaveFeedsToFilefrom a module-scopesaveFeeds()helper. Callers must not change in this task.- Style: match the file you edit.
feeds-persistence.tsandconfig.tsare tab-indented WITH semicolons (some other repo files aren't — don't "fix" that anywhere).
deliverables:
src/utils/feeds-persistence.ts:- New exported constant
PERSISTED_WINDOW_DAYS = 30. - New exported pure function
episodeIsPersistable(ep: Episode, downloadedIds: Set<string>, now: Date): boolean— returnstruewhen:ep.pubDateis missing/not a validDate(fail-safe: never drop an undatable episode), ORep.pubDate.getTime() >= now.getTime() - PERSISTED_WINDOW_DAYS * 24 * 3600 * 1000, ORdownloadedIds.has(ep.id).
- New (module-private) async helper
readDownloadedEpisodeIds(): Promise<Set<string>>— readsgetConfigFilePath("downloads.json")withBun.file, returns theepisodeIds of records whosestatusequalsDownloadStatus.COMPLETED; returns an empty set on any error or missing file. Note: an episode whose download is merely in-flight is NOT exempted; it will be re-included by the next save after completion, since the in-memoryfeed.episodesstill holds it — document this in the function comment. saveFeedsToFile(feeds: Feed[])— before callingupdateConfig, map each feed to{ ...feed, episodes: feed.episodes.filter(ep => episodeIsPersistable(ep, downloadedIds, new Date())) }. The downloaded-ids lookup is async, so wrap the whole body in a fire-and-forget async IIFE (.catch(() => {})) that preserves the existing sync/fire-and-forget signature; on any lookup failure, save the feeds unpruned (never lose data on an error path).loadFeedsFromFile()— afterreviveDates, apply the same prune to the loaded feeds; if the prune removed at least one episode, callsaveFeedsToFile(pruned)to rewriteconfig.json(this is the startup cleanup for legacy configs).awaitthe prune path deterministically (the function is already async).
- New exported constant
src/utils/config.ts:- New exported
whenConfigIdle(): Promise<void>returning the module-internalwriteChainpromise. Tests need a way to await pending serialized writes; todayupdateConfighides the chain and tests cannot observe when a write lands.
- New exported
tests/feed-retention.test.ts(new) — see tests section.
steps:
- Read
src/types/episode.tsto confirmDownloadStatus.COMPLETED's runtime value and theEpisodeshape (id,pubDate). - Read
src/utils/feeds-persistence.tsandsrc/utils/config.tsfully (they are short). - Add
whenConfigIdle()toconfig.tsnext toupdateConfig. - In
feeds-persistence.ts: add imports (getConfigFilePathfrom./config-dir,DownloadStatusandtype Episodefrom../types/episode), the constant,episodeIsPersistable,readDownloadedEpisodeIds, then reworksaveFeedsToFileandloadFeedsFromFileper deliverables. KeepreviveDatesuntouched. - Ensure
saveFeedsinsrc/stores/feed.tsstill compiles unchanged (signature-compatible). - Write
tests/feed-retention.test.ts, run it, then run the full suite and lint.
tests:
- Conventions (copy them):
tests/feed-refresh.test.tsshows the harness —mkdtempSyncintoprocess.env.XDG_CONFIG_HOMEbefore importing anything under test (module-level init reads the config dir),rmSyncinafterAll, tabs/no-semicolon style not required but match repo. - New
tests/feed-retention.test.ts:- Unit (Arrange–Act–Assert) for
episodeIsPersistable:- episode 40 days old, not downloaded →
false. - episode 40 days old, id in
downloadedIds→true. - episode 5 days old →
true. - episode with
pubDate: new Date(NaN)→true(fail-safe).
- episode 40 days old, not downloaded →
- Save-path integration:
- Arrange: write a
downloads.jsonin the temp config dir containing onecompletedrecord forold-downloaded-id(include all fields the loader reads insrc/stores/download.ts'sDownloadRecord: at minimumepisodeId,feedId,status,filePath: null,downloadedAt: null,fileSize: 0,error: null,audioUrl: "",episodeTitle: ""). - Act: call
saveFeedsToFile([feed])where the feed has three episodes — recent, old-not-downloaded (id: "old-plain-id"), old-downloaded (id: "old-downloaded-id"). AwaitwhenConfigIdle()(plus one more microtask/await Promise.resolve()round if the async IIFE resolves after the chain call — flush both). - Assert: parse
config.jsonraw; the feed's persistedepisodescontain the recent andold-downloaded-idepisodes and NOTold-plain-id.
- Arrange: write a
- Load-path cleanup:
- Arrange: seed
config.json(write it directly withBun.write) with one feed holding only over-window episodes; nodownloads.json. - Act:
await loadFeedsFromFile(), thenawait whenConfigIdle(). - Assert: returned feed has zero episodes AND re-reading
config.jsonshows the episodes pruned (cleanup rewrite happened).
- Arrange: seed
- Unit (Arrange–Act–Assert) for
acceptance_criteria:
saveFeedsToFilenever writes an episode older than 30 days unless its id is a completed download indownloads.json.loadFeedsFromFileprunes over-window episodes from legacy configs and rewritesconfig.jsonwhen it pruned anything.- Undatable episodes (
pubDatemissing/invalid) are always persisted. - No call site of
saveFeedsToFile/loadFeedsFromFileneeded to change (compatible signatures). bun test tests/feed-retention.test.tspasses; the existingbun testsuite passes;bun run lintis clean.
validation:
bun test tests/feed-retention.test.tsbun test(full suite — watchfeed-refresh/feed-paginationfor regressions)bun run lint- Manual smoke (optional):
bun start, subscribe to any feed, quit, thencat ~/.config/podtui/config.json | python3 -c "import sys,json; print(max(e['pubDate'] for f in json.load(sys.stdin)['feeds'] for e in f['episodes']))"and confirm no persisted episode is older than 30 days.
notes:
updateConfigcaptures the patched data eagerly at call time (JSON.parse(JSON.stringify(patch))), so pruning insaveFeedsToFilebefore theupdateConfigcall is exactly where the filter must live — filtering later would be silently ineffective for already-queued writes.- This task intentionally does NOT change in-memory behavior, refresh merging, or cache bounds — that is task 02. If both are worked on in parallel, 02 imports nothing from 01 except the documented window semantics; the module-level contract above is the seam.
downloads.jsonis written bysrc/stores/download.ts(saveDownloads); reading it directly here avoids a store→module import cycle (download.ts already imports the feed store).