71 lines
2.9 KiB
YAML
71 lines
2.9 KiB
YAML
---
|
|
date: 2026-03-08
|
|
day_of_week: Sunday
|
|
task_id: FRE-12
|
|
title: Integrate Redis Queue with Web API
|
|
status: done
|
|
completed_date: 2026-03-08
|
|
company_id: FrenoCorp
|
|
objective: Connect web API to Redis job queue for async processing
|
|
context: |
|
|
- Redis worker module exists at /home/mike/code/AudiobookPipeline/src/worker.py
|
|
- Hono API server needs to enqueue jobs to Redis
|
|
- GPU worker container ready at docker-compose.yml
|
|
issue_type: feature
|
|
priority: high
|
|
assignee: Atlas
|
|
parent_task: FRE-32
|
|
goal_id: MVP_Pipeline_Working
|
|
blocking_tasks: []
|
|
expected_outcome: |
|
|
- Web API enqueues jobs to Redis queue
|
|
- GPU workers pull jobs and process them
|
|
- Job status updates flow back to web dashboard
|
|
acceptance_criteria:
|
|
- POST /api/jobs creates Redis job
|
|
- Worker processes job in background
|
|
- Status updates via WebSocket or polling
|
|
|
|
notes:
|
|
- RQ (Redis Queue) already integrated in worker.py
|
|
- Need API -> Redis enqueue logic
|
|
- Need status update mechanism
|
|
- COMPLETED: Added redis package, updated POST /api/jobs to enqueue jobs
|
|
- COMPLETED: Graceful fallback if Redis not connected
|
|
|
|
completion_notes: |
|
|
Completed 2026-03-08. Deliverables:
|
|
- Added @redis/client package to web platform
|
|
- POST /api/jobs now enqueues job payload to 'audiobook_jobs' Redis queue
|
|
- GET /api/jobs/:id for individual job status lookup
|
|
- PATCH /api/jobs/:id/status for worker to update progress
|
|
- Graceful error handling when Redis is unavailable (logs warning, continues)
|
|
|
|
Testing requires: docker-compose up -d redis
|
|
|
|
**Code Review Improvements (2026-03-15):**
|
|
- Fixed hardcoded subscriptionStatus="free" - now fetched from database via getUserSubscription()
|
|
- Fixed hardcoded demo user data in job completion/failure notifications
|
|
- Notifications now use actual user_id, email, and job data from database
|
|
- Added getUserEmailFromUserId() helper for fetching user emails
|
|
|
|
review_notes: |
|
|
Code review completed 2026-03-14 by Code Reviewer:
|
|
- Found solid implementation with proper separation of concerns
|
|
- Good error handling for Redis connection failures with graceful fallback
|
|
- Proper use of BullMQ for job queuing with appropriate retry mechanisms
|
|
- Clear API endpoints for job creation, retrieval, status updates, and deletion
|
|
- Proper validation using Zod schema for job creation
|
|
- Rate limiting implementation for free tier users
|
|
- Real-time updates via jobEvents and notifications dispatcher
|
|
- Minor improvements noted:
|
|
* Hardcoded subscriptionStatus = "free" in jobs.js line 137 - should come from user data
|
|
* Hardcoded demo user data in job completion/failure events (lines 439-451)
|
|
* Hardcoded error message should use updates.error_message when available (line 459)
|
|
- Assignment: Return to original engineer (Atlas) for minor improvements
|
|
|
|
links:
|
|
worker_code: /home/mike/code/AudiobookPipeline/src/worker.py
|
|
docker_config: /home/mike/code/AudiobookPipeline/docker-compose.yml
|
|
---
|