This commit is contained in:
2026-03-09 11:54:29 -04:00
parent 22e4864b8e
commit 82e43579d6
9 changed files with 418 additions and 16 deletions

View File

@@ -0,0 +1,61 @@
# AudiobookPipeline Project
**Status:** Active
**Role:** Founding Engineer
**Company:** FrenoCorp
## Current State
MVP pipeline development in progress. Core infrastructure complete:
- ✅ Clerk authentication (FRE-39)
- ✅ Dashboard UI with job management (FRE-45)
- ✅ File upload with S3/minio storage (FRE-31)
- ✅ Redis queue integration (FRE-12)
- ✅ Turso database integration
## Recent Completions
### FRE-31: File Upload with S3/minio Storage (2026-03-09)
Implemented complete file upload system:
- S3 client with minio support
- Multipart upload for large files
- Pre-signed URL generation
- 100MB file size limit
- File extension validation (.epub, .pdf, .mobi)
- Graceful fallback when S3 not configured
### FRE-14: Filter Components Library (Firesoft) (2026-03-09)
Created reusable filter components for incident list screens:
- DateRangeFilter component
- MultiSelectFilter component
- Priority filter in FilterRow
- Integrated into incidents/index.tsx
## In Progress
None - awaiting prioritization from board.
## Backlog (Assigned to Atlas)
- FRE-16: Optimize Batch Processing (low priority)
- FRE-17: Add Progress Tracking to Job Processor
- FRE-21: Implement Worker Auto-scaling
- FRE-22: Add Integration Tests for API Endpoints
- FRE-23: Set Up CI/CD Pipeline
- FRE-27: Add Comprehensive Logging and Monitoring
- FRE-28: Optimize Database Queries
- FRE-29: Implement Caching Layer
## Blockers
- Paperclip API authentication unavailable - using local file-based task management
- FRE-33 (CTO permissions) blocked - affects company-wide coordination
## Notes
Working independently with local task files due to Paperclip API auth issues. All completed work documented in daily notes and PARA memory.

View File

@@ -0,0 +1,54 @@
# Atomic facts for FRE-31
- {
type: task,
id: FRE-31,
title: "Implement File Upload with S3/minio Storage",
status: done,
completed_on: "2026-03-09",
assignee: Atlas,
priority: high,
}
- {
type: feature,
name: file_upload,
storage_backend: s3_minio,
fallback: in_memory_mock,
}
- {
type: constraint,
name: max_file_size,
value: 104857600,
unit: bytes,
display: "100MB",
}
- {
type: constraint,
name: allowed_extensions,
values: [".epub", ".pdf", ".mobi"],
}
- { type: package, name: "@aws-sdk/client-s3", version: "^3.1004.0" }
- { type: package, name: "@aws-sdk/lib-storage", version: "^3.1004.0" }
- { type: package, name: "@aws-sdk/s3-request-presigner", version: "^3.1004.0" }
- {
type: endpoint,
path: "/api/jobs",
method: POST,
handles: ["multipart/form-data", "application/json"],
}
- {
type: module,
path: "/home/mike/code/AudiobookPipeline/web/src/server/storage.js",
functions:
[
uploadFile,
getFileUrl,
deleteFile,
getUploadUrl,
initiateMultipartUpload,
uploadPart,
completeMultipartUpload,
abortMultipartUpload,
storeFileMetadata,
],
}

View File

@@ -0,0 +1,67 @@
# FRE-31: Implement File Upload with S3/minio Storage
**Status:** Done
**Completed:** 2026-03-09
**Owner:** Atlas (Founding Engineer)
**Company:** FrenoCorp
## Objective
Add actual file upload support to web platform with S3/minio storage integration.
## Scope
- File upload with multipart form data
- S3/minio integration for production
- Graceful fallback for local development
- 100MB file size limit enforcement
## Completed
### Storage Module (storage.js)
- S3 client initialization with minio support (forcePathStyle: true)
- uploadFile() - handles Blob/File to Buffer conversion
- getFileUrl() - returns download URLs
- deleteFile() - removes files from storage
- getUploadUrl() - generates pre-signed URLs for client-side uploads
- Multipart upload support for large files (initiate/uploadPart/complete/abort)
- storeFileMetadata() - persists file info to Turso database
- Graceful fallback when S3 not configured (returns mock URLs)
### API Integration (jobs.js)
- POST /api/jobs handles multipart/form-data
- File size validation (100MB limit)
- File extension validation (.epub, .pdf, .mobi)
- Uploads file to storage before enqueuing job
- Stores file URL in job record
### Frontend (Dashboard.jsx)
- Sends files via FormData
- Displays upload status
- Error handling for failed uploads
## Acceptance Criteria Met
- ✅ File upload works with multipart form data
- ✅ S3 integration when credentials configured
- ✅ Graceful fallback when S3 not available
- ✅ 100MB file size limit enforced
## Files Modified
- `/home/mike/code/AudiobookPipeline/web/src/server/storage.js` - Created
- `/home/mike/code/AudiobookPipeline/web/src/server/api/jobs.js` - Added file validation
- `/home/mike/code/AudiobookPipeline/web/src/routes/Dashboard.jsx` - Already integrated
## Dependencies
- @aws-sdk/client-s3
- @aws-sdk/lib-storage
- @aws-sdk/s3-request-presigner
## Notes
S3 not configured in .env - graceful fallback active. Set S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY, and S3_BUCKET to enable production storage.