updates
This commit is contained in:
61
agents/atlas/life/projects/audiobook-pipeline/summary.md
Normal file
61
agents/atlas/life/projects/audiobook-pipeline/summary.md
Normal file
@@ -0,0 +1,61 @@
|
||||
# AudiobookPipeline Project
|
||||
|
||||
**Status:** Active
|
||||
**Role:** Founding Engineer
|
||||
**Company:** FrenoCorp
|
||||
|
||||
## Current State
|
||||
|
||||
MVP pipeline development in progress. Core infrastructure complete:
|
||||
|
||||
- ✅ Clerk authentication (FRE-39)
|
||||
- ✅ Dashboard UI with job management (FRE-45)
|
||||
- ✅ File upload with S3/minio storage (FRE-31)
|
||||
- ✅ Redis queue integration (FRE-12)
|
||||
- ✅ Turso database integration
|
||||
|
||||
## Recent Completions
|
||||
|
||||
### FRE-31: File Upload with S3/minio Storage (2026-03-09)
|
||||
|
||||
Implemented complete file upload system:
|
||||
|
||||
- S3 client with minio support
|
||||
- Multipart upload for large files
|
||||
- Pre-signed URL generation
|
||||
- 100MB file size limit
|
||||
- File extension validation (.epub, .pdf, .mobi)
|
||||
- Graceful fallback when S3 not configured
|
||||
|
||||
### FRE-14: Filter Components Library (Firesoft) (2026-03-09)
|
||||
|
||||
Created reusable filter components for incident list screens:
|
||||
|
||||
- DateRangeFilter component
|
||||
- MultiSelectFilter component
|
||||
- Priority filter in FilterRow
|
||||
- Integrated into incidents/index.tsx
|
||||
|
||||
## In Progress
|
||||
|
||||
None - awaiting prioritization from board.
|
||||
|
||||
## Backlog (Assigned to Atlas)
|
||||
|
||||
- FRE-16: Optimize Batch Processing (low priority)
|
||||
- FRE-17: Add Progress Tracking to Job Processor
|
||||
- FRE-21: Implement Worker Auto-scaling
|
||||
- FRE-22: Add Integration Tests for API Endpoints
|
||||
- FRE-23: Set Up CI/CD Pipeline
|
||||
- FRE-27: Add Comprehensive Logging and Monitoring
|
||||
- FRE-28: Optimize Database Queries
|
||||
- FRE-29: Implement Caching Layer
|
||||
|
||||
## Blockers
|
||||
|
||||
- Paperclip API authentication unavailable - using local file-based task management
|
||||
- FRE-33 (CTO permissions) blocked - affects company-wide coordination
|
||||
|
||||
## Notes
|
||||
|
||||
Working independently with local task files due to Paperclip API auth issues. All completed work documented in daily notes and PARA memory.
|
||||
54
agents/atlas/life/projects/fre-31-file-upload/items.yaml
Normal file
54
agents/atlas/life/projects/fre-31-file-upload/items.yaml
Normal file
@@ -0,0 +1,54 @@
|
||||
# Atomic facts for FRE-31
|
||||
|
||||
- {
|
||||
type: task,
|
||||
id: FRE-31,
|
||||
title: "Implement File Upload with S3/minio Storage",
|
||||
status: done,
|
||||
completed_on: "2026-03-09",
|
||||
assignee: Atlas,
|
||||
priority: high,
|
||||
}
|
||||
- {
|
||||
type: feature,
|
||||
name: file_upload,
|
||||
storage_backend: s3_minio,
|
||||
fallback: in_memory_mock,
|
||||
}
|
||||
- {
|
||||
type: constraint,
|
||||
name: max_file_size,
|
||||
value: 104857600,
|
||||
unit: bytes,
|
||||
display: "100MB",
|
||||
}
|
||||
- {
|
||||
type: constraint,
|
||||
name: allowed_extensions,
|
||||
values: [".epub", ".pdf", ".mobi"],
|
||||
}
|
||||
- { type: package, name: "@aws-sdk/client-s3", version: "^3.1004.0" }
|
||||
- { type: package, name: "@aws-sdk/lib-storage", version: "^3.1004.0" }
|
||||
- { type: package, name: "@aws-sdk/s3-request-presigner", version: "^3.1004.0" }
|
||||
- {
|
||||
type: endpoint,
|
||||
path: "/api/jobs",
|
||||
method: POST,
|
||||
handles: ["multipart/form-data", "application/json"],
|
||||
}
|
||||
- {
|
||||
type: module,
|
||||
path: "/home/mike/code/AudiobookPipeline/web/src/server/storage.js",
|
||||
functions:
|
||||
[
|
||||
uploadFile,
|
||||
getFileUrl,
|
||||
deleteFile,
|
||||
getUploadUrl,
|
||||
initiateMultipartUpload,
|
||||
uploadPart,
|
||||
completeMultipartUpload,
|
||||
abortMultipartUpload,
|
||||
storeFileMetadata,
|
||||
],
|
||||
}
|
||||
67
agents/atlas/life/projects/fre-31-file-upload/summary.md
Normal file
67
agents/atlas/life/projects/fre-31-file-upload/summary.md
Normal file
@@ -0,0 +1,67 @@
|
||||
# FRE-31: Implement File Upload with S3/minio Storage
|
||||
|
||||
**Status:** Done
|
||||
**Completed:** 2026-03-09
|
||||
**Owner:** Atlas (Founding Engineer)
|
||||
**Company:** FrenoCorp
|
||||
|
||||
## Objective
|
||||
|
||||
Add actual file upload support to web platform with S3/minio storage integration.
|
||||
|
||||
## Scope
|
||||
|
||||
- File upload with multipart form data
|
||||
- S3/minio integration for production
|
||||
- Graceful fallback for local development
|
||||
- 100MB file size limit enforcement
|
||||
|
||||
## Completed
|
||||
|
||||
### Storage Module (storage.js)
|
||||
|
||||
- S3 client initialization with minio support (forcePathStyle: true)
|
||||
- uploadFile() - handles Blob/File to Buffer conversion
|
||||
- getFileUrl() - returns download URLs
|
||||
- deleteFile() - removes files from storage
|
||||
- getUploadUrl() - generates pre-signed URLs for client-side uploads
|
||||
- Multipart upload support for large files (initiate/uploadPart/complete/abort)
|
||||
- storeFileMetadata() - persists file info to Turso database
|
||||
- Graceful fallback when S3 not configured (returns mock URLs)
|
||||
|
||||
### API Integration (jobs.js)
|
||||
|
||||
- POST /api/jobs handles multipart/form-data
|
||||
- File size validation (100MB limit)
|
||||
- File extension validation (.epub, .pdf, .mobi)
|
||||
- Uploads file to storage before enqueuing job
|
||||
- Stores file URL in job record
|
||||
|
||||
### Frontend (Dashboard.jsx)
|
||||
|
||||
- Sends files via FormData
|
||||
- Displays upload status
|
||||
- Error handling for failed uploads
|
||||
|
||||
## Acceptance Criteria Met
|
||||
|
||||
- ✅ File upload works with multipart form data
|
||||
- ✅ S3 integration when credentials configured
|
||||
- ✅ Graceful fallback when S3 not available
|
||||
- ✅ 100MB file size limit enforced
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `/home/mike/code/AudiobookPipeline/web/src/server/storage.js` - Created
|
||||
- `/home/mike/code/AudiobookPipeline/web/src/server/api/jobs.js` - Added file validation
|
||||
- `/home/mike/code/AudiobookPipeline/web/src/routes/Dashboard.jsx` - Already integrated
|
||||
|
||||
## Dependencies
|
||||
|
||||
- @aws-sdk/client-s3
|
||||
- @aws-sdk/lib-storage
|
||||
- @aws-sdk/s3-request-presigner
|
||||
|
||||
## Notes
|
||||
|
||||
S3 not configured in .env - graceful fallback active. Set S3_ENDPOINT, S3_ACCESS_KEY, S3_SECRET_KEY, and S3_BUCKET to enable production storage.
|
||||
Reference in New Issue
Block a user