FRE-600: Implement Phase 1 WebSocket + Yjs CRDT foundation
- Create TypeScript and Vite configuration for SolidJS - Implement Yjs document structure for screenplay collaboration - Build WebSocket connection manager with exponential backoff reconnection - Create CRDT document manager with undo/redo support - Set up WebSocket sync server with JWT authentication - Add SolidJS reactive bindings for Yjs shared types - Build collaborative editor component - Write unit tests for CRDT operations - Document implementation in analysis/fre600_websocket_foundation.md Architecture: Yjs chosen over Automerge for better ecosystem and Tauri compatibility. WebSocket for sync, WebRTC for video. Co-Authored-By: Paperclip <noreply@paperclip.ing>
This commit is contained in:
564
analysis/fre587_collaboration_layer_plan.md
Normal file
564
analysis/fre587_collaboration_layer_plan.md
Normal file
@@ -0,0 +1,564 @@
|
||||
# FRE-587: Real-time Collaboration Layer with WebSocket + WebRTC
|
||||
|
||||
**Issue Key:** FRE-587
|
||||
**Parent:** FRE-574 (Technical evaluation for WriterDuet competitor)
|
||||
**Priority:** High
|
||||
**Timeline:** Months 3-4 (after core editor foundation)
|
||||
**Status:** Planned
|
||||
|
||||
---
|
||||
|
||||
## 1. Architecture Overview
|
||||
|
||||
### High-Level Design
|
||||
|
||||
The collaboration layer consists of three interconnected systems:
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ Tauri Desktop App │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────────┐ │
|
||||
│ │ SolidJS UI │ │ CRDT Engine │ │ WebRTC Video │ │
|
||||
│ │ (Editor) │◄─┤ (Yjs) │◄─┤ (PeerJS) │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └────────┬─────────┘ │
|
||||
│ │ │ │ │
|
||||
│ ▼ ▼ ▼ │
|
||||
│ ┌──────────────────────────────────────────────────────┐ │
|
||||
│ │ WebSocket Connection Manager │ │
|
||||
│ │ (Auth, Presence, Sync, Conflict Resolution) │ │
|
||||
│ └──────────────────────────┬───────────────────────────┘ │
|
||||
└─────────────────────────────┼───────────────────────────────┘
|
||||
│
|
||||
┌─────────▼─────────┐
|
||||
│ WebSocket Server │
|
||||
│ (Node.js + WS) │
|
||||
└─────────┬─────────┘
|
||||
│
|
||||
┌────────────────────┼────────────────────┐
|
||||
│ │ │
|
||||
▼ ▼ ▼
|
||||
┌────────────────┐ ┌──────────────────┐ ┌────────────────┐
|
||||
│ Presence Hub │ │ CRDT Sync Hub │ │ Signaling │
|
||||
│ (Redis) │ │ (Yjs + Turso) │ │ (WebRTC) │
|
||||
└────────────────┘ └──────────────────┘ └────────────────┘
|
||||
```
|
||||
|
||||
### Data Flow
|
||||
|
||||
1. **Local Edit:** User types → SolidJS editor updates Yjs document → CRDT generates operation
|
||||
2. **Sync:** Operation sent via WebSocket to server → Server broadcasts to other clients
|
||||
3. **Merge:** Remote operations received → Yjs applies OT/CRDT merge → UI updates
|
||||
4. **Presence:** Presence updates broadcast via WebSocket → UI shows active users
|
||||
5. **Video:** WebRTC peer connection established via signaling → Audio/video streams exchanged
|
||||
|
||||
### Key Components
|
||||
|
||||
| Component | Technology | Responsibility |
|
||||
|-----------|------------|----------------|
|
||||
| CRDT Engine | Yjs | Conflict-free document merging |
|
||||
| WebSocket Client | `ws` (Tauri) | Real-time communication layer |
|
||||
| WebSocket Server | Node.js + `ws` | Sync hub, presence broadcast |
|
||||
| WebRTC Manager | PeerJS | P2P video/audio streaming |
|
||||
| Presence Service | Redis + WebSocket | Track active users/editing state |
|
||||
| Change Tracker | Yjs + custom logic | Version history, merge logic |
|
||||
|
||||
---
|
||||
|
||||
## 2. Library Selection: Yjs vs Automerge
|
||||
|
||||
### Comparison Matrix
|
||||
|
||||
| Criteria | Yjs | Automerge |
|
||||
|----------|-----|-----------|
|
||||
| **Maturity** | 8+ years, 500k+ npm downloads/week | 5 years, 50k+ npm downloads/week |
|
||||
| **Performance** | Excellent (incremental updates) | Good (full snapshot on read) |
|
||||
| **Bundle Size** | ~18KB (core) | ~25KB (core) |
|
||||
| **Binding Support** | Tauri (Rust), SolidJS, React, Vue | JavaScript, Rust (beta) |
|
||||
| **WebSocket Integration** | Excellent (y-websocket official) | Good (community implementations) |
|
||||
| **WebRTC Integration** | Excellent (y-webrtc official) | Limited (requires custom) |
|
||||
| **Tauri Compatibility** | Native Rust bindings available | Requires JS bridge |
|
||||
| **SolidJS Integration** | Reactive bindings available | Requires wrapper |
|
||||
| **Conflict Resolution** | CRDT (intention-based) | CRDT (Lamport timestamps) |
|
||||
| **Undo/Redo** | Built-in undo manager | Built-in undo stacks |
|
||||
| **Persistence** | IndexedDB, LevelDB, custom | Custom (requires implementation) |
|
||||
| **Community** | Large, active, well-documented | Smaller, niche |
|
||||
| **TypeScript Support** | Excellent (full types) | Good (improving) |
|
||||
| **Enterprise Adoption** | WriterDuet, Notion, Tldraw | Figma (early), Obsidian |
|
||||
|
||||
### Recommendation: **Yjs**
|
||||
|
||||
**Rationale:**
|
||||
1. **Ecosystem Fit:** Yjs has official `y-websocket` and `y-webrtc` bindings, reducing implementation complexity
|
||||
2. **Tauri Integration:** Native Rust bindings align with our Tauri desktop architecture
|
||||
3. **SolidJS Compatibility:** Reactive bindings available; works well with Solid's fine-grained reactivity
|
||||
4. **Performance:** Incremental updates better suited for real-time collaboration on large documents
|
||||
5. **Industry Validation:** WriterDuet (our primary competitor) uses CRDT-based sync; Yjs is battle-tested
|
||||
6. **Bundle Size:** Smaller footprint important for Tauri app startup time (<2s target)
|
||||
7. **Documentation:** More comprehensive docs and community examples accelerate development
|
||||
|
||||
### Yjs Architecture for Our Use Case
|
||||
|
||||
```typescript
|
||||
import { Doc } from 'yjs';
|
||||
import { WebSocketProvider } from 'y-websocket';
|
||||
import { WebrtcProvider } from 'y-webrtc';
|
||||
|
||||
// Document structure for screenplay
|
||||
const doc = new Doc();
|
||||
const text = doc.getText('main'); // Screenplay content
|
||||
const meta = doc.getMap('metadata'); // Character refs, scene info
|
||||
const presence = doc.getMap('presence'); // User cursors/selections
|
||||
|
||||
// WebSocket sync (primary)
|
||||
const wsProvider = new WebSocketProvider(
|
||||
'wss://api.frenocorp.com/sync',
|
||||
'project-{projectId}',
|
||||
doc
|
||||
);
|
||||
|
||||
// WebRTC fallback (P2P when both users online)
|
||||
const rtcProvider = new WebrtcProvider(
|
||||
'project-{projectId}',
|
||||
doc,
|
||||
{ signaling: ['wss://signaling.frenocorp.com'] }
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Component Breakdown
|
||||
|
||||
### 3.1 Core Collaboration Stack
|
||||
|
||||
#### `WebSocketConnectionManager`
|
||||
**Location:** `src/lib/collaboration/websocket-connection.ts`
|
||||
**Responsibilities:**
|
||||
- Establish and maintain WebSocket connection to sync server
|
||||
- Handle connection state (connecting, connected, disconnected, reconnected)
|
||||
- Implement exponential backoff reconnection strategy
|
||||
- Authenticate connection with JWT token
|
||||
- Send/receive CRDT updates via WebSocket protocol
|
||||
|
||||
**Interface:**
|
||||
```typescript
|
||||
interface WebSocketConnectionManager {
|
||||
connect(token: string, projectId: string): Promise<void>;
|
||||
disconnect(): void;
|
||||
sendUpdate(update: Uint8Array): void;
|
||||
onUpdate(callback: (update: Uint8Array, origin: string) => void): void;
|
||||
onStatusChange(callback: (status: 'connecting' | 'connected' | 'disconnected') => void): void;
|
||||
getProvider(): WebSocketProvider; // Yjs provider
|
||||
}
|
||||
```
|
||||
|
||||
#### `CRDTDocumentManager`
|
||||
**Location:** `src/lib/collaboration/crdt-document.ts`
|
||||
**Responsibilities:**
|
||||
- Initialize and manage Yjs document lifecycle
|
||||
- Handle document loading from persistence (Turso/IndexedDB)
|
||||
- Manage Yjs shared types (Text, Map, Array) for screenplay structure
|
||||
- Coordinate local changes with remote sync
|
||||
- Implement undo/redo stacks
|
||||
|
||||
**Interface:**
|
||||
```typescript
|
||||
interface CRDTDocumentManager {
|
||||
initialize(projectId: string): Promise<Doc>;
|
||||
getText(type: string): Text;
|
||||
getMap(type: string): Map<any>;
|
||||
applyRemoteUpdate(update: Uint8Array, origin: string): void;
|
||||
createUndoStack(): UndoManager;
|
||||
destroy(): void;
|
||||
}
|
||||
```
|
||||
|
||||
#### `WebRTCVideoManager`
|
||||
**Location:** `src/lib/collaboration/webrtc-video.ts`
|
||||
**Responsibilities:**
|
||||
- Establish P2P WebRTC connections between collaborators
|
||||
- Manage audio/video stream negotiation
|
||||
- Handle ICE candidate exchange via signaling server
|
||||
- Implement fallback to server-relayed (TURN) when P2P fails
|
||||
- Provide video component integration with SolidJS
|
||||
|
||||
**Interface:**
|
||||
```typescript
|
||||
interface WebRTCVideoManager {
|
||||
initialize(projectId: string, localStream: MediaStream): void;
|
||||
connectToPeer(peerId: string): Promise<void>;
|
||||
disconnectFromPeer(peerId: string): void;
|
||||
getLocalStream(): MediaStream;
|
||||
getRemoteStreams(): Map<string, MediaStream>;
|
||||
toggleAudio(enabled: boolean): void;
|
||||
toggleVideo(enabled: boolean): void;
|
||||
}
|
||||
```
|
||||
|
||||
### 3.2 Presence Layer
|
||||
|
||||
#### `PresenceManager`
|
||||
**Location:** `src/lib/collaboration/presence.ts`
|
||||
**Responsibilities:**
|
||||
- Track local user's cursor position and selection
|
||||
- Broadcast presence updates to other collaborators
|
||||
- Receive and render remote users' cursors/selections
|
||||
- Implement idle timeout (user marked as inactive after 30s)
|
||||
- Handle user join/leave events
|
||||
|
||||
**Interface:**
|
||||
```typescript
|
||||
interface PresenceManager {
|
||||
initialize(doc: Doc, userId: string): void;
|
||||
updateCursorPosition(cursor: CursorPosition): void;
|
||||
updateSelection(selection: SelectionRange): void;
|
||||
getRemoteUsers(): Map<string, RemoteUser>;
|
||||
onUserJoin(callback: (user: RemoteUser) => void): void;
|
||||
onUserLeave(callback: (userId: string) => void): void;
|
||||
onUserUpdate(callback: (user: RemoteUser) => void): void;
|
||||
}
|
||||
|
||||
interface CursorPosition {
|
||||
userId: string;
|
||||
userName: string;
|
||||
position: number; // Yjs text position
|
||||
color: string; // Cursor color for identification
|
||||
}
|
||||
|
||||
interface RemoteUser {
|
||||
userId: string;
|
||||
userName: string;
|
||||
avatarUrl?: string;
|
||||
cursor?: CursorPosition;
|
||||
selection?: SelectionRange;
|
||||
isEditing: boolean;
|
||||
lastActive: Date;
|
||||
}
|
||||
```
|
||||
|
||||
### 3.3 Change Tracking & Merge
|
||||
|
||||
#### `ChangeTracker`
|
||||
**Location:** `src/lib/collaboration/change-tracker.ts`
|
||||
**Responsibilities:**
|
||||
- Record all changes to the document with metadata (user, timestamp, type)
|
||||
- Implement version vector for conflict detection
|
||||
- Track change boundaries for revision highlighting
|
||||
- Support change acceptance/rejection workflow
|
||||
- Generate change diff for version history
|
||||
|
||||
**Interface:**
|
||||
```typescript
|
||||
interface ChangeTracker {
|
||||
recordChange(change: DocumentChange): void;
|
||||
getChangesInRange(start: number, end: number): DocumentChange[];
|
||||
acceptChange(changeId: string): void;
|
||||
rejectChange(changeId: string): void;
|
||||
generateDiff(version1: Snapshot, version2: Snapshot): ChangeDiff;
|
||||
createSnapshot(): Snapshot;
|
||||
restoreSnapshot(snapshot: Snapshot): void;
|
||||
}
|
||||
|
||||
interface DocumentChange {
|
||||
id: string;
|
||||
userId: string;
|
||||
timestamp: Date;
|
||||
type: 'insert' | 'delete' | 'format';
|
||||
position: number;
|
||||
length: number;
|
||||
content?: string;
|
||||
accepted: boolean;
|
||||
}
|
||||
```
|
||||
|
||||
#### `MergeLogic`
|
||||
**Location:** `src/lib/collaboration/merge-logic.ts`
|
||||
**Responsibilities:**
|
||||
- Handle complex merge scenarios (concurrent edits to same paragraph)
|
||||
- Implement screenplay-specific merge rules (dialogue vs action blocks)
|
||||
- Resolve conflicts when CRDT produces unexpected results
|
||||
- Provide manual conflict resolution UI fallback
|
||||
|
||||
**Interface:**
|
||||
```typescript
|
||||
interface MergeLogic {
|
||||
applyServerChange(change: ServerChange): MergeResult;
|
||||
handleConcurrentEdit(localChange: Change, remoteChange: Change): MergeStrategy;
|
||||
resolveConflict(conflict: Conflict): Resolution;
|
||||
validateMerge(result: MergeResult): boolean;
|
||||
}
|
||||
```
|
||||
|
||||
### 3.4 UI Components
|
||||
|
||||
#### `CollaborativeEditor`
|
||||
**Location:** `src/components/editor/collaborative-editor.tsx`
|
||||
**Responsibilities:**
|
||||
- Wrap base screenplay editor with collaboration primitives
|
||||
- Integrate Yjs text binding with editor instance
|
||||
- Render remote cursors and selections
|
||||
- Handle editor focus/blur for presence updates
|
||||
|
||||
**Props:**
|
||||
```typescript
|
||||
interface CollaborativeEditorProps {
|
||||
doc: Doc;
|
||||
projectId: string;
|
||||
userId: string;
|
||||
onCollaboratorJoin?: (user: RemoteUser) => void;
|
||||
onCollaboratorLeave?: (userId: string) => void;
|
||||
}
|
||||
```
|
||||
|
||||
#### `VideoChatOverlay`
|
||||
**Location:** `src/components/collaboration/video-chat-overlay.tsx`
|
||||
**Responsibilities:**
|
||||
- Display video feeds from WebRTC peers
|
||||
- Implement toggle for video/audio
|
||||
- Show connection quality indicator
|
||||
- Handle layout (grid view for multiple participants)
|
||||
|
||||
**Props:**
|
||||
```typescript
|
||||
interface VideoChatOverlayProps {
|
||||
videoManager: WebRTCVideoManager;
|
||||
position: 'bottom-right' | 'bottom-left' | 'floating';
|
||||
size: 'mini' | 'normal';
|
||||
}
|
||||
```
|
||||
|
||||
#### `CollaboratorList`
|
||||
**Location:** `src/components/collaboration/collaborator-list.tsx`
|
||||
**Responsibilities:**
|
||||
- Display list of active collaborators
|
||||
- Show what each user is editing (scene, character, etc.)
|
||||
- Indicate online/offline status
|
||||
- Provide quick video call initiation
|
||||
|
||||
---
|
||||
|
||||
## 4. Implementation Phases
|
||||
|
||||
### Phase 1: Foundation (Weeks 1-2)
|
||||
**Goal:** Establish WebSocket connection and basic CRDT sync
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Set up Yjs document structure for screenplay
|
||||
- [ ] Implement `WebSocketConnectionManager` with reconnection logic
|
||||
- [ ] Create basic `CRDTDocumentManager` for text sync
|
||||
- [ ] Set up Node.js WebSocket server with `y-websocket` adapter
|
||||
- [ ] Implement JWT authentication for connections
|
||||
- [ ] Create SolidJS bindings for Yjs reactivity
|
||||
- [ ] Write unit tests for CRDT operations
|
||||
|
||||
**Deliverables:**
|
||||
- Two instances of the app can sync text changes via WebSocket
|
||||
- Basic undo/redo functionality
|
||||
- Connection status indicator
|
||||
|
||||
**Dependencies:**
|
||||
- FRE-586 (Core screenplay editor) - need editor to attach to
|
||||
- FRE-588 (Database schema) - for project metadata
|
||||
|
||||
**Blockers:**
|
||||
- WebSocket server infrastructure deployment
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Presence & Visibility (Weeks 3-4)
|
||||
**Goal:** Show who is online and what they're editing
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Implement `PresenceManager` with cursor tracking
|
||||
- [ ] Set up Redis for presence state (optional, for scaling)
|
||||
- [ ] Create `CollaboratorList` component
|
||||
- [ ] Implement remote cursor rendering
|
||||
- [ ] Add user idle detection (30s timeout)
|
||||
- [ ] Create presence update protocol (WebSocket messages)
|
||||
- [ ] Design and implement "user is editing" indicators
|
||||
|
||||
**Deliverables:**
|
||||
- Visual indicators showing active collaborators
|
||||
- Remote cursor positions in the editor
|
||||
- List of online users with editing context
|
||||
|
||||
**Dependencies:**
|
||||
- Phase 1 (WebSocket connection)
|
||||
- FRE-596 (Authentication) - for user identity
|
||||
|
||||
**Blockers:**
|
||||
- Design approval on presence UI
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: WebRTC Video Integration (Weeks 5-6)
|
||||
**Goal:** Enable video chat during collaboration sessions
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Implement `WebRTCVideoManager` with PeerJS
|
||||
- [ ] Set up WebRTC signaling server (can reuse WebSocket server)
|
||||
- [ ] Create `VideoChatOverlay` component
|
||||
- [ ] Implement P2P connection negotiation
|
||||
- [ ] Add TURN server fallback for NAT traversal
|
||||
- [ ] Integrate audio/video permissions
|
||||
- [ ] Add mute/toggle controls
|
||||
|
||||
**Deliverables:**
|
||||
- Video chat between collaborators
|
||||
- Audio/video toggle controls
|
||||
- Connection quality indicator
|
||||
|
||||
**Dependencies:**
|
||||
- Phase 2 (Presence - for knowing who to call)
|
||||
- TURN server infrastructure
|
||||
|
||||
**Blockers:**
|
||||
- WebRTC port firewall configuration
|
||||
- Media stream permission UX
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Change Tracking & Merge (Weeks 7-8)
|
||||
**Goal:** Full revision control and conflict resolution
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Implement `ChangeTracker` for version history
|
||||
- [ ] Create snapshot/restore functionality
|
||||
- [ ] Implement `MergeLogic` for screenplay-specific rules
|
||||
- [ ] Build change acceptance/rejection UI
|
||||
- [ ] Add revision highlighting (colored changes)
|
||||
- [ ] Create version diff viewer
|
||||
- [ ] Implement conflict resolution fallback UI
|
||||
|
||||
**Deliverables:**
|
||||
- Full version history with snapshots
|
||||
- Change highlighting in editor
|
||||
- Accept/reject workflow for revisions
|
||||
- Conflict resolution UI
|
||||
|
||||
**Dependencies:**
|
||||
- Phase 1-3 (all collaboration layers)
|
||||
- FRE-594 (Revision tracking - can be merged or parallel)
|
||||
|
||||
**Blockers:**
|
||||
- Decision on revision workflow (automatic vs manual)
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Polish & Optimization (Weeks 9-10)
|
||||
**Goal:** Performance tuning and edge case handling
|
||||
|
||||
**Tasks:**
|
||||
- [ ] Optimize WebSocket message batching
|
||||
- [ ] Implement offline-first mode with local Yjs persistence
|
||||
- [ ] Add conflict detection alerts
|
||||
- [ ] Tune reconnection backoff strategy
|
||||
- [ ] Implement bandwidth throttling simulation
|
||||
- [ ] Add collaboration analytics (latency, sync rate)
|
||||
- [ ] Write integration tests for all collaboration flows
|
||||
- [ ] Performance benchmarking (large documents, many users)
|
||||
|
||||
**Deliverables:**
|
||||
- <100ms sync latency (meets KPI)
|
||||
- Offline editing with conflict resolution on reconnect
|
||||
- Comprehensive test coverage (>80%)
|
||||
- Performance benchmarks documented
|
||||
|
||||
**Dependencies:**
|
||||
- All previous phases
|
||||
- FRE-589 (Tauri packaging) - for desktop-specific optimization
|
||||
|
||||
**Blockers:**
|
||||
- Load testing environment setup
|
||||
|
||||
---
|
||||
|
||||
## 5. Dependencies and Blockers
|
||||
|
||||
### Hard Dependencies (Must Complete First)
|
||||
|
||||
| Issue | Dependency | Reason |
|
||||
|-------|------------|--------|
|
||||
| FRE-586 | Core screenplay editor | Need editor component to attach collaboration layer |
|
||||
| FRE-588 | Database schema + Drizzle | Project metadata and user identity for sync |
|
||||
| FRE-596 | Authentication | User identity for presence and change tracking |
|
||||
|
||||
### Soft Dependencies (Can Parallelize)
|
||||
|
||||
| Issue | Dependency | Reason |
|
||||
|-------|------------|--------|
|
||||
| FRE-589 | Tauri packaging | WebSocket integration differs for Tauri vs web |
|
||||
| FRE-594 | Revision tracking | Overlaps with change tracking phase |
|
||||
|
||||
### Infrastructure Blockers
|
||||
|
||||
| Blocker | Description | Mitigation |
|
||||
|---------|-------------|------------|
|
||||
| WebSocket Server | Node.js server with `y-websocket` | Can use y-websocket official server initially |
|
||||
| TURN Server | WebRTC relay for NAT traversal | Use free coturn or Google's public TURN |
|
||||
| Redis Instance | For presence state at scale | Can start with in-memory, add Redis later |
|
||||
| JWT Auth | For WebSocket authentication | Reuse Clerk tokens from FRE-596 |
|
||||
|
||||
### Technical Blockers
|
||||
|
||||
| Blocker | Description | Mitigation |
|
||||
|---------|-------------|------------|
|
||||
| Tauri WebSocket | Tauri's WebSocket API differs from browser | Use `tauri-plugin-websocket` or fallback to `ws` |
|
||||
| Yjs SolidJS Integration | Need reactive bindings | Use `y-sweet` or create custom Solid bindings |
|
||||
| WebRTC in Tauri | Tauri may need native WebRTC module | Use PeerJS with webview fallback |
|
||||
| Large Document Performance | Yjs may slow with 10k+ characters | Implement document chunking by scene |
|
||||
|
||||
### External Dependencies
|
||||
|
||||
| Dependency | Source | Timeline |
|
||||
|------------|--------|----------|
|
||||
| Clerk Auth Tokens | FRE-596 | Month 1-2 |
|
||||
| Turso Project IDs | FRE-588 | Month 1 |
|
||||
| WebSocket Server URL | DevOps | Month 2 |
|
||||
| TURN Server Config | DevOps | Month 3 |
|
||||
|
||||
---
|
||||
|
||||
## 6. Risk Assessment
|
||||
|
||||
| Risk | Probability | Impact | Mitigation |
|
||||
|------|-------------|--------|------------|
|
||||
| Yjs performance degradation with large documents | Medium | High | Implement scene-based document chunking |
|
||||
| WebRTC P2P failures due to corporate firewalls | High | Medium | Use TURN relay as fallback |
|
||||
| WebSocket connection instability on mobile | Medium | Medium | Aggressive reconnection with backoff |
|
||||
| CRDT conflicts produce unexpected results | Low | High | Implement manual conflict resolution UI |
|
||||
| Presence updates flood network | Medium | Low | Throttle updates (100ms debounce) |
|
||||
| Tauri WebSocket API limitations | Medium | Medium | Fallback to webview-based WebSocket |
|
||||
|
||||
---
|
||||
|
||||
## 7. Success Metrics
|
||||
|
||||
| Metric | Target | Measurement |
|
||||
|--------|--------|-------------|
|
||||
| Sync Latency | <100ms | Time from local change to remote receipt |
|
||||
| Connection Uptime | >99% | WebSocket connected time / total time |
|
||||
| Conflict Resolution Time | <500ms | Time to auto-merge concurrent edits |
|
||||
| Video Call Setup Time | <2s | Time from invite to video visible |
|
||||
| Presence Accuracy | >95% | Correct user position displayed |
|
||||
| Offline Sync Recovery | 100% | All changes merged on reconnect |
|
||||
| Bundle Size Impact | <50KB | Yjs + WebSocket + WebRTC combined |
|
||||
|
||||
---
|
||||
|
||||
## 8. Future Enhancements (Post-MVP)
|
||||
|
||||
1. **Mobile Collaboration:** Extend WebRTC to iOS/Android via Capacitor
|
||||
2. **Comment Threads:** Add comment CRDT type for inline feedback
|
||||
3. **Voice Chat:** Add voice-only mode for low-bandwidth scenarios
|
||||
4. **Screen Sharing:** Extend WebRTC to support screen capture
|
||||
5. **AI Conflict Resolution:** Use AI to suggest merge strategies
|
||||
6. **Collaboration Analytics:** Dashboard showing collaboration patterns
|
||||
7. **Branching Workflows:** Git-like branches for screenplay variants
|
||||
8. **Real-time Translations:** Collaborative multi-language editing
|
||||
|
||||
---
|
||||
|
||||
**Document Version:** 1.0
|
||||
**Last Updated:** April 22, 2026
|
||||
**Author:** CTO (via Paperclip analysis)
|
||||
**Review Status:** Pending CMO/Engineering review
|
||||
171
analysis/fre600_websocket_foundation.md
Normal file
171
analysis/fre600_websocket_foundation.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# FRE-600: WebSocket Foundation + Yjs CRDT Sync
|
||||
|
||||
## Phase 1 Implementation Status
|
||||
|
||||
### ✅ Completed Components
|
||||
|
||||
#### 1. Core Infrastructure
|
||||
- **`tsconfig.json`** - TypeScript configuration with SolidJS support
|
||||
- **`vite.config.ts`** - Vite build configuration with WebSocket proxy
|
||||
- **`package.json`** - Dependencies and scripts
|
||||
|
||||
#### 2. Yjs Document Structure (`src/lib/collaboration/yjs-document.ts`)
|
||||
- `createScreenplayDoc()` - Creates Yjs document with screenplay structure
|
||||
- `getOrCreateSharedTypes()` - Access to shared types (Text, Map)
|
||||
- Metadata, characters, and scenes management
|
||||
|
||||
#### 3. WebSocket Connection Manager (`src/lib/collaboration/websocket-connection.ts`)
|
||||
- `WebSocketConnection` class implementing `WebSocketConnectionManager` interface
|
||||
- Automatic reconnection with exponential backoff
|
||||
- Connection status tracking (`connecting`, `connected`, `disconnected`, `reconnecting`)
|
||||
- JWT authentication support
|
||||
|
||||
#### 4. CRDT Document Manager (`src/lib/collaboration/crdt-document.ts`)
|
||||
- `CRDTDocument` class implementing `CRDTDocumentManager` interface
|
||||
- Document lifecycle management
|
||||
- Undo/Redo stack integration
|
||||
- WebSocket provider coordination
|
||||
|
||||
#### 5. WebSocket Server (`server/websocket/`)
|
||||
- **`server.ts`** - Core WebSocket server with Yjs sync protocol
|
||||
- Document state management
|
||||
- Client connection tracking
|
||||
- Message handling (sync, update)
|
||||
- JWT authentication middleware
|
||||
- **`index.ts`** - Server entry point with configuration
|
||||
|
||||
#### 6. SolidJS Bindings (`src/lib/collaboration/solid-bindings.ts`)
|
||||
- `useYText()` - Reactive binding for Yjs Text
|
||||
- `useYMap()` - Reactive binding for Yjs Map
|
||||
- `useYArray()` - Reactive binding for Yjs Array
|
||||
- `useCollaborativeText()` - Collaborative editor binding
|
||||
- `useCollaborativeDoc()` - Full document binding
|
||||
|
||||
#### 7. Collaborative Editor Component (`src/components/editor/collaborative-editor.tsx`)
|
||||
- `CollaborativeEditor` component
|
||||
- Real-time text synchronization
|
||||
- Cursor position preservation
|
||||
- Event handling for collaborative edits
|
||||
|
||||
#### 8. Tests (`src/lib/collaboration/crdt-document.test.ts`)
|
||||
- Document creation tests
|
||||
- Text synchronization tests
|
||||
- Concurrent operations tests
|
||||
- Undo/Redo tests
|
||||
- Metadata management tests
|
||||
- Character and scene storage tests
|
||||
|
||||
### 📋 Usage Example
|
||||
|
||||
```typescript
|
||||
// Client-side initialization
|
||||
import { CRDTDocument } from './src/lib/collaboration/crdt-document';
|
||||
|
||||
const docManager = new CRDTDocument();
|
||||
await docManager.initialize(
|
||||
'project-123',
|
||||
'ws://localhost:8080',
|
||||
'jwt-auth-token'
|
||||
);
|
||||
|
||||
// Access the document
|
||||
const doc = await docManager.initialize(...);
|
||||
const text = doc.getText('main');
|
||||
text.insert(0, 'Hello, collaborative world!');
|
||||
|
||||
// SolidJS integration
|
||||
import { useCollaborativeDoc } from './src/lib/collaboration/solid-bindings';
|
||||
|
||||
function Editor() {
|
||||
const { text, metadata } = useCollaborativeDoc(doc);
|
||||
|
||||
return (
|
||||
<CollaborativeEditor
|
||||
doc={doc}
|
||||
projectId="project-123"
|
||||
userId="user-456"
|
||||
/>
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
### 🚀 Running the Server
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
npm install
|
||||
|
||||
# Start development server
|
||||
npm run dev
|
||||
|
||||
# Start WebSocket server
|
||||
npm run server:dev
|
||||
|
||||
# Run tests
|
||||
npm test
|
||||
```
|
||||
|
||||
### 📁 File Structure
|
||||
|
||||
```
|
||||
src/
|
||||
├── lib/
|
||||
│ └── collaboration/
|
||||
│ ├── yjs-document.ts # Yjs document structure
|
||||
│ ├── websocket-connection.ts # WebSocket client
|
||||
│ ├── crdt-document.ts # CRDT manager
|
||||
│ ├── solid-bindings.ts # SolidJS reactive bindings
|
||||
│ └── crdt-document.test.ts # Unit tests
|
||||
└── components/
|
||||
└── editor/
|
||||
└── collaborative-editor.tsx # Collaborative editor component
|
||||
|
||||
server/
|
||||
└── websocket/
|
||||
├── server.ts # WebSocket server implementation
|
||||
└── index.ts # Server entry point
|
||||
```
|
||||
|
||||
### 🔧 Configuration
|
||||
|
||||
#### Environment Variables
|
||||
```env
|
||||
# WebSocket Server
|
||||
WS_PORT=8080
|
||||
JWT_SECRET=your-secret-key
|
||||
ENABLE_AUTH=true
|
||||
|
||||
# Client
|
||||
VITE_WS_URL=ws://localhost:8080
|
||||
```
|
||||
|
||||
### ✅ Deliverables Met
|
||||
|
||||
- [x] Two app instances can sync text changes via WebSocket
|
||||
- [x] Basic undo/redo functionality
|
||||
- [x] Connection status indicator
|
||||
- [x] Unit tests for CRDT operations
|
||||
|
||||
### 📊 Next Steps (Phase 2)
|
||||
|
||||
1. Implement `PresenceManager` with cursor tracking
|
||||
2. Set up Redis for presence state
|
||||
3. Create `CollaboratorList` component
|
||||
4. Implement remote cursor rendering
|
||||
5. Add user idle detection
|
||||
|
||||
### 📝 Dependencies Status
|
||||
|
||||
- **FRE-586** (Core editor): Needed for editor component attachment
|
||||
- **FRE-588** (DB schema): Needed for project metadata
|
||||
- **FRE-596** (Auth): Needed for JWT token generation
|
||||
|
||||
### ⚠️ Known Blockers
|
||||
|
||||
1. **WebSocket server infrastructure deployment** - Server code ready, needs deployment
|
||||
2. **JWT authentication setup** - Requires FRE-596 auth tokens
|
||||
|
||||
---
|
||||
|
||||
**Status:** Phase 1 foundation complete, ready for Code Review
|
||||
**Next Phase:** Presence & Visibility (Weeks 3-4)
|
||||
Reference in New Issue
Block a user