# FRE-587: Real-time Collaboration Layer with WebSocket + WebRTC

**Issue Key:** FRE-587  
**Parent:** FRE-574 (Technical evaluation for WriterDuet competitor)  
**Priority:** High  
**Timeline:** Months 3-4 (after core editor foundation)  
**Status:** Planned  

---

## 1. Architecture Overview

### High-Level Design

The collaboration layer consists of three interconnected systems:

```
┌─────────────────────────────────────────────────────────────┐
│                     Tauri Desktop App                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐   │
│  │  SolidJS UI  │  │  CRDT Engine │  │  WebRTC Video    │   │
│  │  (Editor)    │◄─┤  (Yjs)       │◄─┤  (PeerJS)        │   │
│  └──────┬───────┘  └──────┬───────┘  └────────┬─────────┘   │
│         │                 │                    │            │
│         ▼                 ▼                    ▼            │
│  ┌──────────────────────────────────────────────────────┐   │
│  │              WebSocket Connection Manager            │   │
│  │  (Auth, Presence, Sync, Conflict Resolution)         │   │
│  └──────────────────────────┬───────────────────────────┘   │
└─────────────────────────────┼───────────────────────────────┘
                              │
                    ┌─────────▼─────────┐
                    │  WebSocket Server │
                    │  (Node.js + WS)   │
                    └─────────┬─────────┘
                              │
         ┌────────────────────┼────────────────────┐
         │                    │                    │
         ▼                    ▼                    ▼
┌────────────────┐  ┌──────────────────┐  ┌────────────────┐
│  Presence Hub  │  │  CRDT Sync Hub   │  │  Signaling     │
│  (Redis)       │  │  (Yjs + Turso)   │  │  (WebRTC)      │
└────────────────┘  └──────────────────┘  └────────────────┘
```

### Data Flow

1. **Local Edit:** User types → SolidJS editor updates Yjs document → CRDT generates operation
2. **Sync:** Operation sent via WebSocket to server → Server broadcasts to other clients
3. **Merge:** Remote operations received → Yjs applies OT/CRDT merge → UI updates
4. **Presence:** Presence updates broadcast via WebSocket → UI shows active users
5. **Video:** WebRTC peer connection established via signaling → Audio/video streams exchanged

### Key Components

| Component | Technology | Responsibility |
|-----------|------------|----------------|
| CRDT Engine | Yjs | Conflict-free document merging |
| WebSocket Client | `ws` (Tauri) | Real-time communication layer |
| WebSocket Server | Node.js + `ws` | Sync hub, presence broadcast |
| WebRTC Manager | PeerJS | P2P video/audio streaming |
| Presence Service | Redis + WebSocket | Track active users/editing state |
| Change Tracker | Yjs + custom logic | Version history, merge logic |

---

## 2. Library Selection: Yjs vs Automerge

### Comparison Matrix

| Criteria | Yjs | Automerge |
|----------|-----|-----------|
| **Maturity** | 8+ years, 500k+ npm downloads/week | 5 years, 50k+ npm downloads/week |
| **Performance** | Excellent (incremental updates) | Good (full snapshot on read) |
| **Bundle Size** | ~18KB (core) | ~25KB (core) |
| **Binding Support** | Tauri (Rust), SolidJS, React, Vue | JavaScript, Rust (beta) |
| **WebSocket Integration** | Excellent (y-websocket official) | Good (community implementations) |
| **WebRTC Integration** | Excellent (y-webrtc official) | Limited (requires custom) |
| **Tauri Compatibility** | Native Rust bindings available | Requires JS bridge |
| **SolidJS Integration** | Reactive bindings available | Requires wrapper |
| **Conflict Resolution** | CRDT (intention-based) | CRDT (Lamport timestamps) |
| **Undo/Redo** | Built-in undo manager | Built-in undo stacks |
| **Persistence** | IndexedDB, LevelDB, custom | Custom (requires implementation) |
| **Community** | Large, active, well-documented | Smaller, niche |
| **TypeScript Support** | Excellent (full types) | Good (improving) |
| **Enterprise Adoption** | WriterDuet, Notion, Tldraw | Figma (early), Obsidian |

### Recommendation: **Yjs**

**Rationale:**
1. **Ecosystem Fit:** Yjs has official `y-websocket` and `y-webrtc` bindings, reducing implementation complexity
2. **Tauri Integration:** Native Rust bindings align with our Tauri desktop architecture
3. **SolidJS Compatibility:** Reactive bindings available; works well with Solid's fine-grained reactivity
4. **Performance:** Incremental updates better suited for real-time collaboration on large documents
5. **Industry Validation:** WriterDuet (our primary competitor) uses CRDT-based sync; Yjs is battle-tested
6. **Bundle Size:** Smaller footprint important for Tauri app startup time (<2s target)
7. **Documentation:** More comprehensive docs and community examples accelerate development

### Yjs Architecture for Our Use Case

```typescript
import { Doc } from 'yjs';
import { WebSocketProvider } from 'y-websocket';
import { WebrtcProvider } from 'y-webrtc';

// Document structure for screenplay
const doc = new Doc();
const text = doc.getText('main'); // Screenplay content
const meta = doc.getMap('metadata'); // Character refs, scene info
const presence = doc.getMap('presence'); // User cursors/selections

// WebSocket sync (primary)
const wsProvider = new WebSocketProvider(
  'wss://api.frenocorp.com/sync',
  'project-{projectId}',
  doc
);

// WebRTC fallback (P2P when both users online)
const rtcProvider = new WebrtcProvider(
  'project-{projectId}',
  doc,
  { signaling: ['wss://signaling.frenocorp.com'] }
);
```

---

## 3. Component Breakdown

### 3.1 Core Collaboration Stack

#### `WebSocketConnectionManager`
**Location:** `src/lib/collaboration/websocket-connection.ts`  
**Responsibilities:**
- Establish and maintain WebSocket connection to sync server
- Handle connection state (connecting, connected, disconnected, reconnected)
- Implement exponential backoff reconnection strategy
- Authenticate connection with JWT token
- Send/receive CRDT updates via WebSocket protocol

**Interface:**
```typescript
interface WebSocketConnectionManager {
  connect(token: string, projectId: string): Promise<void>;
  disconnect(): void;
  sendUpdate(update: Uint8Array): void;
  onUpdate(callback: (update: Uint8Array, origin: string) => void): void;
  onStatusChange(callback: (status: 'connecting' | 'connected' | 'disconnected') => void): void;
  getProvider(): WebSocketProvider; // Yjs provider
}
```

#### `CRDTDocumentManager`
**Location:** `src/lib/collaboration/crdt-document.ts`  
**Responsibilities:**
- Initialize and manage Yjs document lifecycle
- Handle document loading from persistence (Turso/IndexedDB)
- Manage Yjs shared types (Text, Map, Array) for screenplay structure
- Coordinate local changes with remote sync
- Implement undo/redo stacks

**Interface:**
```typescript
interface CRDTDocumentManager {
  initialize(projectId: string): Promise<Doc>;
  getText(type: string): Text;
  getMap(type: string): Map<any>;
  applyRemoteUpdate(update: Uint8Array, origin: string): void;
  createUndoStack(): UndoManager;
  destroy(): void;
}
```

#### `WebRTCVideoManager`
**Location:** `src/lib/collaboration/webrtc-video.ts`  
**Responsibilities:**
- Establish P2P WebRTC connections between collaborators
- Manage audio/video stream negotiation
- Handle ICE candidate exchange via signaling server
- Implement fallback to server-relayed (TURN) when P2P fails
- Provide video component integration with SolidJS

**Interface:**
```typescript
interface WebRTCVideoManager {
  initialize(projectId: string, localStream: MediaStream): void;
  connectToPeer(peerId: string): Promise<void>;
  disconnectFromPeer(peerId: string): void;
  getLocalStream(): MediaStream;
  getRemoteStreams(): Map<string, MediaStream>;
  toggleAudio(enabled: boolean): void;
  toggleVideo(enabled: boolean): void;
}
```

### 3.2 Presence Layer

#### `PresenceManager`
**Location:** `src/lib/collaboration/presence.ts`  
**Responsibilities:**
- Track local user's cursor position and selection
- Broadcast presence updates to other collaborators
- Receive and render remote users' cursors/selections
- Implement idle timeout (user marked as inactive after 30s)
- Handle user join/leave events

**Interface:**
```typescript
interface PresenceManager {
  initialize(doc: Doc, userId: string): void;
  updateCursorPosition(cursor: CursorPosition): void;
  updateSelection(selection: SelectionRange): void;
  getRemoteUsers(): Map<string, RemoteUser>;
  onUserJoin(callback: (user: RemoteUser) => void): void;
  onUserLeave(callback: (userId: string) => void): void;
  onUserUpdate(callback: (user: RemoteUser) => void): void;
}

interface CursorPosition {
  userId: string;
  userName: string;
  position: number; // Yjs text position
  color: string; // Cursor color for identification
}

interface RemoteUser {
  userId: string;
  userName: string;
  avatarUrl?: string;
  cursor?: CursorPosition;
  selection?: SelectionRange;
  isEditing: boolean;
  lastActive: Date;
}
```

### 3.3 Change Tracking & Merge

#### `ChangeTracker`
**Location:** `src/lib/collaboration/change-tracker.ts`  
**Responsibilities:**
- Record all changes to the document with metadata (user, timestamp, type)
- Implement version vector for conflict detection
- Track change boundaries for revision highlighting
- Support change acceptance/rejection workflow
- Generate change diff for version history

**Interface:**
```typescript
interface ChangeTracker {
  recordChange(change: DocumentChange): void;
  getChangesInRange(start: number, end: number): DocumentChange[];
  acceptChange(changeId: string): void;
  rejectChange(changeId: string): void;
  generateDiff(version1: Snapshot, version2: Snapshot): ChangeDiff;
  createSnapshot(): Snapshot;
  restoreSnapshot(snapshot: Snapshot): void;
}

interface DocumentChange {
  id: string;
  userId: string;
  timestamp: Date;
  type: 'insert' | 'delete' | 'format';
  position: number;
  length: number;
  content?: string;
  accepted: boolean;
}
```

#### `MergeLogic`
**Location:** `src/lib/collaboration/merge-logic.ts`  
**Responsibilities:**
- Handle complex merge scenarios (concurrent edits to same paragraph)
- Implement screenplay-specific merge rules (dialogue vs action blocks)
- Resolve conflicts when CRDT produces unexpected results
- Provide manual conflict resolution UI fallback

**Interface:**
```typescript
interface MergeLogic {
  applyServerChange(change: ServerChange): MergeResult;
  handleConcurrentEdit(localChange: Change, remoteChange: Change): MergeStrategy;
  resolveConflict(conflict: Conflict): Resolution;
  validateMerge(result: MergeResult): boolean;
}
```

### 3.4 UI Components

#### `CollaborativeEditor`
**Location:** `src/components/editor/collaborative-editor.tsx`  
**Responsibilities:**
- Wrap base screenplay editor with collaboration primitives
- Integrate Yjs text binding with editor instance
- Render remote cursors and selections
- Handle editor focus/blur for presence updates

**Props:**
```typescript
interface CollaborativeEditorProps {
  doc: Doc;
  projectId: string;
  userId: string;
  onCollaboratorJoin?: (user: RemoteUser) => void;
  onCollaboratorLeave?: (userId: string) => void;
}
```

#### `VideoChatOverlay`
**Location:** `src/components/collaboration/video-chat-overlay.tsx`  
**Responsibilities:**
- Display video feeds from WebRTC peers
- Implement toggle for video/audio
- Show connection quality indicator
- Handle layout (grid view for multiple participants)

**Props:**
```typescript
interface VideoChatOverlayProps {
  videoManager: WebRTCVideoManager;
  position: 'bottom-right' | 'bottom-left' | 'floating';
  size: 'mini' | 'normal';
}
```

#### `CollaboratorList`
**Location:** `src/components/collaboration/collaborator-list.tsx`  
**Responsibilities:**
- Display list of active collaborators
- Show what each user is editing (scene, character, etc.)
- Indicate online/offline status
- Provide quick video call initiation

---

## 4. Implementation Phases

### Phase 1: Foundation (Weeks 1-2)
**Goal:** Establish WebSocket connection and basic CRDT sync

**Tasks:**
- [ ] Set up Yjs document structure for screenplay
- [ ] Implement `WebSocketConnectionManager` with reconnection logic
- [ ] Create basic `CRDTDocumentManager` for text sync
- [ ] Set up Node.js WebSocket server with `y-websocket` adapter
- [ ] Implement JWT authentication for connections
- [ ] Create SolidJS bindings for Yjs reactivity
- [ ] Write unit tests for CRDT operations

**Deliverables:**
- Two instances of the app can sync text changes via WebSocket
- Basic undo/redo functionality
- Connection status indicator

**Dependencies:**
- FRE-586 (Core screenplay editor) - need editor to attach to
- FRE-588 (Database schema) - for project metadata

**Blockers:**
- WebSocket server infrastructure deployment

---

### Phase 2: Presence & Visibility (Weeks 3-4)
**Goal:** Show who is online and what they're editing

**Tasks:**
- [ ] Implement `PresenceManager` with cursor tracking
- [ ] Set up Redis for presence state (optional, for scaling)
- [ ] Create `CollaboratorList` component
- [ ] Implement remote cursor rendering
- [ ] Add user idle detection (30s timeout)
- [ ] Create presence update protocol (WebSocket messages)
- [ ] Design and implement "user is editing" indicators

**Deliverables:**
- Visual indicators showing active collaborators
- Remote cursor positions in the editor
- List of online users with editing context

**Dependencies:**
- Phase 1 (WebSocket connection)
- FRE-596 (Authentication) - for user identity

**Blockers:**
- Design approval on presence UI

---

### Phase 3: WebRTC Video Integration (Weeks 5-6)
**Goal:** Enable video chat during collaboration sessions

**Tasks:**
- [ ] Implement `WebRTCVideoManager` with PeerJS
- [ ] Set up WebRTC signaling server (can reuse WebSocket server)
- [ ] Create `VideoChatOverlay` component
- [ ] Implement P2P connection negotiation
- [ ] Add TURN server fallback for NAT traversal
- [ ] Integrate audio/video permissions
- [ ] Add mute/toggle controls

**Deliverables:**
- Video chat between collaborators
- Audio/video toggle controls
- Connection quality indicator

**Dependencies:**
- Phase 2 (Presence - for knowing who to call)
- TURN server infrastructure

**Blockers:**
- WebRTC port firewall configuration
- Media stream permission UX

---

### Phase 4: Change Tracking & Merge (Weeks 7-8)
**Goal:** Full revision control and conflict resolution

**Tasks:**
- [ ] Implement `ChangeTracker` for version history
- [ ] Create snapshot/restore functionality
- [ ] Implement `MergeLogic` for screenplay-specific rules
- [ ] Build change acceptance/rejection UI
- [ ] Add revision highlighting (colored changes)
- [ ] Create version diff viewer
- [ ] Implement conflict resolution fallback UI

**Deliverables:**
- Full version history with snapshots
- Change highlighting in editor
- Accept/reject workflow for revisions
- Conflict resolution UI

**Dependencies:**
- Phase 1-3 (all collaboration layers)
- FRE-594 (Revision tracking - can be merged or parallel)

**Blockers:**
- Decision on revision workflow (automatic vs manual)

---

### Phase 5: Polish & Optimization (Weeks 9-10)
**Goal:** Performance tuning and edge case handling

**Tasks:**
- [ ] Optimize WebSocket message batching
- [ ] Implement offline-first mode with local Yjs persistence
- [ ] Add conflict detection alerts
- [ ] Tune reconnection backoff strategy
- [ ] Implement bandwidth throttling simulation
- [ ] Add collaboration analytics (latency, sync rate)
- [ ] Write integration tests for all collaboration flows
- [ ] Performance benchmarking (large documents, many users)

**Deliverables:**
- <100ms sync latency (meets KPI)
- Offline editing with conflict resolution on reconnect
- Comprehensive test coverage (>80%)
- Performance benchmarks documented

**Dependencies:**
- All previous phases
- FRE-589 (Tauri packaging) - for desktop-specific optimization

**Blockers:**
- Load testing environment setup

---

## 5. Dependencies and Blockers

### Hard Dependencies (Must Complete First)

| Issue | Dependency | Reason |
|-------|------------|--------|
| FRE-586 | Core screenplay editor | Need editor component to attach collaboration layer |
| FRE-588 | Database schema + Drizzle | Project metadata and user identity for sync |
| FRE-596 | Authentication | User identity for presence and change tracking |

### Soft Dependencies (Can Parallelize)

| Issue | Dependency | Reason |
|-------|------------|--------|
| FRE-589 | Tauri packaging | WebSocket integration differs for Tauri vs web |
| FRE-594 | Revision tracking | Overlaps with change tracking phase |

### Infrastructure Blockers

| Blocker | Description | Mitigation |
|---------|-------------|------------|
| WebSocket Server | Node.js server with `y-websocket` | Can use y-websocket official server initially |
| TURN Server | WebRTC relay for NAT traversal | Use free coturn or Google's public TURN |
| Redis Instance | For presence state at scale | Can start with in-memory, add Redis later |
| JWT Auth | For WebSocket authentication | Reuse Clerk tokens from FRE-596 |

### Technical Blockers

| Blocker | Description | Mitigation |
|---------|-------------|------------|
| Tauri WebSocket | Tauri's WebSocket API differs from browser | Use `tauri-plugin-websocket` or fallback to `ws` |
| Yjs SolidJS Integration | Need reactive bindings | Use `y-sweet` or create custom Solid bindings |
| WebRTC in Tauri | Tauri may need native WebRTC module | Use PeerJS with webview fallback |
| Large Document Performance | Yjs may slow with 10k+ characters | Implement document chunking by scene |

### External Dependencies

| Dependency | Source | Timeline |
|------------|--------|----------|
| Clerk Auth Tokens | FRE-596 | Month 1-2 |
| Turso Project IDs | FRE-588 | Month 1 |
| WebSocket Server URL | DevOps | Month 2 |
| TURN Server Config | DevOps | Month 3 |

---

## 6. Risk Assessment

| Risk | Probability | Impact | Mitigation |
|------|-------------|--------|------------|
| Yjs performance degradation with large documents | Medium | High | Implement scene-based document chunking |
| WebRTC P2P failures due to corporate firewalls | High | Medium | Use TURN relay as fallback |
| WebSocket connection instability on mobile | Medium | Medium | Aggressive reconnection with backoff |
| CRDT conflicts produce unexpected results | Low | High | Implement manual conflict resolution UI |
| Presence updates flood network | Medium | Low | Throttle updates (100ms debounce) |
| Tauri WebSocket API limitations | Medium | Medium | Fallback to webview-based WebSocket |

---

## 7. Success Metrics

| Metric | Target | Measurement |
|--------|--------|-------------|
| Sync Latency | <100ms | Time from local change to remote receipt |
| Connection Uptime | >99% | WebSocket connected time / total time |
| Conflict Resolution Time | <500ms | Time to auto-merge concurrent edits |
| Video Call Setup Time | <2s | Time from invite to video visible |
| Presence Accuracy | >95% | Correct user position displayed |
| Offline Sync Recovery | 100% | All changes merged on reconnect |
| Bundle Size Impact | <50KB | Yjs + WebSocket + WebRTC combined |

---

## 8. Future Enhancements (Post-MVP)

1. **Mobile Collaboration:** Extend WebRTC to iOS/Android via Capacitor
2. **Comment Threads:** Add comment CRDT type for inline feedback
3. **Voice Chat:** Add voice-only mode for low-bandwidth scenarios
4. **Screen Sharing:** Extend WebRTC to support screen capture
5. **AI Conflict Resolution:** Use AI to suggest merge strategies
6. **Collaboration Analytics:** Dashboard showing collaboration patterns
7. **Branching Workflows:** Git-like branches for screenplay variants
8. **Real-time Translations:** Collaborative multi-language editing

---

**Document Version:** 1.0  
**Last Updated:** May 22, 2026  
**Author:** CTO (via Paperclip analysis)  
**Review Status:** Pending CMO/Engineering review