FrenoCorp/analysis/fre587_collaboration_layer_plan.md

FRE-587: Real-time Collaboration Layer with WebSocket + WebRTC

Issue Key: FRE-587
Parent: FRE-574 (Technical evaluation for WriterDuet competitor)
Priority: High
Timeline: Months 3-4 (after core editor foundation)
Status: Planned

---

1. Architecture Overview

High-Level Design

The collaboration layer consists of three interconnected systems:

┌─────────────────────────────────────────────────────────────┐
│                     Tauri Desktop App                        │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────────┐   │
│  │  SolidJS UI  │  │  CRDT Engine │  │  WebRTC Video    │   │
│  │  (Editor)    │◄─┤  (Yjs)       │◄─┤  (PeerJS)        │   │
│  └──────┬───────┘  └──────┬───────┘  └────────┬─────────┘   │
│         │                 │                    │            │
│         ▼                 ▼                    ▼            │
│  ┌──────────────────────────────────────────────────────┐   │
│  │              WebSocket Connection Manager            │   │
│  │  (Auth, Presence, Sync, Conflict Resolution)         │   │
│  └──────────────────────────┬───────────────────────────┘   │
└─────────────────────────────┼───────────────────────────────┘
                              │
                    ┌─────────▼─────────┐
                    │  WebSocket Server │
                    │  (Node.js + WS)   │
                    └─────────┬─────────┘
                              │
         ┌────────────────────┼────────────────────┐
         │                    │                    │
         ▼                    ▼                    ▼
┌────────────────┐  ┌──────────────────┐  ┌────────────────┐
│  Presence Hub  │  │  CRDT Sync Hub   │  │  Signaling     │
│  (Redis)       │  │  (Yjs + Turso)   │  │  (WebRTC)      │
└────────────────┘  └──────────────────┘  └────────────────┘

Data Flow

1. Local Edit: User types → SolidJS editor updates Yjs document → CRDT generates operation
2. Sync: Operation sent via WebSocket to server → Server broadcasts to other clients
3. Merge: Remote operations received → Yjs applies OT/CRDT merge → UI updates
4. Presence: Presence updates broadcast via WebSocket → UI shows active users
5. Video: WebRTC peer connection established via signaling → Audio/video streams exchanged

Key Components

Component	Technology	Responsibility
CRDT Engine	Yjs	Conflict-free document merging
WebSocket Client	ws (Tauri)	Real-time communication layer
WebSocket Server	Node.js + ws	Sync hub, presence broadcast
WebRTC Manager	PeerJS	P2P video/audio streaming
Presence Service	Redis + WebSocket	Track active users/editing state
Change Tracker	Yjs + custom logic	Version history, merge logic

---

2. Library Selection: Yjs vs Automerge

Comparison Matrix

Criteria	Yjs	Automerge
Maturity	8+ years, 500k+ npm downloads/week	5 years, 50k+ npm downloads/week
Performance	Excellent (incremental updates)	Good (full snapshot on read)
Bundle Size	~18KB (core)	~25KB (core)
Binding Support	Tauri (Rust), SolidJS, React, Vue	JavaScript, Rust (beta)
WebSocket Integration	Excellent (y-websocket official)	Good (community implementations)
WebRTC Integration	Excellent (y-webrtc official)	Limited (requires custom)
Tauri Compatibility	Native Rust bindings available	Requires JS bridge
SolidJS Integration	Reactive bindings available	Requires wrapper
Conflict Resolution	CRDT (intention-based)	CRDT (Lamport timestamps)
Undo/Redo	Built-in undo manager	Built-in undo stacks
Persistence	IndexedDB, LevelDB, custom	Custom (requires implementation)
Community	Large, active, well-documented	Smaller, niche
TypeScript Support	Excellent (full types)	Good (improving)
Enterprise Adoption	WriterDuet, Notion, Tldraw	Figma (early), Obsidian

Recommendation: Yjs

Rationale:

1. Ecosystem Fit: Yjs has official y-websocket and y-webrtc bindings, reducing implementation complexity
2. Tauri Integration: Native Rust bindings align with our Tauri desktop architecture
3. SolidJS Compatibility: Reactive bindings available; works well with Solid's fine-grained reactivity
4. Performance: Incremental updates better suited for real-time collaboration on large documents
5. Industry Validation: WriterDuet (our primary competitor) uses CRDT-based sync; Yjs is battle-tested
6. Bundle Size: Smaller footprint important for Tauri app startup time (<2s target)
7. Documentation: More comprehensive docs and community examples accelerate development

Yjs Architecture for Our Use Case

import { Doc } from 'yjs';
import { WebSocketProvider } from 'y-websocket';
import { WebrtcProvider } from 'y-webrtc';

// Document structure for screenplay
const doc = new Doc();
const text = doc.getText('main'); // Screenplay content
const meta = doc.getMap('metadata'); // Character refs, scene info
const presence = doc.getMap('presence'); // User cursors/selections

// WebSocket sync (primary)
const wsProvider = new WebSocketProvider(
  'wss://api.frenocorp.com/sync',
  'project-{projectId}',
  doc
);

// WebRTC fallback (P2P when both users online)
const rtcProvider = new WebrtcProvider(
  'project-{projectId}',
  doc,
  { signaling: ['wss://signaling.frenocorp.com'] }
);

---

3. Component Breakdown

3.1 Core Collaboration Stack

WebSocketConnectionManager

Location: src/lib/collaboration/websocket-connection.ts
Responsibilities:

• Establish and maintain WebSocket connection to sync server
• Handle connection state (connecting, connected, disconnected, reconnected)
• Implement exponential backoff reconnection strategy
• Authenticate connection with JWT token
• Send/receive CRDT updates via WebSocket protocol

Interface:

interface WebSocketConnectionManager {
  connect(token: string, projectId: string): Promise<void>;
  disconnect(): void;
  sendUpdate(update: Uint8Array): void;
  onUpdate(callback: (update: Uint8Array, origin: string) => void): void;
  onStatusChange(callback: (status: 'connecting' | 'connected' | 'disconnected') => void): void;
  getProvider(): WebSocketProvider; // Yjs provider
}

CRDTDocumentManager

Location: src/lib/collaboration/crdt-document.ts
Responsibilities:

• Initialize and manage Yjs document lifecycle
• Handle document loading from persistence (Turso/IndexedDB)
• Manage Yjs shared types (Text, Map, Array) for screenplay structure
• Coordinate local changes with remote sync
• Implement undo/redo stacks

Interface:

interface CRDTDocumentManager {
  initialize(projectId: string): Promise<Doc>;
  getText(type: string): Text;
  getMap(type: string): Map<any>;
  applyRemoteUpdate(update: Uint8Array, origin: string): void;
  createUndoStack(): UndoManager;
  destroy(): void;
}

WebRTCVideoManager

Location: src/lib/collaboration/webrtc-video.ts
Responsibilities:

• Establish P2P WebRTC connections between collaborators
• Manage audio/video stream negotiation
• Handle ICE candidate exchange via signaling server
• Implement fallback to server-relayed (TURN) when P2P fails
• Provide video component integration with SolidJS

Interface:

interface WebRTCVideoManager {
  initialize(projectId: string, localStream: MediaStream): void;
  connectToPeer(peerId: string): Promise<void>;
  disconnectFromPeer(peerId: string): void;
  getLocalStream(): MediaStream;
  getRemoteStreams(): Map<string, MediaStream>;
  toggleAudio(enabled: boolean): void;
  toggleVideo(enabled: boolean): void;
}

3.2 Presence Layer

PresenceManager

Location: src/lib/collaboration/presence.ts
Responsibilities:

• Track local user's cursor position and selection
• Broadcast presence updates to other collaborators
• Receive and render remote users' cursors/selections
• Implement idle timeout (user marked as inactive after 30s)
• Handle user join/leave events

Interface:

interface PresenceManager {
  initialize(doc: Doc, userId: string): void;
  updateCursorPosition(cursor: CursorPosition): void;
  updateSelection(selection: SelectionRange): void;
  getRemoteUsers(): Map<string, RemoteUser>;
  onUserJoin(callback: (user: RemoteUser) => void): void;
  onUserLeave(callback: (userId: string) => void): void;
  onUserUpdate(callback: (user: RemoteUser) => void): void;
}

interface CursorPosition {
  userId: string;
  userName: string;
  position: number; // Yjs text position
  color: string; // Cursor color for identification
}

interface RemoteUser {
  userId: string;
  userName: string;
  avatarUrl?: string;
  cursor?: CursorPosition;
  selection?: SelectionRange;
  isEditing: boolean;
  lastActive: Date;
}

3.3 Change Tracking & Merge

ChangeTracker

Location: src/lib/collaboration/change-tracker.ts
Responsibilities:

• Record all changes to the document with metadata (user, timestamp, type)
• Implement version vector for conflict detection
• Track change boundaries for revision highlighting
• Support change acceptance/rejection workflow
• Generate change diff for version history

Interface:

interface ChangeTracker {
  recordChange(change: DocumentChange): void;
  getChangesInRange(start: number, end: number): DocumentChange[];
  acceptChange(changeId: string): void;
  rejectChange(changeId: string): void;
  generateDiff(version1: Snapshot, version2: Snapshot): ChangeDiff;
  createSnapshot(): Snapshot;
  restoreSnapshot(snapshot: Snapshot): void;
}

interface DocumentChange {
  id: string;
  userId: string;
  timestamp: Date;
  type: 'insert' | 'delete' | 'format';
  position: number;
  length: number;
  content?: string;
  accepted: boolean;
}

MergeLogic

Location: src/lib/collaboration/merge-logic.ts
Responsibilities:

• Handle complex merge scenarios (concurrent edits to same paragraph)
• Implement screenplay-specific merge rules (dialogue vs action blocks)
• Resolve conflicts when CRDT produces unexpected results
• Provide manual conflict resolution UI fallback

Interface:

interface MergeLogic {
  applyServerChange(change: ServerChange): MergeResult;
  handleConcurrentEdit(localChange: Change, remoteChange: Change): MergeStrategy;
  resolveConflict(conflict: Conflict): Resolution;
  validateMerge(result: MergeResult): boolean;
}

3.4 UI Components

CollaborativeEditor

Location: src/components/editor/collaborative-editor.tsx
Responsibilities:

• Wrap base screenplay editor with collaboration primitives
• Integrate Yjs text binding with editor instance
• Render remote cursors and selections
• Handle editor focus/blur for presence updates

Props:

interface CollaborativeEditorProps {
  doc: Doc;
  projectId: string;
  userId: string;
  onCollaboratorJoin?: (user: RemoteUser) => void;
  onCollaboratorLeave?: (userId: string) => void;
}

VideoChatOverlay

Location: src/components/collaboration/video-chat-overlay.tsx
Responsibilities:

• Display video feeds from WebRTC peers
• Implement toggle for video/audio
• Show connection quality indicator
• Handle layout (grid view for multiple participants)

Props:

interface VideoChatOverlayProps {
  videoManager: WebRTCVideoManager;
  position: 'bottom-right' | 'bottom-left' | 'floating';
  size: 'mini' | 'normal';
}

CollaboratorList

Location: src/components/collaboration/collaborator-list.tsx
Responsibilities:

• Display list of active collaborators
• Show what each user is editing (scene, character, etc.)
• Indicate online/offline status
• Provide quick video call initiation

---

4. Implementation Phases

Phase 1: Foundation (Weeks 1-2)

Goal: Establish WebSocket connection and basic CRDT sync

Tasks:

• Set up Yjs document structure for screenplay
• Implement WebSocketConnectionManager with reconnection logic
• Create basic CRDTDocumentManager for text sync
• Set up Node.js WebSocket server with y-websocket adapter
• Implement JWT authentication for connections
• Create SolidJS bindings for Yjs reactivity
• Write unit tests for CRDT operations

Deliverables:

• Two instances of the app can sync text changes via WebSocket
• Basic undo/redo functionality
• Connection status indicator

Dependencies:

• FRE-586 (Core screenplay editor) - need editor to attach to
• FRE-588 (Database schema) - for project metadata

Blockers:

• WebSocket server infrastructure deployment

---

Phase 2: Presence & Visibility (Weeks 3-4)

Goal: Show who is online and what they're editing

Tasks:

• Implement PresenceManager with cursor tracking
• Set up Redis for presence state (optional, for scaling)
• Create CollaboratorList component
• Implement remote cursor rendering
• Add user idle detection (30s timeout)
• Create presence update protocol (WebSocket messages)
• Design and implement "user is editing" indicators

Deliverables:

• Visual indicators showing active collaborators
• Remote cursor positions in the editor
• List of online users with editing context

Dependencies:

• Phase 1 (WebSocket connection)
• FRE-596 (Authentication) - for user identity

Blockers:

• Design approval on presence UI

---

Phase 3: WebRTC Video Integration (Weeks 5-6)

Goal: Enable video chat during collaboration sessions

Tasks:

• Implement WebRTCVideoManager with PeerJS
• Set up WebRTC signaling server (can reuse WebSocket server)
• Create VideoChatOverlay component
• Implement P2P connection negotiation
• Add TURN server fallback for NAT traversal
• Integrate audio/video permissions
• Add mute/toggle controls

Deliverables:

• Video chat between collaborators
• Audio/video toggle controls
• Connection quality indicator

Dependencies:

• Phase 2 (Presence - for knowing who to call)
• TURN server infrastructure

Blockers:

• WebRTC port firewall configuration
• Media stream permission UX

---

Phase 4: Change Tracking & Merge (Weeks 7-8)

Goal: Full revision control and conflict resolution

Tasks:

• Implement ChangeTracker for version history
• Create snapshot/restore functionality
• Implement MergeLogic for screenplay-specific rules
• Build change acceptance/rejection UI
• Add revision highlighting (colored changes)
• Create version diff viewer
• Implement conflict resolution fallback UI

Deliverables:

• Full version history with snapshots
• Change highlighting in editor
• Accept/reject workflow for revisions
• Conflict resolution UI

Dependencies:

• Phase 1-3 (all collaboration layers)
• FRE-594 (Revision tracking - can be merged or parallel)

Blockers:

• Decision on revision workflow (automatic vs manual)

---

Phase 5: Polish & Optimization (Weeks 9-10)

Goal: Performance tuning and edge case handling

Tasks:

• Optimize WebSocket message batching
• Implement offline-first mode with local Yjs persistence
• Add conflict detection alerts
• Tune reconnection backoff strategy
• Implement bandwidth throttling simulation
• Add collaboration analytics (latency, sync rate)
• Write integration tests for all collaboration flows
• Performance benchmarking (large documents, many users)

Deliverables:

• <100ms sync latency (meets KPI)
• Offline editing with conflict resolution on reconnect
• Comprehensive test coverage (>80%)
• Performance benchmarks documented

Dependencies:

• All previous phases
• FRE-589 (Tauri packaging) - for desktop-specific optimization

Blockers:

• Load testing environment setup

---

5. Dependencies and Blockers

Hard Dependencies (Must Complete First)

Issue	Dependency	Reason
FRE-586	Core screenplay editor	Need editor component to attach collaboration layer
FRE-588	Database schema + Drizzle	Project metadata and user identity for sync
FRE-596	Authentication	User identity for presence and change tracking

Soft Dependencies (Can Parallelize)

Issue	Dependency	Reason
FRE-589	Tauri packaging	WebSocket integration differs for Tauri vs web
FRE-594	Revision tracking	Overlaps with change tracking phase

Infrastructure Blockers

Blocker	Description	Mitigation
WebSocket Server	Node.js server with y-websocket	Can use y-websocket official server initially
TURN Server	WebRTC relay for NAT traversal	Use free coturn or Google's public TURN
Redis Instance	For presence state at scale	Can start with in-memory, add Redis later
JWT Auth	For WebSocket authentication	Reuse Clerk tokens from FRE-596

Technical Blockers

Blocker	Description	Mitigation
Tauri WebSocket	Tauri's WebSocket API differs from browser	Use tauri-plugin-websocket or fallback to ws
Yjs SolidJS Integration	Need reactive bindings	Use y-sweet or create custom Solid bindings
WebRTC in Tauri	Tauri may need native WebRTC module	Use PeerJS with webview fallback
Large Document Performance	Yjs may slow with 10k+ characters	Implement document chunking by scene

External Dependencies

Dependency	Source	Timeline
Clerk Auth Tokens	FRE-596	Month 1-2
Turso Project IDs	FRE-588	Month 1
WebSocket Server URL	DevOps	Month 2
TURN Server Config	DevOps	Month 3

---

6. Risk Assessment

Risk	Probability	Impact	Mitigation
Yjs performance degradation with large documents	Medium	High	Implement scene-based document chunking
WebRTC P2P failures due to corporate firewalls	High	Medium	Use TURN relay as fallback
WebSocket connection instability on mobile	Medium	Medium	Aggressive reconnection with backoff
CRDT conflicts produce unexpected results	Low	High	Implement manual conflict resolution UI
Presence updates flood network	Medium	Low	Throttle updates (100ms debounce)
Tauri WebSocket API limitations	Medium	Medium	Fallback to webview-based WebSocket

---

7. Success Metrics

Metric	Target	Measurement
Sync Latency	<100ms	Time from local change to remote receipt
Connection Uptime	>99%	WebSocket connected time / total time
Conflict Resolution Time	<500ms	Time to auto-merge concurrent edits
Video Call Setup Time	<2s	Time from invite to video visible
Presence Accuracy	>95%	Correct user position displayed
Offline Sync Recovery	100%	All changes merged on reconnect
Bundle Size Impact	<50KB	Yjs + WebSocket + WebRTC combined

---

8. Future Enhancements (Post-MVP)

1. Mobile Collaboration: Extend WebRTC to iOS/Android via Capacitor
2. Comment Threads: Add comment CRDT type for inline feedback
3. Voice Chat: Add voice-only mode for low-bandwidth scenarios
4. Screen Sharing: Extend WebRTC to support screen capture
5. AI Conflict Resolution: Use AI to suggest merge strategies
6. Collaboration Analytics: Dashboard showing collaboration patterns
7. Branching Workflows: Git-like branches for screenplay variants
8. Real-time Translations: Collaborative multi-language editing

---

Document Version: 1.0
Last Updated: May 22, 2026
Author: CTO (via Paperclip analysis)
Review Status: Pending CMO/Engineering review