Collaborative note-taking
Real-Time Collaborative Editor

1. Design philosophy

Collaborative tools such as Google Docs are already standard in university teaching, but commercial tools have two pain points: (1) Student work and Learning history are scattered across external platforms, making them impossible to incorporate into research data; (2) they cannot be integrated with in-course RAG, Aida, or Bloom analysers.

Uedu collaborative notes (UeduNote) are built into the platform, enabling teachers and students to complete group discussion drafts, shared notes and project report drafts in Uedu. When finished, they can export to Course RAG with one click, allowing the class AI assistant to understand the document.

2. CRDT selection: Yjs + pycrdt

The UeduNote backend uses pycrdt (Python implementation of Yjs), and the frontend uses the original Yjs (JavaScript). Both sides share the same Y.Doc binary representation, meaning:

• The backend can continue receiving updates from other collaborators while the Student is offline, preparing accumulated diff for the next person to connect
• Version snapshots can be extracted as plain text directly on the backend server (for example, for RAG exports), without requiring frontend rendering
• APScheduler can periodically persist Y.Doc in memory to the DB, and restore it from the DB when needed

Backend management layer: yjs_doc_manager

Located in utils/yjs_doc_manager.py. The global singleton yjs_manager is responsible for:

• Lazy load: when the first request for a given doc_uuid arrives, load the Y.Doc into memory from the yjs_state BLOB field in MySQL
• Diff calculation: call get_update_for_client(state_vector) to generate incremental updates
• Apply update: receive an incremental update from any client, apply it to the in-memory Y.Doc, and broadcast it to the other clients
• Idle eviction: remove from cache when the document has been idle for more than 5 minutes and the room is empty

3. Synchronisation protocol (Socket.IO)

Namespace: /collab-editor. The event flow follows Yjs's standard three-stage handshake + continuous update:

Permission protection

All socket events use cached authentication via _collab_socket_room_auth() (one DB lookup, stored in the in-memory collab_editor_rooms structure). Subsequent events use an in-memory lookup to determine whether edit, comment or view is allowed. This avoids hitting the DB for every event.

4. Data model

The core schema is defined in sql/collab_editor.sql and sql/collab_editor_yjs.sql:

5. Permission model

Uses a 4-level role + two dimensions:

In addition to the per-user collab_document_collaborators.collab_role, documents have a document-level permission field, which can be set to editable, commentable, view_only or private. The permissions of Group members and community members are also taken into account.

Authorisation cache

Permission calculation is centralised in _collab_get_doc_perm(doc_uuid, user_id). The result is stored in the collab_editor_rooms in-memory structure; subsequent socket events use in-memory lookup only, avoiding DB queries on the hot path.

6. Version strategy and persistence

When to write the version

• Manual save: the front end triggers yjs_save_version, immediately writes to collab_documents and adds a collab_document_versions record
• Deduplication: if the latest version content is the same as the current content, skip creating this version (to avoid version spikes from repeated clicks)
• Periodic persistence: APScheduler scans dirty Y.Doc every 60 seconds and writes to yjs_state + content, but does not create version records
• Idle eviction: room empty + 5 minutes of inactivity → removed from cache after landing

Restore previous version

POST /api/editor/documents/<doc_uuid>/versions/<version_id>/restore behaviour:

1. Create a new Y.Doc from the content of this version
2. Call yjs_manager.reset_doc() to replace the in-memory version
3. Broadcast yjs_full_reset to all connected clients, forcing re-synchronisation
4. At the same time, create a new version record 'restored from version N', preserving the restore event in history

Legacy document migration

Early UeduNote stored plain text only, with no CRDT. The first time an old document is loaded, an initial Y.Doc is created from content and written back to yjs_state, after which it operates in CRDT mode. This is a lazy migration, so no downtime is required.

7. Comments, highlighting and images

Comment anchoring (line-based)

Comments are anchored to line numbers (line_ref), rather than CRDT position indexes. The benefits are:

• Even if the text content has been edited, line numbers still carry stable meaning, making them easy to understand
• No need to couple with Y.Doc item-level tracking, reducing maintenance costs

The trade-off is this: after making major additions and deletions inline, the meaning of the “line in question” in the comment may drift. In practice, this is suitable for university teaching scenarios (moderate comment frequency, and a separate conversation can be opened after substantial revisions).

Highlight

Highlight colour settings are handled by the front end and ultimately written into the Y.Text attribute, synchronised via CRDT together with the text content. Highlight data are not stored separately on the back end.

Image upload and clipboard paste

• Path: in production, NAS_UEDU_PATH/collab_editor_images/; on the test machine, local uploads/collab_editor_images/
• Naming: {timestamp}_{uuid}.{ext}, to avoid naming collisions
• Capacity: 15 MB limit per file
• Security: dual verification with Magic byte + Pillow (see utils/file_validation.py), to prevent polyglot attacks
• Clipboard: the front end listens for paste events and uploads image blobs in multipart form
• serving: /api/editor/documents/<doc_uuid>/images/<filename>, verify view permission; secure_filename() prevents path traversal

8. Export to RAG knowledge base

endpoint: POST /api/editor/documents/<doc_uuid>/export-to-rag. Process:

1. Extract plain text from collab_documents.content
2. Write to a temporary .md file
3. Insert one record into the rag_documents table (classroom_id, file_type=md, status=completed)
4. Trigger the existing RAG pipeline to chunk + embed the file

At present, export is one-way (document → RAG). Future plans include support for two-way synchronisation: when RAG content is updated, the original note owner will be notified automatically.

9. Research applications

Collaborative note-taking provides fine-grained data for the Sociomics dimension (social interaction). Research questions include:

• Collaborative task allocation analysis: reconstruct the author of each character from Y.Doc update clientIDs, and quantify each group member's contribution
• Editing sequence: who wrote first, who revised, who added structure (headings / lists), identifying roles such as “organiser”, “contributor” and “editor”
• Editing time distribution: the impact on final output quality of concentrated pre-deadline rushes versus dispersed incremental accumulation
• Comment → revision conversion rate: after peer comments, what proportion is adopted? What is the adoption delay?
• Co-editing + AI integration: after export to RAG, do the citation frequency and accuracy of AI conversations improve?