<SYSTEM>This is the full developer documentation for Teleportal</SYSTEM>

# Teleportal

> Real-time collaborative editing framework for Y.js <br/>Sync documents with confidence.

## Try it out!

[Section titled “Try it out!”](#try-it-out)

Tip Click on Pepper, the parrot, to open the Teleportal devtools!

## Features

Collaborative Editing

Integrates with any editor supporting Y.js - ProseMirror, Quill, Monaco, BlockNote, Tiptap, and more.

Storage Agnostic

Use any storage backend - Redis, PostgreSQL, S3, Cloudflare R2, or implement your own. Documents are never stored in-memory, making it perfect for scalable deployments.

Transport Flexible

Works with WebSockets, HTTP, Server-Sent Events, or any transport that supports bidirectional communication. Built on web-native Streams APIs.

Runtime Agnostic

Built on web primitives and works on any JavaScript runtime - Node.js, Bun, Deno, or any modern JavaScript environment. It’s only plain JavaScript.

Modular Architecture

Teleportal is built as a composable set of modules that can be used together to make a complete sync server. Use your own auth, storage, and more.

Encryption

End-to-end content encryption is on by default — AES-GCM encrypts document content into sidecars the server never sees, while the CRDT structure stays plaintext so sync still works. Includes key management utilities, encrypted file transfers, and optional storage encryption at rest.

Monitoring & Observability

Built-in Prometheus metrics, health checks, and status endpoints. Track clients, sessions, messages, and performance metrics.

## Show me the code!

* server.ts

  ```typescript
  7 collapsed lines
  import { serve } from "crossws/server";
  import { createTokenManager } from "teleportal/token";
  import { tokenAuthenticatedWebsocketHandler } from "teleportal/websocket-server";
  import { tokenAuthenticatedHTTPHandler } from "teleportal/http";
  import { checkPermissionWithTokenManager, Server } from "teleportal/server";
  import { MemoryDocumentStorage } from "teleportal/storage";


  // token manager is a JWT token verifier and manager.
  const tokenManager = createTokenManager({
  secret: "your-secret-key-here",
  expiresIn: 3600, // 1 hour
  issuer: "teleportal.tools",
  });


  // Create a Teleportal server instance
  const server = new Server({
  // you can use any storage backend you want, this one is in-memory
  storage: new MemoryDocumentStorage(),
  // every message is verified against the token's permissions to the document
  checkPermission: checkPermissionWithTokenManager(tokenManager),
  });


  // Serve the Teleportal server with crossws (for multi-runtime support)
  serve({
  // websocket upgrades are denied if the token is invalid
  websocket: tokenAuthenticatedWebsocketHandler({ server, tokenManager }),
  // HTTP requests are denied if the token is invalid
  fetch: tokenAuthenticatedHTTPHandler({ server, tokenManager }),
  });
  ```

* client.ts

  ```typescript
  4 collapsed lines
  // Client
  import { Provider } from "teleportal/providers";
  import { createEncryptionKey } from "teleportal/encryption-key";


  // Create a provider that connects to the server.
  // Content is end-to-end encrypted by default — pass an encryptionKey.
  const provider = await Provider.create({
    url: "https://teleportal.tools?token=your-token-here",
    document: "my-document",
    encryptionKey: createEncryptionKey(),
  });


  // Wait for the document to sync with the server
  await provider.synced;


  // Insert text into the document
  const ytext = provider.doc.getText("content");
  ytext.insert(0, "Hello, world!");
  ```

With that, you have token-authenticated, real-time collaborative editing with Y.js!


***

## Why Teleportal?

Teleportal is designed not as a standalone server that you need to setup and deploy, but as a framework to build your own sync server, with just the features you need. You don’t need to wait until Teleportal has all of the features you need, no more waiting for new APIs or plugins, just build it!

This allows you to use:

Any storage

Use any storage backend you want, from Redis to PostgreSQL to S3

Any transport

Use any transport you want, from WebSockets to HTTP to HTTP SSE

Any JS runtime

Use any JS runtime you want, from Node.js to Bun to Deno (it even works in the browser!)

Any authentication

Use any authentication provider you want, from JWT to OAuth to custom

[Learn more about Teleportal](/what-is-teleportal/)

# Custom Storage Architecture

> Deep dive into the two-layer storage architecture, the DocumentState model, transactions, and storage composition

Teleportal decouples storage from compute so you can persist documents to any backend – Redis, PostgreSQL, S3, SQLite, or a custom API. The storage system is built on two layers of abstraction that let you write a backend without ever touching the Y.js sync protocol.

## Two-Layer Architecture

[Section titled “Two-Layer Architecture”](#two-layer-architecture)

```
graph TD
    A["DocumentStorage (interface)"] -->|"implements"| B["AbstractDocumentStorage (base class)"]
    B -->|"extends"| C["MemoryDocumentStorage"]
    B -->|"extends"| D["UnstorageDocumentStorage"]
    B -->|"extends"| E["IdbDocumentStorage"]
    B -->|"extends"| G["PostgresDocumentStorage"]
    B -->|"extends"| F["YourCustomStorage"]

    style A stroke-dasharray: 5 5
    style F stroke-dasharray: 5 5
```

**`DocumentStorage`** is the low-level interface the server uses for raw protocol messages. It defines methods for sync-step-1/2, handling updates, reading documents, managing metadata, and transactions. The server talks exclusively to this interface – it never knows which backend is behind it.

**`AbstractDocumentStorage`** is the convenient base class that implements the entire `DocumentStorage` protocol for you. It handles the content-encrypted payload decoding, Y.js update merging, state vector computation, sidecar filtering, attribution storage, deduplication, and metadata bookkeeping. Your backend only implements a small set of persistence primitives.

The abstraction exists because the sync protocol is complex. A raw `DocumentStorage` implementation would need to understand content-encrypted envelopes, V2 update merging, state vector diffing, and sidecar index filtering. `AbstractDocumentStorage` absorbs all of that complexity. Your backend never has to understand the wire protocol – it just stores and retrieves bytes.

## The DocumentState Model

[Section titled “The DocumentState Model”](#the-documentstate-model)

Every persisted document is represented as a `DocumentState`:

```typescript
type DocumentState = {
  update: Uint8Array; // merged Y.js V2 update (CRDT structure)
  sidecars: IndexedSidecar[]; // encrypted content sidecars
};
```

For **unencrypted documents**, the `update` field contains the full Y.js V2 update with all content, and `sidecars` is an empty array.

For **encrypted documents**, the `update` field contains only the CRDT structure (operation metadata without content), and `sidecars` holds the encrypted content blobs. Each sidecar carries an index (which client IDs and clock ranges it covers) and a hash for deduplication.

The same `AbstractDocumentStorage` class handles both modes – pass `encrypted: true` or `encrypted: false` to the constructor. The `encrypted` flag tags metadata; the server treats encrypted and plaintext documents identically because it never sees decrypted content.

## What the Base Class Handles for You

[Section titled “What the Base Class Handles for You”](#what-the-base-class-handles-for-you)

When you extend `AbstractDocumentStorage`, the base class takes care of:

* **Decoding the content-encrypted payload** – incoming updates arrive as an encoded envelope containing a structure update and encrypted sidecars. The base class decodes this before persisting.
* **Merging incoming updates** – uses a merge-on-read strategy where updates are appended to a pending log on write (O(1)), then batch-merged via `Y.mergeUpdatesV2` when the document is read. This trades storage for CPU: writes are cheap, reads pay the merge cost.
* **Computing state vectors** – for sync-step-1/2, the base class computes state vectors from the merged update and diffs them against the client’s vector to produce minimal sync responses.
* **Filtering relevant sidecars** – during sync, only sidecars whose client ID / clock ranges overlap the computed diff are sent, avoiding unnecessary data transfer.
* **Writing size and attribution metadata** – timestamps, size tracking, and optional attribution data are maintained automatically on each update.
* **Deduplication** – sidecar hashes prevent duplicate encrypted content from accumulating, and sidecar compaction records allow the base class to replace multiple sidecars with a single compacted one.

## What You Implement

[Section titled “What You Implement”](#what-you-implement)

When extending `AbstractDocumentStorage`, you implement these abstract persistence primitives:

```typescript
abstract class AbstractDocumentStorage implements DocumentStorage {
  // Pending log (merge-on-read)
  abstract appendUpdate(key: string, entry: PendingUpdate): Promise<void>;
  abstract getPendingUpdates(key: string): Promise<{ updates: PendingUpdate[]; cursor: number }>;
  abstract clearPendingUpdates(key: string, upToCursor: number): Promise<void>;


  // Base (compacted) state
  abstract getBaseState(key: string): Promise<DocumentState | null>;
  abstract replaceBaseState(
    key: string,
    update: Uint8Array,
    sidecars: IndexedSidecar[],
  ): Promise<void>;


  // Metadata
  abstract getDocumentMetadata(key: string): Promise<DocumentMetadata>;
  abstract writeDocumentMetadata(key: string, metadata: DocumentMetadata): Promise<void>;


  // Cleanup
  abstract deleteDocument(key: string): Promise<void>;


  // Optional overrides
  transaction<T>(key: string, cb: () => Promise<T>): Promise<T>; // default: just calls cb()
  storeAttribution(key: string, attribution: EncodedContentMap): Promise<void>; // default: no-op
}
```

The merge-on-read design splits persistence into two concerns:

1. **The pending log** (`appendUpdate`, `getPendingUpdates`, `clearPendingUpdates`) – an append-only queue of unmerged updates. Writes are O(1) appends.
2. **The base state** (`getBaseState`, `replaceBaseState`) – the last fully-merged document snapshot. When reading, the base class materializes all pending updates against this snapshot.

This means your backend only needs an append-capable store (for the log) and a key-value store (for the base state). There is no requirement to understand Y.js encoding.

## Transactions

[Section titled “Transactions”](#transactions)

Concurrent writes to the same document must be serialized to prevent lost updates and corrupt merges. The `transaction()` method wraps storage operations in an atomic scope.

The default implementation simply calls the callback with no locking – this is safe for in-memory or single-process deployments where JavaScript’s event loop provides natural serialization. For production backends with concurrent access, you should override `transaction()` with an appropriate strategy:

* **Database transactions** (PostgreSQL, MySQL) – use your database’s native transaction support with row-level locks.
* **Distributed locks with TTL** (Redis, etcd) – acquire a per-document lock with a timeout. The unstorage implementation uses this approach with a default TTL of 5000ms to prevent deadlocks.
* **Optimistic locking** – use version numbers or ETags; retry on conflict.
* **Sequential execution** – for in-memory stores, the single-threaded event loop is sufficient.

The `transaction()` method is optional but important for production backends. Without it, two concurrent `handleUpdate` calls for the same document could read the same base state, merge independently, and overwrite each other’s changes.

## VirtualStorage Wrapper

[Section titled “VirtualStorage Wrapper”](#virtualstorage-wrapper)

`VirtualStorage` is a configurable decorator that adds write buffering and batching to any `DocumentStorage` implementation. It wraps an existing storage instance and buffers updates in memory, flushing them to the underlying backend in batches.

```typescript
import { VirtualStorage } from "teleportal/storage";


const bufferedStorage = new VirtualStorage(underlyingStorage, {
  batchMaxSize: 100, // flush after 100 buffered updates
  batchWaitMs: 2000, // or after 2 seconds, whichever comes first
});
```

**How it works:**

* **Writes** are buffered in memory and dispatched to a batch processor. The batch flushes when either the size limit or time limit is reached.
* **Reads** flush all pending writes for the requested document before reading from the underlying storage. This ensures read-after-write consistency.
* **Deletes and sync operations** also flush pending writes first.

**When to use it:**

* High-frequency collaborative updates where many small writes would overwhelm the backend.
* Slow storage backends (remote databases, object storage) where reducing round trips matters.
* Write-heavy applications where acknowledgment latency is more important than immediate durability.

**Performance trade-offs:**

* Faster write acknowledgment because updates are buffered rather than immediately persisted.
* Reduced database calls through batching.
* Slight read overhead from flushing pending writes before each read.
* Memory usage proportional to the batch size and number of active documents.
* Buffered writes are lost if the process crashes before a flush.

## Storage Composition

[Section titled “Storage Composition”](#storage-composition)

Storage types in Teleportal are fully independent. `DocumentStorage`, `FileStorage`, and `MilestoneStorage` have no coupling – you can mix backends freely:

* PostgreSQL for documents (strong consistency, transactions)
* S3 for files (cheap, durable blob storage)
* Redis for milestones (fast reads, TTL support)

When sharing the same backing store across multiple storage types, use key prefixes to namespace them and prevent collisions:

```typescript
new UnstorageDocumentStorage(storage, { keyPrefix: "document" });
new UnstorageFileStorage(storage, { keyPrefix: "file" });
new UnstorageMilestoneStorage(storage, { keyPrefix: "milestone" });
```

Each storage instance is constructed independently and passed to its respective handler. The server only receives `DocumentStorage` via its `storage` option; file and milestone storage are wired through RPC handlers.

## Code Reference

[Section titled “Code Reference”](#code-reference)

The following example shows a minimal in-memory implementation of `AbstractDocumentStorage`. This is a reference for the method signatures and expected behavior – see the [custom storage how-to guide](/docs/guides/custom-storage/) for a step-by-step walkthrough.

```typescript
import {
  AbstractDocumentStorage,
  type DocumentState,
  type PendingUpdate,
} from "teleportal/storage";
import type { DocumentMetadata } from "teleportal/storage";
import type { IndexedSidecar } from "teleportal/protocol/encryption";


export class MyCustomStorage extends AbstractDocumentStorage {
  private baseStates = new Map<string, DocumentState>();
  private pendingLogs = new Map<string, PendingUpdate[]>();
  private metadata = new Map<string, DocumentMetadata>();


  // -- Pending log (merge-on-read) --


  async appendUpdate(key: string, entry: PendingUpdate): Promise<void> {
    const list = this.pendingLogs.get(key) ?? [];
    list.push(entry);
    this.pendingLogs.set(key, list);
  }


  async getPendingUpdates(key: string): Promise<{ updates: PendingUpdate[]; cursor: number }> {
    const list = this.pendingLogs.get(key) ?? [];
    return { updates: [...list], cursor: list.length };
  }


  async clearPendingUpdates(key: string, upToCursor: number): Promise<void> {
    const list = this.pendingLogs.get(key);
    if (!list) return;
    if (upToCursor >= list.length) {
      this.pendingLogs.delete(key);
    } else {
      list.splice(0, upToCursor);
    }
  }


  // -- Base state --


  async getBaseState(key: string): Promise<DocumentState | null> {
    return this.baseStates.get(key) ?? null;
  }


  async replaceBaseState(
    key: string,
    update: Uint8Array,
    sidecars: IndexedSidecar[],
  ): Promise<void> {
    this.baseStates.set(key, { update, sidecars });
  }


  // -- Metadata --


  async writeDocumentMetadata(key: string, meta: DocumentMetadata): Promise<void> {
    this.metadata.set(key, meta);
  }


  async getDocumentMetadata(key: string): Promise<DocumentMetadata> {
    return (
      this.metadata.get(key) ?? {
        createdAt: Date.now(),
        updatedAt: Date.now(),
        encrypted: this.encrypted,
      }
    );
  }


  // -- Cleanup --


  async deleteDocument(key: string): Promise<void> {
    this.baseStates.delete(key);
    this.pendingLogs.delete(key);
    this.metadata.delete(key);
  }
}
```

Wire it into a server like any other storage:

```typescript
const server = new Server({
  storage: async (ctx) => new MyCustomStorage(ctx.encrypted),
});
```

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Custom Storage Guide](/docs/guides/custom-storage/) – step-by-step how-to for building a custom backend
* [Persistent Storage](/docs/guides/persistent-storage/) – setting up storage with built-in implementations
* [Server](/docs/core-concepts/server/) – how the server uses storage during sync

# Custom Transport

> Deep guide to building custom transports, understanding the streaming model, and composing middleware

Teleportal’s transport layer is designed to be extended. Any communication channel that can send and receive bytes – WebSockets, WebRTC DataChannels, HTTP long-polling, raw TCP, shared memory – can be wrapped into a Teleportal transport and immediately gain access to the full middleware stack (encryption, rate limiting, logging, validation, acknowledgments).

This guide explains the design principles behind the transport system, walks through each building block in detail, and shows how to implement a realistic custom transport from scratch.

## Streaming Foundation

[Section titled “Streaming Foundation”](#streaming-foundation)

Teleportal’s transport layer is built on **async iterables** – the same pull-based streaming primitive that powers `for await...of` loops in JavaScript. This is a deliberate architectural choice over raw callbacks, event emitters, or Node-specific streams. The design delivers five properties that matter for a sync protocol:

**Composability.** Transports compose naturally, like Unix pipes. Each middleware wraps another transport, transforming messages on the way in and out. Encryption, rate limiting, logging, and validation can be stacked in any order without those layers needing to know about each other. You never have to wire up event listeners between layers or manage subscription lifecycles – each layer is a simple function that takes a transport and returns a transport.

**Backpressure.** When a slow consumer cannot keep up with a fast producer (a client on a degraded network connection receiving a burst of document updates), the async iterable model naturally pauses the producer. The consumer pulls the next batch only when it is ready, preventing unbounded memory growth from buffered messages. This is critical for a sync protocol where update bursts are common during initial document load.

**Runtime portability.** Async iterables are a language-level primitive available in every JavaScript runtime: browsers, Bun, Deno, Node.js, and Cloudflare Workers. By building on a language standard rather than a runtime-specific abstraction, the transport layer works everywhere without polyfills or compatibility shims.

**Cancellation support.** Async iterators support `return()` for clean teardown. When a connection closes, calling `return()` on the iterator propagates cancellation through the entire middleware chain. Combined with `AbortController` integration in sinks, this gives you structured cleanup without dangling listeners.

**Batch-aware processing.** Sources yield arrays of messages (`AsyncIterable<Message[]>`) rather than individual messages. This preserves the natural batching of the underlying transport (a single WebSocket frame might carry multiple protocol messages) and lets middleware process batches efficiently without forcing artificial one-at-a-time overhead.

## Source / Sink / Transport Model

[Section titled “Source / Sink / Transport Model”](#source--sink--transport-model)

The transport system is built from three primitives:

### Source

[Section titled “Source”](#source)

A **Source** produces messages. It exposes a single `source` property – an async iterable that yields batches (arrays) of messages:

```typescript
type Source<Context, AdditionalProperties = {}> = {
  source: AsyncIterable<Message<Context>[]>;
} & AdditionalProperties;
```

You consume a source with a `for await...of` loop. Each iteration yields an array of messages (a batch). The source closes naturally when the underlying connection closes – the loop simply exits.

### Sink

[Section titled “Sink”](#sink)

A **Sink** consumes messages. It exposes a `write` method to send a single message and a `close` method for cleanup:

```typescript
type Sink<Context, AdditionalProperties = {}> = {
  write(message: Message<Context>): void | Promise<void>;
  close(): void;
} & AdditionalProperties;
```

The `write` method can be synchronous or asynchronous. When it returns a `Promise`, the caller can await delivery confirmation from the underlying transport.

### Transport

[Section titled “Transport”](#transport)

A **Transport** combines both Source and Sink. It can read and write messages:

```typescript
type Transport<Context, AdditionalProperties = {}> = Source<Context, AdditionalProperties> &
  Sink<Context, AdditionalProperties>;
```

This is a simple intersection type. A transport has a `source` for reading, a `write` method for writing, and a `close` method for cleanup.

### BinaryTransport

[Section titled “BinaryTransport”](#binarytransport)

A **BinaryTransport** works with raw `Uint8Array` bytes (`BinaryMessage`) instead of decoded `Message` objects:

```typescript
type BinaryTransport<AdditionalProperties = {}> = {
  source: AsyncIterable<BinaryMessage[]>;
  write(message: BinaryMessage): void | Promise<void>;
  close(): void;
} & AdditionalProperties;
```

Binary transports represent what comes directly off the wire before any protocol decoding. They are the starting point for custom transports that deal in raw bytes.

### How They Relate

[Section titled “How They Relate”](#how-they-relate)

```
graph TB
    subgraph "Transport (read + write)"
        direction TB
        Source["Source<br/><code>source: AsyncIterable&lt;Message[]&gt;</code>"]
        Sink["Sink<br/><code>write(msg): void</code><br/><code>close(): void</code>"]
    end

    subgraph "BinaryTransport (raw bytes)"
        direction TB
        BSource["source: AsyncIterable&lt;BinaryMessage[]&gt;"]
        BSink["write(bytes): void<br/>close(): void"]
    end

    BinaryTransport -- "fromBinaryTransport()" --> Transport
    Transport -- "toBinaryTransport()" --> BinaryTransport

    style Source fill:#e8f4e8
    style Sink fill:#e8f0f8
    style BSource fill:#f4e8e8
    style BSink fill:#f4e8e8
```

### AdditionalProperties

[Section titled “AdditionalProperties”](#additionalproperties)

Both Source, Sink, and Transport accept a second type parameter, `AdditionalProperties`. This lets transports attach extra properties – for example, a PubSub transport adds `subscribe()` and `unsubscribe()` methods, and an ACK-tracking sink adds `waitForAcks()`. These properties are preserved through middleware composition via TypeScript’s intersection types.

## Middleware Composition

[Section titled “Middleware Composition”](#middleware-composition)

Transports compose as layers, following a pattern sometimes called the “onion model.” Each middleware wraps another transport, adding functionality. Messages pass through each layer on the way in (source) and on the way out (sink).

### The Layering Pattern

[Section titled “The Layering Pattern”](#the-layering-pattern)

Every middleware function follows the same signature: it takes a transport (or source/sink) and returns a new transport with the same interface plus the added behavior. This means middleware is stackable in any order:

```typescript
import { withLogger, withMessageValidator } from "teleportal/transports";
import { withRateLimit } from "teleportal/transports/rate-limiter";


// Start with the base transport
let transport = getBaseTransport();


// Layer 1: Validation (closest to the application)
transport = withMessageValidator(transport, {
  isAuthorized: async (message, type) => checkAuth(message),
});


// Layer 2: Rate limiting
transport = withRateLimit(transport, { rules: myRules });


// Layer 3: Logging (closest to the wire)
transport = withLogger(transport);
```

### Message Flow Through Layers

[Section titled “Message Flow Through Layers”](#message-flow-through-layers)

When a message is **written** (outbound), it passes through layers from the application outward to the wire. When a message is **read** (inbound), it passes from the wire inward to the application. Each layer can inspect, transform, filter, or reject messages.

```
graph LR
    App["Application"] --> Validation
    Validation --> RateLimit["Rate Limiting"]
    RateLimit --> Logging
    Logging --> Wire["Network"]

    Wire --> Logging2["Logging"]
    Logging2 --> RateLimit2["Rate Limiting"]
    RateLimit2 --> Validation2["Validation"]
    Validation2 --> App2["Application"]

    subgraph "Write Path (outbound)"
        App
        Validation
        RateLimit
        Logging
        Wire
    end

    subgraph "Read Path (inbound)"
        Wire
        Logging2
        RateLimit2
        Validation2
        App2
    end
```

The key insight is that **each middleware is independent**. The rate limiter does not know about the logger. The validator does not know about rate limiting. They are composed by wrapping, not by configuration. This makes each middleware simple to implement, test, and reason about.

### Writing Your Own Middleware

[Section titled “Writing Your Own Middleware”](#writing-your-own-middleware)

A middleware function wraps a transport’s source, sink, or both. Here is a minimal example that counts messages:

```typescript
import type { Transport, Message } from "teleportal";
import { compose, filterMessages, withPassthroughSink } from "teleportal/transports";


function withCounter<
  Context extends Record<string, unknown>,
  Props extends Record<string, unknown>,
>(transport: Transport<Context, Props>): Transport<Context, Props> {
  let readCount = 0;
  let writeCount = 0;


  // Wrap the source to count reads
  const wrappedSource = {
    ...transport,
    source: filterMessages<Message<Context>>((msg) => {
      readCount++;
      console.log(`Read #${readCount}:`, msg.type);
      return true; // pass all messages through
    })(transport.source),
  };


  // Wrap the sink to count writes
  const wrappedSink = withPassthroughSink(transport, {
    onWrite: (msg) => {
      writeCount++;
      console.log(`Write #${writeCount}:`, msg.type);
    },
  });


  return compose(wrappedSource, wrappedSink);
}
```

## Binary vs Message Transport Conversion

[Section titled “Binary vs Message Transport Conversion”](#binary-vs-message-transport-conversion)

Most communication channels deal in raw bytes. WebSocket `onmessage` gives you an `ArrayBuffer`. WebRTC DataChannels produce `Uint8Array`. HTTP responses carry binary payloads. These raw bytes need to be decoded into Teleportal’s `Message` objects before the protocol layer can process them.

Two functions handle this conversion:

### `fromBinaryTransport(binaryTransport, context)`

[Section titled “fromBinaryTransport(binaryTransport, context)”](#frombinarytransportbinarytransport-context)

Converts a `BinaryTransport` (raw bytes) into a `Transport` (decoded messages). This is the function you will use most often when building custom transports:

```typescript
import { fromBinaryTransport } from "teleportal/transports";


const messageTransport = fromBinaryTransport(binaryTransport, {
  clientId: "client-123",
  document: "my-doc",
});
```

During conversion, `fromBinaryTransport` handles two things automatically:

1. **Decoding**: Each `BinaryMessage` is decoded into the appropriate `Message` subtype (document update, awareness update, ACK, RPC, etc.).
2. **Ping/pong**: Ping messages from the server are intercepted and answered with a pong automatically. They never surface to the decoded source. This keeps the connection alive without the application needing to handle keepalive logic.

### `toBinaryTransport(transport, context)`

[Section titled “toBinaryTransport(transport, context)”](#tobinarytransporttransport-context)

Converts a `Transport` (decoded messages) back into a `BinaryTransport` (raw bytes). This is useful when you need to re-encode messages for transmission:

```typescript
import { toBinaryTransport } from "teleportal/transports";


const binaryTransport = toBinaryTransport(messageTransport, context);
```

### When to Use Each

[Section titled “When to Use Each”](#when-to-use-each)

| Scenario                                                     | Function                                      |
| ------------------------------------------------------------ | --------------------------------------------- |
| Custom transport receives raw bytes from the wire            | `fromBinaryTransport` to get decoded messages |
| You need to forward messages as raw bytes to another channel | `toBinaryTransport` to get encodable bytes    |
| The transport already produces decoded `Message` objects     | Neither – use the transport directly          |

## Utility Functions

[Section titled “Utility Functions”](#utility-functions)

The transport module exports several utility functions for composing and connecting transports.

### `compose(source, sink)`

[Section titled “compose(source, sink)”](#composesource-sink)

Combines a separate `Source` and `Sink` into a single `Transport`. This is the primary way to build a transport from its two halves:

```typescript
import { compose } from "teleportal/transports";


const transport = compose(mySource, mySink);
// transport.source  -- from mySource
// transport.write() -- from mySink
// transport.close() -- from mySink
```

Any additional properties from both the source and sink are preserved on the resulting transport via TypeScript intersection types.

### `connect(source, sink)`

[Section titled “connect(source, sink)”](#connectsource-sink)

Pipes messages from a Source into a Sink. This is a one-way flow – every message yielded by the source is written to the sink. The returned promise resolves when the source closes:

```typescript
import { connect } from "teleportal/transports";


// All messages from source are written to sink
await connect(mySource, mySink);
```

`connect` also accepts a bare `AsyncIterable<Message[]>` as the source argument, not just a `Source` object.

### `sync(transportA, transportB)`

[Section titled “sync(transportA, transportB)”](#synctransporta-transportb)

Bidirectional sync – messages from A go to B, and messages from B go to A. This is equivalent to calling `connect` in both directions simultaneously:

```typescript
import { sync } from "teleportal/transports";


// Messages flow both ways
await sync(transportA, transportB);
```

The returned promise resolves when both directions have completed (both sources have closed).

### `createFanOutWriter()`

[Section titled “createFanOutWriter()”](#createfanoutwriter)

Creates a broadcast writer where one producer sends messages to many consumers. Each call to `getReader()` creates an independent consumer that receives all messages sent after it subscribes:

```typescript
import { createFanOutWriter } from "teleportal/transports";


const fanOut = createFanOutWriter<Message>();


// Create readers for each client
const reader1 = fanOut.getReader();
const reader2 = fanOut.getReader();


// Send once -- both readers receive it
fanOut.send(message);


// Each reader has an async iterable source
for await (const batch of reader1.source) {
  // process batch
}


// Unsubscribe a reader when its client disconnects
reader2.unsubscribe();


// Close the writer when done
fanOut.close();
```

This is the core primitive behind server-side message broadcasting to multiple connected clients.

### `forEachMessage(source, fn)`

[Section titled “forEachMessage(source, fn)”](#foreachmessagesource-fn)

Drains a batched source one item at a time, calling `fn` for each message. A convenience for when you want to process messages individually rather than in batches:

```typescript
import { forEachMessage } from "teleportal/transports";


await forEachMessage(mySource, async (message) => {
  console.log("Received:", message.type, message.document);
});
```

### Transform Helpers

[Section titled “Transform Helpers”](#transform-helpers)

Three higher-order functions create transforms over batched async iterables. These are the building blocks for writing middleware that transforms the source side of a transport:

* **`mapMessages(fn)`** – Map each message to a new value, or drop it by returning `null`/`undefined`.
* **`filterMessages(predicate)`** – Keep only messages that pass the predicate.
* **`flatMapMessages(fn)`** – Expand each message into zero or more output messages.

```typescript
import { mapMessages, filterMessages } from "teleportal/transports";


// Drop awareness messages from a source
const filteredSource = filterMessages<Message<MyContext>>((msg) => msg.type !== "awareness")(
  transport.source,
);


// Transform messages
const mappedSource = mapMessages((msg: Message<MyContext>) => {
  // Return null to drop, or a transformed message to keep
  return msg.type === "doc" ? msg : null;
})(transport.source);
```

## Custom Transport Example: WebRTC DataChannel

[Section titled “Custom Transport Example: WebRTC DataChannel”](#custom-transport-example-webrtc-datachannel)

Here is a realistic example of wrapping a WebRTC DataChannel as a Teleportal transport. The DataChannel sends and receives raw bytes, so we build a `BinaryTransport` first and then convert it to a `Transport` with `fromBinaryTransport`.

```typescript
import {
  createChannel,
  compose,
  fromBinaryTransport,
  withLogger,
  withMessageValidator,
} from "teleportal/transports";
import type { BinaryTransport, BinaryMessage, Message } from "teleportal";


type MyContext = { clientId: string; document: string };


function createDataChannelTransport(dataChannel: RTCDataChannel, context: MyContext) {
  // -- Step 1: Build a BinaryTransport from the DataChannel --


  // Source: read raw bytes from the DataChannel
  const channel = createChannel<BinaryMessage>();


  dataChannel.binaryType = "arraybuffer";
  dataChannel.onmessage = (event) => {
    channel.send(new Uint8Array(event.data));
  };
  dataChannel.onclose = () => {
    channel.close();
  };
  dataChannel.onerror = (err) => {
    channel.error(err);
  };


  // Sink: write raw bytes to the DataChannel
  const binaryTransport: BinaryTransport = {
    source: channel,
    write(message: BinaryMessage) {
      if (dataChannel.readyState === "open") {
        dataChannel.send(message);
      }
    },
    close() {
      dataChannel.close();
      channel.close();
    },
  };


  // -- Step 2: Convert from binary to message transport --
  // This handles protocol decoding and automatic ping/pong
  let transport = fromBinaryTransport(binaryTransport, context);


  // -- Step 3: Layer on middleware --
  transport = withMessageValidator(transport, {
    isAuthorized: async (message, type) => {
      // Only allow messages for the expected document
      return message.document === context.document;
    },
  });


  transport = withLogger(transport);


  return transport;
}
```

The key steps are:

1. **Build a BinaryTransport** by creating a `Channel` (async iterable) for the source side and implementing `write`/`close` for the sink side.
2. **Convert to a message transport** with `fromBinaryTransport`, which handles decoding and ping/pong.
3. **Layer middleware** using the standard `with*` functions.

The same pattern applies to any byte-oriented channel: a custom TCP connection, a shared memory buffer, a Unix domain socket, or a postMessage bridge between workers.

## Available Middleware

[Section titled “Available Middleware”](#available-middleware)

Teleportal ships with several built-in middleware transports. Each follows the same wrapping pattern and can be composed in any order.

### `withLogger`

[Section titled “withLogger”](#withlogger)

Logs all messages read from and written to the transport to the console. Useful for debugging during development:

```typescript
import { withLogger } from "teleportal/transports";


const logged = withLogger(transport);
```

### `withRateLimit`

[Section titled “withRateLimit”](#withratelimit)

Enforces rate limiting using a token bucket algorithm. Supports per-user, per-document, per-user-document, or per-transport tracking. Messages that exceed the rate limit are silently dropped:

```typescript
import { withRateLimit } from "teleportal/transports/rate-limiter";


const limited = withRateLimit(transport, {
  rules: [
    {
      id: "per-user",
      maxMessages: 300,
      windowMs: 1000,
      trackBy: "user",
    },
  ],
  onRateLimitExceeded: (details) => {
    console.warn("Rate limited:", details.ruleId, details.userId);
  },
});
```

### `withMessageValidator`

[Section titled “withMessageValidator”](#withmessagevalidator)

Adds authorization checks to messages on read, write, or both. Unauthorized messages are silently filtered out:

```typescript
import { withMessageValidator } from "teleportal/transports";


const validated = withMessageValidator(transport, {
  isAuthorized: async (message, type) => {
    // type is "read" or "write"
    return await checkPermissions(message.context.userId, message.document);
  },
});
```

Separate `withMessageValidatorSource` and `withMessageValidatorSink` functions are available when you only need to validate one direction.

### `withPassthrough`

[Section titled “withPassthrough”](#withpassthrough)

A general-purpose interceptor for inspecting or filtering messages without modifying them. Return `false` from a callback to drop the message:

```typescript
import { withPassthrough } from "teleportal/transports";


const inspected = withPassthrough(transport, {
  onRead: (message) => {
    metrics.increment("messages.read");
    // return false to drop, void/true to pass through
  },
  onWrite: (message) => {
    metrics.increment("messages.write");
  },
});
```

Separate `withPassthroughSource` and `withPassthroughSink` functions are available for one-sided interception.

### `withAckSink` / `withAckTrackingSink`

[Section titled “withAckSink / withAckTrackingSink”](#withacksink--withacktrackingsink)

Add reliable delivery semantics through acknowledgment messages. `withAckSink` is used on the server side to automatically send ACK messages after processing. `withAckTrackingSink` is used on the client side to track sent messages and wait until all are acknowledged:

```typescript
import { withAckSink, withAckTrackingSink } from "teleportal/transports";


// Server: auto-send ACKs after writing
const serverSink = withAckSink(sink, {
  pubSub,
  ackTopic: "acks",
  sourceId: "server-1",
  context: serverContext,
});


// Client: track messages and wait for ACKs
const clientSink = withAckTrackingSink(sink, {
  pubSub,
  ackTopic: "acks",
  sourceId: "client-1",
  ackTimeout: 10000,
});


// Write a message and wait for all ACKs
clientSink.write(message);
await clientSink.waitForAcks();


// Clean up when done
await clientSink.unsubscribe();
```

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Transport](/docs/core-concepts/transport/) – Core concepts overview of the transport system
* [Server](/docs/core-concepts/server/) – How the server uses transports to handle connections
* [Performance](/docs/advanced/performance/) – Optimizing transport and middleware performance

# DevTools

> Visualize message flow and inspect message contents in real-time

Teleportal includes built-in DevTools for debugging and monitoring your application. The DevTools provide real-time visibility into message flow, connection state, document activity, and system statistics.

## Overview

[Section titled “Overview”](#overview)

The Teleportal DevTools helps you debug and monitor your Teleportal applications by providing:

* **Real-time message monitoring**: Track all sent and received messages with detailed metadata
* **Connection state tracking**: Monitor connection status, transport type, and errors
* **Message filtering**: Filter messages by document, type, direction, and search text
* **Message inspection**: View detailed payloads, metadata, and acknowledgment status
* **Statistics**: Track message counts, rates, document counts, and message types

The UI is organized into three tabs – **Messages**, **Documents**, and **Presence** – plus an always-visible connection status area with a details popover:

* **Header bar**: a tab bar (Messages / Documents / Presence) with count badges (document count, peer count), and a connection status indicator. Clicking the connection status opens a popover with live internals (in-flight/buffered counts, AIMD batch window, reconnect attempts, SharedWorker pooling details, and a timeline of state transitions).
* **Messages tab**: a collapsible filters row (search, direction selector, document checkboxes, message type checkboxes, message limit input), a scrollable message list (left, newest first), and a message inspector (right). RPC request/stream/response messages collapse into a single call row with status, latency, and error details.
* **Documents tab**: a tree of documents and subdocuments with live sync handshake state (sync-step-1 → sync-step-2 → synced), traffic counters, encryption, and last activity. Clicking a document filters the Messages tab.
* **Presence tab**: a live peer roster derived from presence-join/leave/heartbeat messages, with expandable per-peer data and a recent join/leave feed.

The connection status indicator uses color coding: green for connected, yellow for connecting, gray for disconnected, and red for errored.

All settings – including filters, message limit, and panel state – persist in localStorage across sessions.

## Basic Integration

[Section titled “Basic Integration”](#basic-integration)

```typescript
import { createTeleportalDevtools } from "teleportal/devtools";


// Create the devtools element
const devtoolsElement = createTeleportalDevtools();


// Append to your DOM
document.body.appendChild(devtoolsElement);
```

## React Integration

[Section titled “React Integration”](#react-integration)

```tsx
import { useState, useEffect, useRef } from "react";
import { createTeleportalDevtools, getDevtoolsState } from "teleportal/devtools";


export function TeleportalDevtoolsPanel() {
  const containerRef = useRef<HTMLDivElement>(null);
  const devtoolsRef = useRef<HTMLElement | null>(null);
  const [state] = useState(() => getDevtoolsState());


  useEffect(() => {
    if (!containerRef.current) return;


    const devtoolsElement = createTeleportalDevtools(state);
    containerRef.current.appendChild(devtoolsElement);
    devtoolsRef.current = devtoolsElement;


    return () => {
      if (devtoolsRef.current) {
        const cleanup = (devtoolsRef.current as any).__teleportalDevtoolsCleanup;
        if (cleanup) {
          cleanup();
        }
        if (containerRef.current && devtoolsRef.current.parentNode === containerRef.current) {
          containerRef.current.removeChild(devtoolsRef.current);
        }
      }
    };
  }, []);


  return <div ref={containerRef} style={{ height: "100%", width: "100%" }} />;
}
```

## Complete Vite + React Integration Example

[Section titled “Complete Vite + React Integration Example”](#complete-vite--react-integration-example)

In a real application you typically want the devtools to appear only during development and to be toggleable so it does not obscure the rest of the UI. The example below shows a bottom-drawer pattern with conditional rendering and proper cleanup.

src/components/DevtoolsDrawer.tsx

```tsx
import { useState, useEffect, useRef, useCallback } from "react";
import { createTeleportalDevtools, getDevtoolsState } from "teleportal/devtools";


export function DevtoolsDrawer() {
  const [open, setOpen] = useState(false);
  const containerRef = useRef<HTMLDivElement>(null);
  const devtoolsRef = useRef<HTMLElement | null>(null);
  const [state] = useState(() => getDevtoolsState());


  useEffect(() => {
    if (!open || !containerRef.current) return;


    const devtoolsElement = createTeleportalDevtools(state);
    containerRef.current.appendChild(devtoolsElement);
    devtoolsRef.current = devtoolsElement;


    return () => {
      if (devtoolsRef.current) {
        const cleanup = (devtoolsRef.current as any).__teleportalDevtoolsCleanup;
        if (cleanup) cleanup();
        devtoolsRef.current.remove();
        devtoolsRef.current = null;
      }
    };
  }, [open]);


  return (
    <>
      <button
        onClick={() => setOpen((prev) => !prev)}
        style={{
          position: "fixed",
          bottom: open ? 320 : 8,
          right: 8,
          zIndex: 10000,
        }}
      >
        {open ? "Close DevTools" : "Open DevTools"}
      </button>


      {open && (
        <div
          ref={containerRef}
          style={{
            position: "fixed",
            bottom: 0,
            left: 0,
            right: 0,
            height: 300,
            zIndex: 9999,
            borderTop: "2px solid #333",
          }}
        />
      )}
    </>
  );
}
```

src/App.tsx

```tsx
import { DevtoolsDrawer } from "./components/DevtoolsDrawer";


function App() {
  return (
    <>
      {/* Your application content */}
      {import.meta.env.DEV && <DevtoolsDrawer />}
    </>
  );
}
```

Key points:

* `import.meta.env.DEV` is a Vite built-in that is `true` only during development, so the drawer is tree-shaken out of production builds.
* The devtools element is created when the drawer opens and cleaned up when it closes, so resources are not held when the panel is hidden.
* The `__teleportalDevtoolsCleanup` function unsubscribes all event listeners and clears internal state, preventing memory leaks.

## Message Monitoring Capabilities

[Section titled “Message Monitoring Capabilities”](#message-monitoring-capabilities)

The DevTools capture every message flowing through the Teleportal provider in real time. Each message is stored with metadata including its unique ID, direction (sent or received), timestamp, associated document, and the provider instance that handled it.

Key behaviors:

* **ACK tracking**: ACK messages are hidden from the message list to reduce noise, but they are tracked separately. When an ACK is received, the original message it acknowledges is updated with an ACK indicator, and the inspector shows the ACK details (ACK message ID, acknowledged message ID, and timestamp) in an expandable section.
* **Message deduplication**: Messages are deduplicated by their ID using an O(1) index lookup, so duplicate deliveries do not produce duplicate entries in the list.
* **Configurable message limit**: The message limit (default: 200) caps how many messages are kept in memory. When the limit is exceeded, the oldest messages are removed and their contributions to the statistics are subtracted. You can change the limit at any time via the input in the filters header.
* **Message rate**: The rate is calculated as the number of messages received or sent within the last 10 seconds, divided by 10, giving a smoothed messages-per-second value.
* **Per-document tracking**: A `DocumentTracker` maintains a registry of all documents seen in messages, recording their message counts, last activity timestamps, and provider associations.

## Message Types and Color Coding

[Section titled “Message Types and Color Coding”](#message-types-and-color-coding)

The DevTools recognize and color-code messages by the wire protocol’s message kinds – `doc`, `awareness`, `presence`, `rpc`, and `ack`. `getMessageTypeLabel()` derives the row label and `getMessageTypeColor()` its badge color.

### Document (`doc`) Messages

[Section titled “Document (doc) Messages”](#document-doc-messages)

| Type           | Color        |
| -------------- | ------------ |
| `sync-step-1`  | Blue         |
| `sync-step-2`  | Darker blue  |
| `update`       | Green        |
| `sync-done`    | Darker green |
| `auth-message` | Red          |

### Awareness Messages

[Section titled “Awareness Messages”](#awareness-messages)

| Type                | Color         |
| ------------------- | ------------- |
| `awareness-update`  | Yellow        |
| `awareness-request` | Darker yellow |

### Presence Messages

[Section titled “Presence Messages”](#presence-messages)

| Type                                                                            | Color  |
| ------------------------------------------------------------------------------- | ------ |
| `presence-join` / `presence-leave` / `presence-heartbeat` / `presence-announce` | Purple |

The row label is the payload’s `type`; presence messages also feed the roster in the Presence tab.

### RPC (`rpc`) Messages

[Section titled “RPC (rpc) Messages”](#rpc-rpc-messages)

RPC is the transport for milestones (`listMilestones`, …), the key registry, and file transfers (`fileUpload` / `fileDownload`). There is **no** standalone `milestone-*` or `file-*` message type – those are RPC methods carried over `rpc` messages. The badge is indigo, shaded by `requestType`:

| Request type  | Label                | Color                   |
| ------------- | -------------------- | ----------------------- |
| request       | `<rpcMethod>`        | Darkest indigo (`600`)  |
| response      | `<rpcMethod>`        | Medium indigo (`500`)   |
| stream / part | `<rpcMethod> (part)` | Lightest indigo (`400`) |

RPC messages sharing an `originalRequestId` are grouped into a single call row. File-transfer upload chunks never appear as messages – upload progress comes from the file protocol’s progress events instead.

### ACK Messages

[Section titled “ACK Messages”](#ack-messages)

| Type  | Color                               |
| ----- | ----------------------------------- |
| `ack` | Gray (hidden from list but tracked) |

## Filtering

[Section titled “Filtering”](#filtering)

The DevTools provide filtering controls in the collapsible filters panel. All filter settings persist in localStorage and are restored on page reload.

### Document Filter

[Section titled “Document Filter”](#document-filter)

Multi-select checkboxes listing every document that has appeared in a message. Documents are auto-discovered as messages arrive – there is no manual configuration. Select one or more documents to show only their messages.

### Message Type Filter

[Section titled “Message Type Filter”](#message-type-filter)

Checkboxes for each message type allow you to hide specific types from the list. This is useful for focusing on a particular category, for example hiding all awareness updates to concentrate on sync traffic.

### Direction Filter

[Section titled “Direction Filter”](#direction-filter)

A selector with three options:

* **All**: Show both sent and received messages
* **Sent**: Show only outgoing messages
* **Received**: Show only incoming messages

### Search Filter

[Section titled “Search Filter”](#search-filter)

A text input that searches across message payloads and document IDs. The search is case-insensitive and debounced by 300ms so that filtering does not block the UI during rapid typing.

## Debugging Sync Issues

[Section titled “Debugging Sync Issues”](#debugging-sync-issues)

The DevTools are especially useful for diagnosing common synchronization problems.

### Sync not completing

[Section titled “Sync not completing”](#sync-not-completing)

Look for a `sync-step-1` message that was sent but no corresponding `sync-step-2` received. This means the server did not respond to the initial sync request. Check whether an `auth-message` was received instead, which indicates a permission or authentication error.

### Duplicate updates

[Section titled “Duplicate updates”](#duplicate-updates)

If you see an unusually high message rate or the same document data arriving repeatedly, check the message IDs. The DevTools deduplicate by ID, so identical IDs will not appear twice. If many distinct IDs carry the same payload, the application may have an update loop – a common cause is subscribing to Y.js `update` events and re-broadcasting them without checking their origin.

### Connection instability

[Section titled “Connection instability”](#connection-instability)

Watch the connection status indicator in the filters header. If it cycles rapidly between connecting (yellow) and disconnected (gray), there is likely a network issue, a server-side rejection, or a misconfigured reconnection strategy. Check for `auth-message` types with error codes that might explain the rejection.

### Missing awareness updates

[Section titled “Missing awareness updates”](#missing-awareness-updates)

Filter to awareness messages only using the message type filter. Verify that `awareness-request` messages are being sent. If they are sent but no `awareness-update` is received in response, the server may not be forwarding awareness state, or the document ID in the request may not match any active document on the server.

### File upload failures

[Section titled “File upload failures”](#file-upload-failures)

File transfers run over RPC. Filter to `rpc` messages and look for the `fileUpload` / `fileDownload` call rows. A failed call shows its error code and details in the call row and inspector. Note that upload chunks never appear as messages – they stay off the event pipeline for throughput, so upload progress and completion come from the file protocol’s progress events rather than visible `stream` parts. Download parts, by contrast, arrive as received messages and are grouped under their call row.

## Troubleshooting Tips

[Section titled “Troubleshooting Tips”](#troubleshooting-tips)

### DevTools not showing messages

[Section titled “DevTools not showing messages”](#devtools-not-showing-messages)

Ensure the devtools element is created **after** the Provider is initialized. The `EventManager` subscribes to `teleportalEventClient` events at construction time. If the devtools are created before the provider emits events, the subscriptions will be in place but no events will flow until the provider connects.

### High memory usage

[Section titled “High memory usage”](#high-memory-usage)

Lower the message limit in the filters header. The default of 200 is conservative, but for high-traffic applications with many concurrent documents or frequent awareness updates, consider setting it to 50-100. Each stored message retains references to the original message object and the provider, so large payloads can add up.

### Settings not persisting

[Section titled “Settings not persisting”](#settings-not-persisting)

The DevTools store settings in `localStorage` under the key `teleportal-devtools-settings`. If settings are not persisting, check whether localStorage is available in your environment. Private browsing modes, iframe sandbox restrictions, and certain browser extensions can block localStorage access.

### Cleanup

[Section titled “Cleanup”](#cleanup)

Always call the cleanup function when the devtools element is removed from the DOM to prevent memory leaks from lingering event listeners:

```typescript
const devtoolsElement = createTeleportalDevtools();


// Later, when cleaning up:
const cleanup = (devtoolsElement as any).__teleportalDevtoolsCleanup;
if (cleanup) {
  cleanup();
}
```

This unsubscribes from all `teleportalEventClient` event listeners, clears internal state, and removes subscriber callbacks.

## Event System Integration

[Section titled “Event System Integration”](#event-system-integration)

The DevTools integrate with the Teleportal event system through `teleportalEventClient` from `teleportal/providers`. The `EventManager` subscribes to the following events at construction time:

* `received-message` – captures every incoming message with its provider and connection context
* `sent-message` – captures every outgoing message
* `connected` – updates the connection state indicator to connected
* `disconnected` – updates the connection state indicator to disconnected
* `update` – handles connection state transitions (connecting, errored, etc.)
* `load-subdoc` – registers a new subdocument in the document tracker
* `unload-subdoc` – removes a subdocument from the document tracker

Because these are global events emitted by the provider, the DevTools automatically capture traffic from all providers and connections active on the page.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Provider](/docs/core-concepts/provider/) – Learn how providers work and how events flow
* [Protocol](/docs/core-concepts/protocol/) – Understand the message types the DevTools display
* [Performance](/docs/advanced/performance/) – Optimize your Teleportal application

# Performance

> Strategies for optimizing Teleportal throughput, latency, and resource usage

This guide covers performance optimization strategies for Teleportal applications, from document size management and write buffering to connection sharing and monitoring.

## Document Size Management

[Section titled “Document Size Management”](#document-size-management)

Y.js documents grow over time as edits accumulate. Teleportal tracks document size in `DocumentMetadata.sizeBytes` and provides two server-level thresholds you can set to stay ahead of problems.

### Server-Level Configuration

[Section titled “Server-Level Configuration”](#server-level-configuration)

Set default thresholds for all documents via `documentSizeConfig`:

```typescript
const server = new Server({
  documentSizeConfig: {
    warningThreshold: 5 * 1024 * 1024, // 5 MB -- emit a warning event
    limit: 20 * 1024 * 1024, // 20 MB -- emit a limit-exceeded event
  },
});
```

### Per-Document Overrides

[Section titled “Per-Document Overrides”](#per-document-overrides)

Individual documents can override the server defaults through their `DocumentMetadata`:

```typescript
// In your storage implementation or metadata writes
await storage.writeDocumentMetadata(documentId, {
  encrypted: false,
  sizeWarningThreshold: 2 * 1024 * 1024, // 2 MB for this specific document
  sizeLimit: 10 * 1024 * 1024, // 10 MB hard limit
});
```

Per-document values take precedence over the server-level defaults.

### Events

[Section titled “Events”](#events)

When a threshold is crossed, the server emits events you can listen to:

```typescript
server.on("document-size-warning", ({ documentId, sizeBytes, warningThreshold }) => {
  console.warn(`Document ${documentId} is ${sizeBytes} bytes (threshold: ${warningThreshold})`);
});


server.on("document-size-limit-exceeded", ({ documentId, sizeBytes, sizeLimit }) => {
  console.error(`Document ${documentId} exceeded limit: ${sizeBytes} > ${sizeLimit}`);
  // Take action: reject further writes, archive the document, etc.
});
```

Both events are advisory – writes are not blocked automatically. Events are deduplicated: the server emits each event once per crossing. If the document shrinks back below the threshold (for example, after compaction) and then exceeds it again, the event fires again.

### Monitoring Metrics

[Section titled “Monitoring Metrics”](#monitoring-metrics)

Three Prometheus-compatible metrics track document size:

| Metric                                          | Type    | Labels                    | Description                             |
| ----------------------------------------------- | ------- | ------------------------- | --------------------------------------- |
| `teleportal_document_size_bytes`                | Gauge   | `documentId`, `encrypted` | Current size of the document in bytes   |
| `teleportal_document_size_warning_total`        | Counter | `documentId`              | Number of warning events emitted        |
| `teleportal_document_size_limit_exceeded_total` | Counter | `documentId`              | Number of limit-exceeded events emitted |

### Best Practices

[Section titled “Best Practices”](#best-practices)

* **Set warning thresholds proactively.** A warning at 5 MB gives you time to investigate before a document becomes unwieldy.
* **Monitor document growth.** Use the `teleportal_document_size_bytes` gauge to track trends and set alerts.
* **Implement cleanup strategies.** For documents that grow large, consider periodic compaction, archiving old content, or splitting into subdocuments.

## VirtualStorage for Write Buffering

[Section titled “VirtualStorage for Write Buffering”](#virtualstorage-for-write-buffering)

`VirtualStorage` wraps any `DocumentStorage` implementation, buffering writes in memory and flushing them in batches to reduce database round-trips.

```typescript
import { VirtualStorage } from "teleportal/storage";


const storage = new VirtualStorage(underlyingStorage, {
  batchMaxSize: 100, // Flush after 100 buffered updates (default: 100)
  batchWaitMs: 2000, // Or flush after 2 seconds of inactivity (default: 2000)
});
```

### How Batching Works

[Section titled “How Batching Works”](#how-batching-works)

Updates are accumulated in an in-memory buffer, keyed by document ID. A flush happens when **either** condition is met:

1. The buffer reaches `batchMaxSize` updates.
2. `batchWaitMs` milliseconds have elapsed since the first buffered update.

On flush, all buffered updates for a document are written sequentially to the underlying storage in a single batch operation.

### Read Consistency

[Section titled “Read Consistency”](#read-consistency)

Reads always see the latest state. Before any read operation (`getDocument`, `getDocumentMetadata`, `handleSyncStep1`), VirtualStorage flushes all pending writes for that document to the underlying storage. This guarantees consistency but means reads have slight additional latency when there are pending writes.

### Performance Impact

[Section titled “Performance Impact”](#performance-impact)

* **Writes**: faster acknowledgment because they return immediately after buffering. Fewer database calls overall.
* **Reads**: consistent, but may trigger a flush if writes are pending. In practice this is fast because flushes are sequential writes to the same document.
* **Memory**: proportional to `batchMaxSize` multiplied by the number of active documents with pending writes.

### When to Use

[Section titled “When to Use”](#when-to-use)

* High-frequency collaborative updates (many users typing simultaneously).
* Slow storage backends (remote databases, object storage over the network).
* Write-heavy applications where individual update durability is less critical than throughput.

### When Not to Use

[Section titled “When Not to Use”](#when-not-to-use)

* If you need real-time durability guarantees (every update must be persisted before acknowledgment).
* Single-write patterns where batching adds complexity without measurable benefit.

## Y.js Update Encoding

[Section titled “Y.js Update Encoding”](#yjs-update-encoding)

Teleportal supports both V1 and V2 Y.js update encodings. Each has different performance characteristics:

* **V1 encoding**: lower per-update overhead, better for small incremental edits. Individual keystrokes and small changes are more compact in V1.
* **V2 encoding**: more compact encoding for large state, better for full-document snapshots and merged changesets.

### How Teleportal Uses Both

[Section titled “How Teleportal Uses Both”](#how-teleportal-uses-both)

Teleportal handles encoding automatically – you do not need to choose. The system uses each format where it performs best:

* **Wire protocol**: individual updates are sent as `VersionedUpdate` objects that carry a `version` field (1 or 2). Small per-message updates typically use V1 for lower overhead.
* **Storage**: merged document state is always stored in V2 format. When updates of mixed versions are merged, they are converted to V2 before the merge (`mergeUpdatesV2`).

```typescript
// The VersionedUpdate type reflects this dual encoding
type VersionedUpdate = { version: 1; data: UpdateV1 } | { version: 2; data: UpdateV2 };
```

### Why This Matters

[Section titled “Why This Matters”](#why-this-matters)

Understanding the encoding split helps diagnose size issues:

* If **wire traffic** seems high, the issue is likely update frequency rather than encoding. Consider message batching (below).
* If **stored document size** seems high, the V2-merged state may contain redundant history. Consider compaction or milestone-based cleanup.

## Message Batching

[Section titled “Message Batching”](#message-batching)

Teleportal batches messages at multiple layers to reduce network overhead.

### HTTP Transport Batching

[Section titled “HTTP Transport Batching”](#http-transport-batching)

The HTTP transport batches outgoing messages into a single HTTP POST containing a `MessageArray`:

```typescript
import { httpTransport } from "teleportal/providers";


const transport = httpTransport({
  httpBatchingOptions: {
    maxBatchSize: 10, // Batch up to 10 messages (default: 10)
    maxBatchDelay: 100, // Or send after 100ms (default: 100)
  },
});
```

Messages are collected until either `maxBatchSize` messages have accumulated or `maxBatchDelay` milliseconds have elapsed. The batch is then encoded into a single `MessageArray` – a binary blob where each message is prefixed with a varint-encoded length – and sent as one HTTP POST.

### Trade-offs

[Section titled “Trade-offs”](#trade-offs)

* **Larger batches** reduce network overhead (fewer HTTP requests, less header duplication) but increase tail latency because messages wait for the batch to fill or time out.
* **Smaller batches** are more responsive but generate more network traffic.
* For real-time typing, the defaults (10 messages / 100ms) strike a reasonable balance. For bulk operations like import or migration, consider increasing both values.

### Encoding Format

[Section titled “Encoding Format”](#encoding-format)

`MessageArray` uses lib0’s varint-prefixed encoding. Each message in the array is written as a `varUint8Array`: the byte length is encoded as a variable-length integer, followed by the raw message bytes. This avoids fixed-size length headers and keeps small messages compact.

## SharedWorker Connection

[Section titled “SharedWorker Connection”](#sharedworker-connection)

The SharedWorker provider offloads the WebSocket connection to a SharedWorker thread, so all tabs for the same origin share a single connection to the server.

```typescript
import { WorkerProvider } from "teleportal/providers/worker";
import { createEncryptionKey } from "teleportal/encryption-key";


const provider = await WorkerProvider.create({
  workerUrl: new URL("./worker.ts", import.meta.url),
  url: "wss://example.com/sync",
  document: "my-document",
  encryptionKey: createEncryptionKey(),
});
```

### Core Benefits

[Section titled “Core Benefits”](#core-benefits)

* **Connection sharing**: N open tabs produce 1 WebSocket connection instead of N. This directly reduces server load and eliminates duplicate sync messages.
* **Grace period**: when the last tab using a connection closes, the worker waits 5 seconds (default, configurable via `gracePeriodMs`) before tearing down the connection. Page reloads within this window reconnect instantly without a server round-trip.
* **File operations in worker thread**: file uploads and downloads run inside the worker, keeping the main thread free for rendering.
* **Online/offline reconciliation**: the worker uses an any-tab-online policy – if any connected tab reports itself as online, the connection stays online. A single backgrounded or throttled tab cannot take the connection offline.

### Connection Pooling

[Section titled “Connection Pooling”](#connection-pooling)

Connections are pooled by a key derived from URL and token by default:

```typescript
// Default key: `${url}::${token}`
// Two tabs with the same URL and token share one connection.
// Different tokens get separate connections (identity isolation).
```

You can customize the key function through `ConnectionWorkerManagerOptions.getConnectionKey` if you need different sharing behavior (for example, sharing across token refreshes for the same user).

### Configuration

[Section titled “Configuration”](#configuration)

```typescript
// Worker-side configuration (in your worker entry point)
const manager = new ConnectionWorkerManager(transportFactory, {
  gracePeriodMs: 10_000, // 10-second grace period (default: 5000)
  getConnectionKey: (options) => options.url ?? "default", // Custom pooling key
});
```

See [SharedWorker Guide](/docs/guides/shared-worker/) for full setup details including the worker entry point.

## Connection Optimization

[Section titled “Connection Optimization”](#connection-optimization)

### Connection Reuse

[Section titled “Connection Reuse”](#connection-reuse)

Multiple providers can share the same connection to work with different documents over a single WebSocket:

```typescript
const connection = new DirectConnection({
  url: "wss://example.com",
  transports: [websocketTransport()],
});


const key = createEncryptionKey();
const provider1 = new Provider({ connection, document: "doc-1", encryptionKey: key });
const provider2 = new Provider({ connection, document: "doc-2", encryptionKey: key });
```

This reduces the number of open connections and avoids duplicate handshakes.

### Document Switching

[Section titled “Document Switching”](#document-switching)

Use `switchDocument()` to transition to a new document without creating a new connection:

```typescript
const newProvider = provider.switchDocument({ document: "new-doc" });
```

The existing connection is reused. The old document is unsubscribed and the new one is subscribed in a single operation, which is faster than destroying and recreating a provider.

### Multi-Document Access

[Section titled “Multi-Document Access”](#multi-document-access)

Use `openDocument()` to open additional documents on the same connection without closing the current one:

```typescript
const secondProvider = provider.openDocument({ document: "second-doc" });
```

## Storage Performance

[Section titled “Storage Performance”](#storage-performance)

Choose storage backends based on your workload characteristics. Teleportal supports different backends for different storage types (documents, files, milestones), so you can mix and match.

### In-Memory

[Section titled “In-Memory”](#in-memory)

* **Latency**: lowest (no I/O).
* **Durability**: none – data is lost on restart.
* **Best for**: development, testing, ephemeral documents, caches.

### Redis

[Section titled “Redis”](#redis)

* **Latency**: low (in-memory with optional persistence).
* **Durability**: configurable (RDB snapshots, AOF logs).
* **Best for**: high-frequency updates, rate limit state storage.
* **Watch for**: memory limits. Large documents or many concurrent documents can exhaust Redis memory. Monitor `used_memory` and set `maxmemory` policies.

### PostgreSQL / SQL

[Section titled “PostgreSQL / SQL”](#postgresql--sql)

* **Latency**: moderate (disk I/O, network round-trip for remote instances).
* **Durability**: strong (ACID transactions, WAL).
* **Best for**: production document storage where durability matters. Complex queries (listing documents, filtering by metadata).
* **Optimization**: wrap with `VirtualStorage` for high-frequency collaborative workloads to reduce write amplification.

### S3 / Object Storage

[Section titled “S3 / Object Storage”](#s3--object-storage)

* **Latency**: high (HTTP round-trips, eventual consistency for some operations).
* **Durability**: very high (cross-region replication).
* **Best for**: `FileStorage` (large binary assets, images, PDFs). Not recommended for `DocumentStorage` due to latency.

### Mixing Backends

[Section titled “Mixing Backends”](#mixing-backends)

Use different backends for different storage types:

```typescript
import { getFileRpcHandlers } from "teleportal/protocols/file";
import { getMilestoneRpcHandlers } from "teleportal/protocols/milestone";


const server = new Server({
  storage: postgresDocumentStorage, // Durable document storage
  rpcHandlers: {
    ...getFileRpcHandlers(s3FileStorage), // Scalable file storage
    ...getMilestoneRpcHandlers(redisMilestoneStorage), // Fast milestone lookups
  },
});
```

## Monitoring for Performance

[Section titled “Monitoring for Performance”](#monitoring-for-performance)

Track these key metrics to identify bottlenecks before they affect users.

### Message Processing

[Section titled “Message Processing”](#message-processing)

| Metric                                | What to Watch                                                                                                                    |
| ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------- |
| `teleportal_message_duration_seconds` | If p95 exceeds 100ms, investigate message processing. Check storage latency or expensive middleware. Labeled by message `type`.  |
| `teleportal_messages_total`           | Labeled by `type`. Understand your message profile – a high ratio of sync messages may indicate clients reconnecting frequently. |

### Storage

[Section titled “Storage”](#storage)

| Metric                                          | What to Watch                                                                                                           |
| ----------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- |
| `teleportal_storage_operation_duration_seconds` | If consistently high, consider wrapping with `VirtualStorage` or switching to a faster backend. Labeled by `operation`. |
| `teleportal_storage_operations_total`           | Labeled by `operation` and `result`. A high error rate indicates backend health issues.                                 |

### Connections and Load

[Section titled “Connections and Load”](#connections-and-load)

| Metric                       | What to Watch                                                                                             |
| ---------------------------- | --------------------------------------------------------------------------------------------------------- |
| `teleportal_sessions_active` | Number of active document sessions. Correlate with resource usage to plan capacity.                       |
| `teleportal_clients_active`  | Number of connected clients. Compare with `sessions_active` to understand fan-out (clients per document). |

### Document Health

[Section titled “Document Health”](#document-health)

| Metric                                   | What to Watch                                                                                                    |
| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------- |
| `teleportal_document_size_bytes`         | Watch for unbounded growth. Set alerts based on your warning threshold. Labeled by `documentId` and `encrypted`. |
| `teleportal_document_size_warning_total` | Non-zero means documents are approaching your configured thresholds.                                             |

See [Observability Guide](/docs/guides/observability/) for setup instructions, Prometheus scraping configuration, and dashboard examples.

## Rate Limiting

[Section titled “Rate Limiting”](#rate-limiting)

Rate limiting prevents abuse and ensures fair resource usage. Teleportal supports rule-based rate limiting with multiple tracking dimensions:

```typescript
const server = new Server({
  rateLimitConfig: {
    rules: [
      {
        id: "per-user",
        maxMessages: 100,
        windowMs: 1000,
        trackBy: "user",
      },
      {
        id: "per-document",
        maxMessages: 500,
        windowMs: 1000,
        trackBy: "document",
      },
    ],
    getUserId: (message) => message.context.userId,
    getDocumentId: (message) => message.context.documentId,
  },
});
```

Rules can track by `"user"`, `"document"`, `"user-document"`, or `"transport"`. All rules must pass for a message to be allowed. You can also set a `maxMessageSize` (default: 10 MB) to reject oversized payloads.

See [Rate Limiting Guide](/docs/guides/rate-limiting/) for details on dynamic limits, storage backends, skip conditions, and event handling.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Scaling](/docs/advanced/scaling/) – horizontal scaling, PubSub backends, and multi-node deployment
* [Observability](/docs/guides/observability/) – Prometheus metrics, health endpoints, and dashboard setup
* [SharedWorker Guide](/docs/guides/shared-worker/) – full setup walkthrough for the SharedWorker provider

# Protocol Specification

> Detailed specification of the Teleportal protocol

This document provides a comprehensive specification of the Teleportal protocol, a binary messaging protocol built on top of Y.js for real-time collaborative document synchronization and awareness updates. It describes how all the pieces fit together to enable efficient, type-safe communication for collaborative editing applications.

## Protocol Overview

[Section titled “Protocol Overview”](#protocol-overview)

The Teleportal protocol is designed for efficient transmission of Y.js collaborative editing messages over various transport layers. It defines five wire message types — document sync, awareness, ack, presence, and RPC — all with optional encryption and robust error handling. Higher-level features (file transfers, milestones, attribution, key distribution) are all built **on top of the RPC message type**, not as dedicated wire types.

The protocol is built around a flexible message structure that enables:

* **Document Synchronization**: Bidirectional sync of Y.js documents between clients and server
* **Awareness Updates**: Real-time user cursor/selection information
* **Presence**: Client join/leave/heartbeat tracking
* **RPC Operations**: Extensible custom operations (file transfer, milestones, attribution, key registry, and your own methods)
* **Message Acknowledgment**: Delivery confirmation for reliable message handling

## Message Format

[Section titled “Message Format”](#message-format)

### Base Message Structure

[Section titled “Base Message Structure”](#base-message-structure)

All Teleportal messages follow this base structure:

```
graph LR
    A[Message Header] --> B[Message Type]
    B --> C[Payload]

    subgraph Header["Message Header"]
        H1["Magic: YJS<br/>(3 bytes)"]
        H2["Version: 0x01<br/>(1 byte)"]
        H3["Doc Name Length<br/>(varint)"]
        H4["Doc Name<br/>(UTF-8 string)"]
        H5["Encrypted Flag<br/>(1 byte)"]
        H1 --> H2 --> H3 --> H4 --> H5
    end

    subgraph Type["Message Type (1 byte)"]
        T1["0x00: Document"]
        T2["0x01: Awareness"]
        T3["0x02: ACK"]
        T4["0x03: Presence"]
        T5["0x04: RPC"]
    end

    Header --> Type
    Type --> C
```

| Field           | Size    | Value                     |
| --------------- | ------- | ------------------------- |
| Magic Number    | 3 bytes | `0x59 0x4A 0x53` (“YJS”)  |
| Version         | 1 byte  | `0x01`                    |
| Doc Name Length | varint  | length                    |
| Doc Name        | string  | UTF-8 string              |
| Encrypted Flag  | 1 byte  | `0x00`=false, `0x01`=true |
| Message Type    | 1 byte  | `0x00`–`0x04`             |

**Note**: The document name is an empty string for ack messages (they are not tied to a specific document).

### Message Type Hierarchy

[Section titled “Message Type Hierarchy”](#message-type-hierarchy)

The protocol organizes messages into categories, each with specific subtypes:

```
graph TD
    A[Teleportal Message] --> B[Document 0x00]
    A --> C[Awareness 0x01]
    A --> D[ACK 0x02]
    A --> P[Presence 0x03]
    A --> F[RPC 0x04]

    B --> B1[Sync Step 1]
    B --> B2[Sync Step 2]
    B --> B3[Update]
    B --> B4[Sync Done]
    B --> B5[Auth]

    C --> C1[Awareness Update]
    C --> C2[Awareness Request]

    D --> D1[ACK / NACK]

    P --> P1[Presence Announce/Unannounce]
    P --> P2[Presence Join/Leave]
    P --> P3[Presence Heartbeat]

    F --> F1[RPC Request]
    F --> F2[RPC Response]
    F --> F3[RPC Stream]
```

**Note**: Milestones, file transfers, attribution, and key distribution are **not** dedicated wire message types. They are all RPC methods carried by the RPC message (type `0x04`) — e.g. `milestoneList`/`milestoneCreate`, `fileUpload`/`fileDownload`, `attributionActivity`/`attributionGet`. Only the five wire types above (`0x00`–`0x04`) exist at the protocol level.

## Document Messages (Type 0x00)

[Section titled “Document Messages (Type 0x00)”](#document-messages-type-0x00)

Document messages handle Y.js document synchronization and updates. They form the core of the collaborative editing system.

### Document Message Structure

[Section titled “Document Message Structure”](#document-message-structure)

| Sub-type     | Code   | Payload                                                   |
| ------------ | ------ | --------------------------------------------------------- |
| Sync Step 1  | `0x00` | State Vector (`varUint8Array`)                            |
| Sync Step 2  | `0x01` | Version byte (1=V1, 2=V2) + Y.js Update (`varUint8Array`) |
| Doc Update   | `0x02` | Version byte (1=V1, 2=V2) + Y.js Update (`varUint8Array`) |
| Sync Done    | `0x03` | (no payload)                                              |
| Auth Message | `0x04` | Permission (1 byte) + Reason (`varString`)                |

Updates (sync-step-2 and update) carry a leading version byte (`1` = V1, `2` = V2) so the receiver knows how to apply them.

### Document Synchronization Messages

[Section titled “Document Synchronization Messages”](#document-synchronization-messages)

The document synchronization process uses a bidirectional sync protocol to ensure all clients have consistent document state:

```
graph TD
    A[Client State Vector] --> B[Sync Step 1]
    B --> C[Server compares state vectors]
    C --> D[Server sends missing updates]
    D --> E[Sync Step 2]
    E --> F[Client applies updates]
    F --> G[Server sends its state vector]
    G --> H[Client sends missing updates]
    H --> I[Sync Step 2 from client]
    I --> J[Sync Done]
    J --> K[Real-time Updates]
    K --> L[Document Update messages]

    style B fill:#e1f5ff
    style E fill:#e1f5ff
    style J fill:#c8e6c9
    style L fill:#fff9c4
```

#### Sync Step 1 (0x00)

[Section titled “Sync Step 1 (0x00)”](#sync-step-1-0x00)

**Purpose**: Initiates synchronization by sending local state vector\
**Payload**: Y.js state vector as variable-length byte array\
**Usage**: Client sends this to request updates from server. The state vector represents what the client knows about the document’s current state.

#### Sync Step 2 (0x01)

[Section titled “Sync Step 2 (0x01)”](#sync-step-2-0x01)

**Purpose**: Responds to Sync Step 1 with missing updates\
**Payload**: Y.js update containing missing operations\
**Usage**: Server responds with updates not present in client’s state. This enables efficient synchronization by only sending what’s needed.

#### Document Update (0x02)

[Section titled “Document Update (0x02)”](#document-update-0x02)

**Purpose**: Sends incremental document changes\
**Payload**: Y.js update containing new operations\
**Usage**: Real-time propagation of document changes after initial sync is complete.

#### Sync Done (0x03)

[Section titled “Sync Done (0x03)”](#sync-done-0x03)

**Purpose**: Indicates synchronization completion\
**Payload**: None\
**Usage**: Signals that both sync steps have been completed and the client is now in sync with the server.

#### Auth Message (0x04)

[Section titled “Auth Message (0x04)”](#auth-message-0x04)

**Purpose**: Handles authentication and authorization\
**Payload**: Permission flag (1 byte) + reason string (`varString`)\
**Usage**: Server sends to grant/deny access with explanation. Used when a client attempts to access a document they don’t have permission for.

## ACK Messages (Type 0x02)

[Section titled “ACK Messages (Type 0x02)”](#ack-messages-type-0x02)

ACK messages provide message delivery confirmation (and negative acknowledgement / NACK), allowing senders to know when their messages have been received and processed — or that they should retry later.

### ACK Message Structure

[Section titled “ACK Message Structure”](#ack-message-structure)

The ACK payload is:

```plaintext
[varString: messageId]
[uint8: flags]               bit 0 = has retryAfter, bit 1 = has error
[varUint: retryAfter]?       present if bit 0 set (ms to wait before retrying)
[varString: error]?          present if bit 1 set
```

**Purpose**: Acknowledges (or negatively acknowledges) receipt of a specific message\
**Payload**: The `messageId` of the message being acknowledged, encoded as a `varString`, followed by a flags byte and the optional `retryAfter` / `error` fields\
**Usage**:

* Confirms delivery of file chunks during uploads (a NACK with `retryAfter` sheds load and drives client retransmission)
* Allows senders to track which messages have been received
* The `messageId` is the message’s content fingerprint — a 64-bit FNV-1a hash of the encoded bytes rendered as 16 lowercase hex characters (see [Message IDs](#message-ids))

**Note**: ACK messages do not have a document name and are not tied to a specific document context.

## Awareness Messages (Type 0x01)

[Section titled “Awareness Messages (Type 0x01)”](#awareness-messages-type-0x01)

Awareness messages handle user presence and cursor information in collaborative sessions, enabling real-time collaboration features like showing where other users are editing.

### Awareness Message Structure

[Section titled “Awareness Message Structure”](#awareness-message-structure)

| Msg Type          | Code   | Payload                              |
| ----------------- | ------ | ------------------------------------ |
| Awareness Update  | `0x00` | Y.js Awareness Update (varint array) |
| Awareness Request | `0x01` | (no payload)                         |

### Awareness Message Types

[Section titled “Awareness Message Types”](#awareness-message-types)

#### Awareness Update (0x00)

[Section titled “Awareness Update (0x00)”](#awareness-update-0x00)

**Purpose**: Sends user presence and cursor information\
**Payload**: Y.js awareness update as variable-length byte array\
**Usage**: Propagates user activity, cursor position, selection state, and other presence information to all connected clients.

#### Awareness Request (0x01)

[Section titled “Awareness Request (0x01)”](#awareness-request-0x01)

**Purpose**: Requests current awareness state\
**Payload**: None\
**Usage**: Client requests current user presence information when joining a document or when awareness state is needed.

## Presence Messages (Type 0x03)

[Section titled “Presence Messages (Type 0x03)”](#presence-messages-type-0x03)

Presence messages track which clients are in a document (join/leave/heartbeat), distinct from Y.js awareness (cursor/selection state). Presence is a **dedicated wire type** (`0x03`).

| Sub-type            | Code   | Payload                                                                                                            |
| ------------------- | ------ | ------------------------------------------------------------------------------------------------------------------ |
| Presence Announce   | `0x00` | `[varUint: awarenessId]`                                                                                           |
| Presence Join       | `0x01` | `[varUint: awarenessId] [varString: clientId] [varString: userId] [any: data]`                                     |
| Presence Leave      | `0x02` | (same shape as join)                                                                                               |
| Presence Heartbeat  | `0x03` | `[varUint: count]` then per client: `[varUint: awarenessId] [varString: clientId] [varString: userId] [any: data]` |
| Presence Unannounce | `0x04` | `[varUint: awarenessId]`                                                                                           |

## File Transfer (RPC methods)

[Section titled “File Transfer (RPC methods)”](#file-transfer-rpc-methods)

File transfer is **not** a dedicated wire message type. It is implemented entirely as RPC (type `0x04`) using two methods — `fileUpload` and `fileDownload` — with streamed chunks. `fileUpload` uses the RPC “multipart” method kind (an initiation request plus a chunk stream); `fileDownload` is a request-response whose response streams the file parts back. Files are chunked and verified with a Merkle tree so transfers are content-addressed, resumable, and deduplicated.

### RPC method shapes

[Section titled “RPC method shapes”](#rpc-method-shapes)

The request/response/stream payloads (encoded inside the RPC envelope, see [RPC Messages](#rpc-messages-type-0x04)) are:

* **`fileUpload` request** — `{ fileId, filename, size, mimeType, lastModified, encrypted, chunkSize? }`. Here `fileId` is the **content-addressed Merkle root** (the `contentId`), which the client computes by encrypting the whole file and folding its Merkle tree *before* sending the request. This makes the id stable across retries — the basis for resume and dedup.
* **`fileUpload` response** — `{ fileId, allowed, reason?, statusCode?, chunkSize?, existingChunks?, alreadyExists?, chunkSizeMismatch? }`. `alreadyExists: true` means the content already exists durably (dedup hit — the client streams nothing); `existingChunks` lists chunks the server already has (resume). `chunkSizeMismatch` asks the client to re-chunk at the server’s size.
* **`fileUpload` stream** (per chunk) — `{ fileId, chunkIndex, chunkData, merkleProof, totalChunks, bytesUploaded, encrypted }`. On **upload** the stream omits per-chunk proofs (`merkleProof: []`) — the server recomputes the tree from stored chunks and verifies it equals the client-claimed `contentId`. On **download** the server populates `merkleProof` and the client verifies each chunk.
* **`fileDownload` request** — `{ fileId }` (the Merkle root / `contentId`).
* **`fileDownload` response** — `{ fileId, filename, size, mimeType, lastModified, encrypted, allowed, reason?, statusCode?, totalChunks? }`, followed by a stream of file parts.

Errors (permission denied, not found, size limit) surface through the normal RPC error channel (status code + details), not through a dedicated wire message.

### File Chunking and Merkle Trees

[Section titled “File Chunking and Merkle Trees”](#file-chunking-and-merkle-trees)

Files are split into **1MB chunks** (configurable) for efficient transfer. Each chunk is hashed using SHA-256, and a Merkle tree is constructed to verify file integrity.

#### Merkle Tree Structure

[Section titled “Merkle Tree Structure”](#merkle-tree-structure)

```
graph TD
    Root["Root Hash<br/>(ContentId/FileId)"] --> H1["Hash 1"]
    Root --> H2["Hash 2"]

    H1 --> H3["Hash 3"]
    H1 --> H4["Hash 4"]
    H2 --> H5["Hash 5"]
    H2 --> H6["Hash 6"]

    H3 --> C0["Chunk 0<br/>SHA-256"]
    H3 --> C1["Chunk 1<br/>SHA-256"]
    H4 --> C2["Chunk 2<br/>SHA-256"]
    H4 --> C3["Chunk 3<br/>SHA-256"]
    H5 --> C4["Chunk 4<br/>SHA-256"]
    H5 --> C5["Chunk 5<br/>SHA-256"]
    H6 --> C6["Chunk 6<br/>SHA-256"]
    H6 --> C7["Chunk 7<br/>SHA-256"]

    style Root fill:#ffccbc
    style C0 fill:#c8e6c9
    style C1 fill:#c8e6c9
    style C2 fill:#c8e6c9
    style C3 fill:#c8e6c9
    style C4 fill:#c8e6c9
    style C5 fill:#c8e6c9
    style C6 fill:#c8e6c9
    style C7 fill:#c8e6c9
```

**Structure Components**:

* **Leaf nodes**: SHA-256 hash of each chunk
* **Internal nodes**: Hash of concatenated child hashes
* **Root hash**: Content ID used to uniquely identify the file
* **Merkle proof**: Path from chunk hash to root (sibling hashes at each level)

#### Merkle Proof Example

[Section titled “Merkle Proof Example”](#merkle-proof-example)

When sending Chunk 2, the client includes a Merkle proof that allows the server to verify the chunk:

```
graph LR
    C2["Chunk 2<br/>Data"] --> H2["Hash Chunk 2"]
    H2 --> P1["Proof: Hash 3<br/>(sibling)"]
    P1 --> P2["Proof: Hash 2<br/>(sibling)"]
    P2 --> Verify["Verify Root<br/>matches FileId"]

    style C2 fill:#c8e6c9
    style Verify fill:#ffccbc
```

#### Chunk Verification

[Section titled “Chunk Verification”](#chunk-verification)

The verification process ensures data integrity by reconstructing the Merkle tree path:

```
sequenceDiagram
    participant C as Client
    participant S as Server

    Note over C: Prepare Chunk
    C->>C: Hash chunk data (SHA-256)
    C->>C: Build Merkle proof path

    Note over C,S: Send Chunk
    C->>S: fileUpload stream chunk<br/>(download: chunk + Merkle proof)

    Note over S: Verify (download path)
    S->>S: Hash received chunk data
    S->>S: Reconstruct path using proof
    S->>S: Compute root hash
    S->>S: Compare with expected fileId

    alt Verification Success
        S->>S: Store chunk
        S->>C: ACK Message
    else Verification Fails
        S->>C: RPC error / NACK
    end
```

> **Note**: On **upload** the client omits per-chunk proofs (`merkleProof: []`) and the server recomputes the whole tree at completion, verifying it equals the client-claimed `contentId`. On **download** the server generates the proofs and the client verifies each chunk against the root. Download is the integrity boundary.

**Verification Steps (download)**:

1. **Server sends**: Chunk data + Merkle proof (sibling hashes) + Chunk index
2. **Client receives**: Hashes the chunk data to get leaf hash
3. **Client reconstructs**: Uses proof hashes to build path from leaf to root
4. **Client verifies**: Compares computed root hash with expected fileId
5. **Client rejects** the chunk if verification fails

## RPC Messages (Type 0x04)

[Section titled “RPC Messages (Type 0x04)”](#rpc-messages-type-0x04)

RPC messages provide extensible custom operations for application-specific needs. They enable the protocol to be extended without modifying the core message types. Built-in features (milestones, file transfer, attribution, key registry) are all RPC methods.

### RPC Message Structure

[Section titled “RPC Message Structure”](#rpc-message-structure)

The RPC payload is:

```plaintext
[varString: method]
[uint8: requestType]              0=request, 1=stream, 2=response
[varString: originalRequestId]?   present for stream/response only
[uint8: isError]                  0=success, 1=error
```

Success payload: `[varUint8Array: serialized payload]`.\
Error payload: `[varUint: statusCode] [varString: details] [uint8: hasPayload] [any: payload]?`.

### Request Types

[Section titled “Request Types”](#request-types)

#### RPC Request (requestType 0)

[Section titled “RPC Request (requestType 0)”](#rpc-request-requesttype-0)

**Purpose**: Client requests a custom operation\
**Payload**: Method name + serialized request data\
**Usage**: Enables custom operations beyond the standard protocol messages.

#### RPC Stream (requestType 1)

[Section titled “RPC Stream (requestType 1)”](#rpc-stream-requesttype-1)

**Purpose**: Streaming data for an in-flight RPC operation\
**Payload**: Method name + `originalRequestId` + serialized stream chunk\
**Usage**: Carries streamed chunks (e.g. file parts) tied back to the originating request.

#### RPC Response (requestType 2)

[Section titled “RPC Response (requestType 2)”](#rpc-response-requesttype-2)

**Purpose**: Server responds to an RPC request\
**Payload**: Method name + `originalRequestId` + success or error payload\
**Usage**: Returns the result of an RPC operation.

A custom serializer/deserializer can override the default `writeAny`/`readAny` encoding for RPC payloads.

## Special Message Types

[Section titled “Special Message Types”](#special-message-types)

### Ping/Pong Messages

[Section titled “Ping/Pong Messages”](#pingpong-messages)

Keep-alive messages for connection health monitoring:

**Ping Message:**

| Field        | Size    | Value                          |
| ------------ | ------- | ------------------------------ |
| Magic Number | 3 bytes | `0x59 0x4A 0x53` (“YJS”)       |
| Ping         | 4 bytes | `0x70 0x69 0x6E 0x67` (“ping”) |

**Pong Message:**

| Field        | Size    | Value                          |
| ------------ | ------- | ------------------------------ |
| Magic Number | 3 bytes | `0x59 0x4A 0x53` (“YJS”)       |
| Pong         | 4 bytes | `0x70 0x6F 0x6E 0x67` (“pong”) |

### Message Arrays

[Section titled “Message Arrays”](#message-arrays)

Multiple messages can be batched into a single transmission for efficiency. Messages are concatenated sequentially without an explicit count field:

```
graph LR
    A[Message Array] --> B["Message 1<br/>Length varint"]
    B --> C["Message 1<br/>Binary Data"]
    C --> D["Message 2<br/>Length varint"]
    D --> E["Message 2<br/>Binary Data"]
    E --> F["..."]
    F --> G["Message N<br/>Length varint"]
    G --> H["Message N<br/>Binary Data"]

    style A fill:#e1f5ff
    style C fill:#c8e6c9
    style E fill:#c8e6c9
    style H fill:#c8e6c9
```

| Component        | Encoding                                        |
| ---------------- | ----------------------------------------------- |
| Message 1 Length | varint                                          |
| Message 1 Data   | BinaryMessage                                   |
| Message 2 Length | varint                                          |
| Message 2 Data   | BinaryMessage                                   |
| …                | (repeated for all messages until end of buffer) |

**Encoding**: Each message in the array is encoded as a varint-prefixed byte array. The decoder reads messages sequentially until the buffer is exhausted.

**Usage**: Useful for reducing network overhead when sending multiple related messages (e.g., multiple document updates or file chunks).

## Encoding Details

[Section titled “Encoding Details”](#encoding-details)

### Message Encoding Flow

[Section titled “Message Encoding Flow”](#message-encoding-flow)

The encoding process transforms structured message data into binary format:

```
graph LR
    A[Message Object] --> B[Encode Header]
    B --> C[Encode Type]
    C --> D[Encode Payload]
    D --> E[Binary Message]
    E --> F[FNV-1a hash<br/>of encoded bytes]
    F --> G[16-hex-char Message ID]

    H[Binary Message] --> I[Decode Header]
    I --> J[Decode Type]
    J --> K[Decode Payload]
    K --> L[Message Object]

    style E fill:#c8e6c9
    style G fill:#ffccbc
    style H fill:#e1f5ff
```

### Message IDs

[Section titled “Message IDs”](#message-ids)

Every message has a unique identifier computed from its encoded bytes:

```
graph TD
    A[Message Object] --> B[Encode to Binary]
    B --> C[64-bit FNV-1a hash]
    C --> D[Render as 16 hex chars]
    D --> E[Message ID]
    E --> F[Used in ACK Messages]
    E --> G[Message Deduplication]
    E --> H[Idempotency Tracking]

    style C fill:#ffccbc
    style E fill:#c8e6c9
```

* **Computation**: a fast 64-bit **FNV-1a-style hash** of the message’s encoded bytes — a content fingerprint, **not** SHA-256 and **not** base64
* **Encoding**: rendered as 16 lowercase hex characters; carried as a `varString` in ACK messages
* **Purpose**: Enables message deduplication, acknowledgment tracking, and idempotency
* **Lazy Computation**: Message IDs are computed on first access and cached; `valueOf()` returns the id

### Variable-Length Encoding

[Section titled “Variable-Length Encoding”](#variable-length-encoding)

The protocol uses variable-length encoding for efficiency:

```
graph TD
    A[Variable-Length Encoding] --> B[Varint Integers]
    A --> C[Varint Arrays]
    A --> D[UTF-8 Strings]

    B --> B1["Small values: 1 byte<br/>Large values: multiple bytes"]
    C --> C1["Length varint<br/>+ Raw bytes"]
    D --> D1["Length varint<br/>+ UTF-8 bytes"]

    style B fill:#e1f5ff
    style C fill:#e1f5ff
    style D fill:#e1f5ff
```

#### Variable-Length Integers (varint)

[Section titled “Variable-Length Integers (varint)”](#variable-length-integers-varint)

* Used for lengths and counts
* Follows lib0 encoding standard
* Efficient for small values, expandable for large ones

#### Variable-Length Byte Arrays (varint array)

[Section titled “Variable-Length Byte Arrays (varint array)”](#variable-length-byte-arrays-varint-array)

* Length-prefixed byte arrays
* Length encoded as varint, followed by raw bytes
* Used for Y.js updates, state vectors, and string data

#### String Encoding

[Section titled “String Encoding”](#string-encoding)

* UTF-8 encoded strings
* Length-prefixed with varint length
* Used for document names and reason strings

## Message Flow Examples

[Section titled “Message Flow Examples”](#message-flow-examples)

### Document Synchronization Flow

[Section titled “Document Synchronization Flow”](#document-synchronization-flow)

```
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: Sync Step 1 (with state vector)
    Server->>Client: Sync Step 2 (with missing updates)
    Client->>Server: Sync Done
    Server->>Client: Sync Done
    Client->>Server: Doc Update (real-time changes)
    Server->>Client: Doc Update (propagated to other clients)
```

### Awareness Flow

[Section titled “Awareness Flow”](#awareness-flow)

```
sequenceDiagram
    participant Client
    participant Server

    Server->>Client: Awareness Request (request current user states)
    Client->>Server: Awareness Update (user cursor/selection)
    Server->>Client: Awareness Update (other clients' user states)
```

### File Upload Flow

[Section titled “File Upload Flow”](#file-upload-flow)

```
sequenceDiagram
    participant Client
    participant Server

    Client->>Client: Encrypt file, fold Merkle root → contentId
    Client->>Server: fileUpload request (fileId=contentId, metadata, chunkSize)
    Note over Server: Permission check; dedup/resume via contentId
    Server->>Client: fileUpload response (allowed, existingChunks?, alreadyExists?)
    Client->>Server: fileUpload stream (chunk 0)
    Server->>Client: ACK
    Note over Client,Server: ... (missing chunks only)
    Client->>Server: fileUpload stream (final chunk)
    Note over Server: Recompute Merkle tree,<br/>verify == contentId,<br/>move to durable storage
    Server->>Client: ACK (upload complete)
```

### File Download Flow

[Section titled “File Download Flow”](#file-download-flow)

```
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: fileDownload request (fileId: merkle root hash)
    Note over Server: Looks up file by fileId/contentId
    Server->>Client: fileDownload stream (chunk 0 + merkle proof)
    Note over Client: Verifies chunk
    Server->>Client: fileDownload stream (chunk 1 + merkle proof)
    Note over Client: Verifies chunk
    Note over Client,Server: ... (more chunks)
    Server->>Client: fileDownload stream (final chunk + merkle proof)
    Note over Client: Verifies all chunks,<br/>reconstructs file
    Server->>Client: fileDownload response (metadata; or RPC error if not found)
```

### Milestone Operations Flow

[Section titled “Milestone Operations Flow”](#milestone-operations-flow)

```
sequenceDiagram
    participant Client
    participant Server

    Client->>Server: List Request (request milestone list)
    Server->>Client: List Response (returns milestone metadata)
    Client->>Server: Snapshot Request (request specific snapshot)
    Server->>Client: Snapshot Response (returns snapshot data)
    Client->>Server: Create Request (create milestone with snapshot, optional name)
    Note over Server: Validates snapshot, stores milestone
    Server->>Client: Create Response (returns created milestone metadata)
    Client->>Server: Update Name Request (update milestone name)
    Server->>Client: Update Name Response (returns updated milestone)
```

## Error Handling

[Section titled “Error Handling”](#error-handling)

The protocol includes robust error handling:

* **Magic Number Validation**: Ensures message is valid Teleportal format (must start with `0x59 0x4A 0x53` / “YJS”)
* **Version Checking**: Verifies protocol version compatibility (currently only version `0x01` is supported)
* **Type Validation**: Validates message and payload types
* **Length Validation**: Ensures proper message boundaries using varint encoding
* **Decoding Errors**: Invalid messages throw descriptive errors with context about the failure
* **RPC Errors**: File, milestone, attribution, and other RPC operations fail through the RPC error payload (status code + details), which the client surfaces as an `RpcOperationError`

## Security Considerations

[Section titled “Security Considerations”](#security-considerations)

* **Encryption Flag**: Built-in support for encrypted payloads at the message level
* **Authentication**: Auth messages provide access control for documents and operations
* **Validation**: All inputs are validated before processing
* **Merkle Tree Verification**: File transfers use cryptographic proofs to ensure data integrity
* **Message IDs**: A non-cryptographic content fingerprint (64-bit FNV-1a) used for deduplication and ack correlation. It is *not* a security primitive — do not rely on it for authenticity or anti-replay

## How It All Fits Together

[Section titled “How It All Fits Together”](#how-it-all-fits-together)

The Teleportal protocol integrates multiple subsystems into a cohesive collaborative editing platform. The following diagram illustrates how all components interact:

```
graph TB
    subgraph Transport["Transport Layer"]
        WS[WebSocket]
        HTTP[HTTP]
        CUSTOM[Custom Transport]
    end

    subgraph Protocol["Teleportal Protocol"]
        subgraph Core["Wire Message Types (0x00–0x04)"]
            DOC[Document 0x00]
            AWARE[Awareness 0x01]
            ACK[ACK 0x02]
            PRESENCE[Presence 0x03]
            RPC[RPC 0x04]
        end

        subgraph Extended["RPC Methods (carried by 0x04)"]
            FILE[File Transfer]
            MILESTONE[Milestones]
            ATTR[Attribution]
        end

        ENCODE[Message Encoding]
        DECODE[Message Decoding]
    end

    subgraph Features["Protocol Features"]
        SYNC[Document Sync]
        USERPRESENCE[User Presence]
        TRANSFER[File Transfer]
        VERSIONING[Milestone Versioning]
        EXTENSIBILITY[RPC Extensibility]
    end

    subgraph Storage["Storage Layer"]
        DOC_STORE[Document Storage]
        FILE_STORE[File Storage]
        MILESTONE_STORE[Milestone Storage]
    end

    Transport --> Protocol
    Protocol --> Features
    Features --> Storage

    DOC --> SYNC
    AWARE --> USERPRESENCE
    PRESENCE --> USERPRESENCE
    FILE --> TRANSFER
    MILESTONE --> VERSIONING
    RPC --> EXTENSIBILITY

    ACK -.->|Reliability| FILE
    ACK -.->|Delivery Confirmation| DOC

    SYNC --> DOC_STORE
    TRANSFER --> FILE_STORE
    VERSIONING --> MILESTONE_STORE

    style Protocol fill:#e1f5ff
    style Features fill:#c8e6c9
    style Storage fill:#fff9c4
```

### System Integration

[Section titled “System Integration”](#system-integration)

1. **Document Synchronization**: The core Y.js sync protocol (sync-step-1, sync-step-2, updates) ensures all clients have consistent document state. The bidirectional sync allows both client and server to request missing updates.

2. **Awareness System**: Runs in parallel with document sync, providing real-time presence information. This enables collaborative features like showing cursors and selections without affecting document state.

3. **File Transfer**: An RPC method (`fileUpload`/`fileDownload`) with chunked streaming and Merkle tree verification. Files are content-addressable (identified by Merkle root hash), enabling deduplication and integrity verification.

4. **Milestone Management**: An RPC method group (`milestoneList`/`milestoneCreate`/…) that captures document snapshots at specific points. The lazy loading design (metadata first, snapshots on demand) enables efficient browsing of document history.

5. **RPC System**: The single extension point (wire type `0x04`). File transfer, milestones, attribution, and key registry are all built on it, as are your own custom methods — no core protocol changes required.

6. **Message Acknowledgment**: ACK messages enable reliable delivery tracking, particularly important for file transfers where chunks must be verified and confirmed.

7. **Transport Layer**: The protocol is transport-agnostic, working over WebSockets, HTTP, or any binary-capable transport. The message format is self-contained and doesn’t depend on transport-specific features.

### Message Flow Integration

[Section titled “Message Flow Integration”](#message-flow-integration)

```
sequenceDiagram
    participant C1 as Client 1
    participant S as Server
    participant C2 as Client 2

    Note over C1,S: Document Synchronization
    C1->>S: Sync Step 1
    S->>C1: Sync Step 2
    S->>C1: Sync Step 1
    C1->>S: Sync Step 2
    S->>C1: Sync Done

    Note over C1,S: Awareness (Parallel)
    C1->>S: Awareness Update
    S->>C2: Awareness Update

    Note over C1,S: Real-time Updates
    C1->>S: Document Update
    S->>C2: Document Update

    Note over C1,S: File Transfer (RPC)
    C1->>S: fileUpload request
    S->>C1: fileUpload response
    C1->>S: fileUpload stream (chunk)
    S->>C1: ACK
    C1->>S: fileUpload stream (final)
    S->>C1: ACK (complete)

    Note over C1,S: Milestones (RPC)
    C1->>S: milestoneCreate request
    S->>C1: milestoneCreate response
```

All these systems work together through the unified message format, enabling efficient, type-safe, and extensible collaborative editing while maintaining compatibility with the Y.js ecosystem.

# Scaling

> Scaling Teleportal deployments from a single node to a multi-node cluster

This guide covers scaling strategies for Teleportal deployments, from the simplest single-server setup to horizontally scaled clusters with PubSub coordination.

## Single-Node Deployment

[Section titled “Single-Node Deployment”](#single-node-deployment)

A single-node deployment is the simplest model. One server instance handles all client connections, with in-memory or local storage and no coordination overhead.

This is a good fit for prototyping, small teams, or low-traffic applications where a single process can handle all concurrent sessions.

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  storage: async (ctx) => {
    return documentStorage;
  },
});
```

No `pubSub` or `nodeId` configuration is needed. All clients connect to the same process, so document state is always consistent.

## Multi-Node with PubSub

[Section titled “Multi-Node with PubSub”](#multi-node-with-pubsub)

When a single server is not enough, you can run multiple Teleportal instances behind a load balancer. The challenge is coordination: if Client A connects to Server 1 and Client B connects to Server 2, and both are editing the same document, the two servers need a way to exchange updates. Without PubSub, clients on different servers would diverge into inconsistent states.

A PubSub backend solves this by acting as a message bus between server instances. When one server receives a document update from a client, it publishes that update to a PubSub topic. Every other server subscribed to that topic receives the update and forwards it to its own connected clients.

### Message Flow

[Section titled “Message Flow”](#message-flow)

```
sequenceDiagram
    participant A as Client A
    participant S1 as Server 1
    participant PS as PubSub Topic
    participant S2 as Server 2
    participant B as Client B

    A->>S1: Document update
    S1->>S1: Apply update locally
    S1->>PS: Publish update (sourceId: server-1)
    PS->>S2: Deliver update
    S2->>S2: Apply update locally
    S2->>B: Forward update to client
    Note over S1,PS: Server 1 ignores its own<br/>published messages via sourceId
```

When Server 1 receives an update from Client A, it applies the update to its local document state and then publishes the update to the PubSub topic for that document. Server 2, which also has clients editing the same document, receives the update via its subscription and forwards it to Client B. The result is that both clients converge on the same document state, even though they are connected to different servers.

### Source ID Filtering

[Section titled “Source ID Filtering”](#source-id-filtering)

Every PubSub message includes a `sourceId` identifying the server that published it. When a server receives a message from the PubSub topic, it checks the `sourceId` and ignores messages it published itself. This prevents message loops where a server would re-process its own updates.

### Topic-Based Routing

[Section titled “Topic-Based Routing”](#topic-based-routing)

Messages are routed to PubSub topics based on document ID (by default, `document/{documentId}`). Servers only subscribe to topics for documents that have active sessions. This means a server with no clients editing a particular document will not receive updates for that document, keeping the message volume proportional to actual activity.

## PubSub Implementation Options

[Section titled “PubSub Implementation Options”](#pubsub-implementation-options)

Teleportal ships with two PubSub backends and an in-memory implementation for testing.

### Redis PubSub

[Section titled “Redis PubSub”](#redis-pubsub)

Redis is widely supported, easy to set up, and familiar to most teams. It is a good choice for moderate scale.

```typescript
import { Server } from "teleportal/server";
import { RedisPubSub } from "teleportal/transports/redis";


const server = new Server({
  storage: async (ctx) => {
    return documentStorage;
  },
  pubSub: new RedisPubSub({
    path: "redis://localhost:6379",
  }),
  nodeId: process.env.NODE_ID || "node-1",
});
```

Trade-offs to be aware of:

* Messages are fire-and-forget. Redis PubSub does not persist messages or support replay. If a subscriber is temporarily disconnected, it will miss any messages published during that window.
* Fan-out to many subscribers on a single Redis instance can add latency under high load.
* Redis requires separate connections for publishing and subscribing. The `RedisPubSub` class manages this internally, creating two connections per instance.

### NATS

[Section titled “NATS”](#nats)

NATS is a lightweight, high-performance messaging system designed specifically for this kind of inter-service communication. It handles high message throughput well and supports clustering for availability.

```typescript
import { Server } from "teleportal/server";
import { NatsPubSub } from "teleportal/transports/nats";
import { connect } from "@nats-io/transport-node";


const server = new Server({
  storage: async (ctx) => {
    return documentStorage;
  },
  pubSub: new NatsPubSub(() => connect({ servers: "nats://localhost:4222" })),
  nodeId: process.env.NODE_ID || "node-1",
});
```

Trade-offs:

* NATS is additional infrastructure that may be less familiar to your team.
* Core NATS PubSub is also fire-and-forget (like Redis). For durability, NATS JetStream can persist messages, but Teleportal uses the core PubSub interface.
* NATS supports native clustering and leaf nodes for multi-region deployments.

### In-Memory PubSub

[Section titled “In-Memory PubSub”](#in-memory-pubsub)

The in-memory PubSub implementation is useful for testing multi-server behavior in a single process. It is not shared across nodes and should never be used in production multi-node deployments.

### Running Multiple Instances

[Section titled “Running Multiple Instances”](#running-multiple-instances)

With either backend, run each instance with a unique `NODE_ID`:

```bash
# Instance 1
NODE_ID=node-1 PORT=3000 bun run server.ts


# Instance 2
NODE_ID=node-2 PORT=3001 bun run server.ts


# Instance 3
NODE_ID=node-3 PORT=3002 bun run server.ts
```

## PubSub Pressure and Considerations

[Section titled “PubSub Pressure and Considerations”](#pubsub-pressure-and-considerations)

PubSub becomes a bottleneck when many documents are actively edited simultaneously or when individual documents generate high update frequency (for example, many users typing rapidly in the same document). There are several strategies to reduce cross-server PubSub traffic.

### Co-locating Users

[Section titled “Co-locating Users”](#co-locating-users)

Use session affinity to route users who are editing the same document to the same server instance. When all collaborators on a document are on the same server, updates stay local and do not need to traverse the PubSub layer at all. PubSub is only needed for the (ideally rare) case where a document has active sessions on multiple servers.

### Document Affinity / Sticky Sessions

[Section titled “Document Affinity / Sticky Sessions”](#document-affinity--sticky-sessions)

Route documents to specific servers using document ID hashing so that most updates stay local. Only documents that happen to span multiple servers (for example, during rebalancing) generate cross-server traffic.

### Monitoring PubSub Health

[Section titled “Monitoring PubSub Health”](#monitoring-pubsub-health)

Watch these signals to detect PubSub pressure:

* **Message rates**: A sustained increase in published messages per second may indicate that session affinity is not working effectively.
* **Message latency**: Growing latency between publish and delivery suggests the PubSub backend is overloaded.
* **Subscriber counts per topic**: High subscriber counts on a single topic mean many servers are serving the same document. Consider improving session affinity to consolidate those sessions.

## Session Affinity Strategies

[Section titled “Session Affinity Strategies”](#session-affinity-strategies)

Session affinity ensures that clients editing the same document are routed to the same server, reducing cross-server coordination.

### Document ID Hashing

[Section titled “Document ID Hashing”](#document-id-hashing)

The simplest approach: compute `hash(documentId) % serverCount` and route to the corresponding server. This is deterministic and requires no shared state, but rebalancing when servers are added or removed causes all clients to reconnect.

```typescript
function getServerForDocument(documentId: string, serverCount: number): number {
  let hash = 0;
  for (let i = 0; i < documentId.length; i++) {
    hash = (hash * 31 + documentId.charCodeAt(i)) | 0;
  }
  return Math.abs(hash) % serverCount;
}
```

### Consistent Hashing

[Section titled “Consistent Hashing”](#consistent-hashing)

Consistent hashing minimizes rebalancing when servers are added or removed. Instead of rehashing all documents, only a fraction of documents are reassigned to new servers. This is the preferred approach for deployments that scale up and down frequently.

### Load Balancer Sticky Sessions

[Section titled “Load Balancer Sticky Sessions”](#load-balancer-sticky-sessions)

Configure your load balancer to route based on a document identifier in the request. Common approaches:

* **Header-based**: Route on an `X-Document-Id` header sent by the client.
* **Query parameter**: Route on a `document` query parameter in the WebSocket upgrade URL (for example, `wss://example.com/sync?document=my-doc`).
* **Cookie-based**: Set a cookie on the first connection and route subsequent requests to the same server.

### DNS-Based Routing

[Section titled “DNS-Based Routing”](#dns-based-routing)

For regional deployments, route at the DNS level to send users to the nearest cluster. Within each region, use one of the above strategies for document-level affinity.

## Multi-Node with HTTP Load Balancer (No PubSub)

[Section titled “Multi-Node with HTTP Load Balancer (No PubSub)”](#multi-node-with-http-load-balancer-no-pubsub)

If you do not want to run a PubSub backend, you can still scale horizontally with a load balancer. The critical constraint is that **all clients editing the same document must connect to the same server instance**. Sticky sessions are mandatory, not optional. Without PubSub, there is no mechanism to synchronize document state between servers.

```typescript
const server = new Server({
  storage: async (ctx) => {
    // Shared storage backend (e.g., PostgreSQL, S3)
    return documentStorage;
  },
  // No pubSub configured
});
```

This model is simpler to operate (no PubSub infrastructure to manage), but it means that a single server failure will disconnect all clients for the documents it was serving. Those clients must reconnect to another server and reload the document from shared storage.

## Document Sharding

[Section titled “Document Sharding”](#document-sharding)

For very large deployments, you can shard documents across server groups. Each shard handles a subset of documents, and routing is determined by the document ID:

```typescript
function getShardForDocument(documentId: string, shardCount: number): number {
  let hash = 0;
  for (let i = 0; i < documentId.length; i++) {
    hash = (hash * 31 + documentId.charCodeAt(i)) | 0;
  }
  return Math.abs(hash) % shardCount;
}
```

Each shard can be an independent cluster with its own PubSub backend and storage, allowing you to scale different shards independently based on their load characteristics.

## Storage Scaling

[Section titled “Storage Scaling”](#storage-scaling)

Storage is independent of the server topology. You can use different backends for different storage types to optimize cost and performance:

```typescript
import { UnstorageMilestoneStorage } from "teleportal/storage";
import { PostgresDocumentStorage } from "teleportal/storage/postgres";
import { S3FileStorage } from "teleportal/storage/s3";
import { createStorage } from "unstorage";
import redisDriver from "unstorage/drivers/redis";


// PostgreSQL for document state (strong consistency, querying)
const documentStorage = new PostgresDocumentStorage(sql);


// S3 for file uploads (cheap, durable, high throughput)
const fileStorage = new S3FileStorage(s3Config);


// Redis (via unstorage) for milestones (fast reads, TTL support)
const redisStore = createStorage({ driver: redisDriver({ base: "milestone:" }) });
const milestoneStorage = new UnstorageMilestoneStorage(redisStore, { keyPrefix: "milestone" });
```

When scaling storage, consider:

* **Read replicas**: Offload read traffic from the primary database.
* **Connection pooling**: Use connection pools to avoid exhausting database connections as the number of server instances grows.
* **Regional replication**: Replicate storage across regions to reduce latency for geographically distributed users.

## Monitoring a Scaled Deployment

[Section titled “Monitoring a Scaled Deployment”](#monitoring-a-scaled-deployment)

Monitoring becomes essential once you are running multiple server instances.

### Key Metrics Per Node

[Section titled “Key Metrics Per Node”](#key-metrics-per-node)

Each Teleportal server exposes Prometheus-compatible metrics:

* `teleportal_clients_active` – current number of connected clients on this node
* `teleportal_sessions_active` – current number of active document sessions on this node
* `teleportal_messages_total_all` – total messages processed by this node (all types); `teleportal_messages_total` is the same count labeled by `type`

### Cross-Node Aggregation

[Section titled “Cross-Node Aggregation”](#cross-node-aggregation)

Aggregate metrics across all nodes to understand total system load. Use Prometheus federation or a central metrics collector to combine per-node metrics into cluster-wide dashboards.

### Signs of Scaling Issues

[Section titled “Signs of Scaling Issues”](#signs-of-scaling-issues)

* **Uneven session distribution**: Some servers are overloaded while others are idle. Improve your load balancing or session affinity strategy.
* **High PubSub message latency**: The PubSub backend is becoming a bottleneck. Consider improving session affinity to reduce cross-server traffic, or scaling the PubSub infrastructure itself.
* **Increasing storage operation durations**: Storage is under pressure. Add read replicas, improve connection pooling, or shard storage.

### Health Check Endpoints

[Section titled “Health Check Endpoints”](#health-check-endpoints)

Use the `/health` and `/status` endpoints for load balancer health checks. Configure your load balancer to remove unhealthy instances from the pool automatically. Note that the built-in `/health` handler is a liveness stub – it returns `status: "healthy"` with an empty `checks` object whenever the process is up. For deeper readiness signals (storage or PubSub reachability), derive them from `/status` fields or extend the server’s health logic.

See [Observability](/docs/guides/observability/) for full setup details on metrics, health endpoints, and structured logging.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Pub/Sub Guide](/docs/guides/pub-sub/) – Set up PubSub for multi-server coordination
* [Performance](/docs/advanced/performance/) – Optimize throughput and latency
* [Observability](/docs/guides/observability/) – Configure metrics, health checks, and logging

# Attribution

> Track who wrote or deleted every piece of content in a collaborative document

Attribution tracks **who** inserted or deleted each piece of content in a Y.js document, and **when**. It answers questions like “who wrote this paragraph?” and “what changed between yesterday and today?” — the foundation for features like author highlighting, activity feeds, and audit trails.

## Overview

[Section titled “Overview”](#overview)

Every Y.js CRDT operation has a unique ID: `(clientID, clock)`. The attribution system maps these operation IDs to authorship metadata — userId, timestamp, and optional custom attributes — in a compact binary structure called a **ContentMap**.

```
flowchart LR
    Client[Client sends update] --> Server[Server extracts operation IDs]
    Server --> Tag[Tag with userId + timestamp]
    Tag --> Store[Store ContentMap alongside update]
    Store --> Query[Clients query attribution on demand]
```

The server computes and stores attribution automatically. Clients query it on demand via RPC — attribution data is **not** synced continuously, only fetched when requested.

## How It Works

[Section titled “How It Works”](#how-it-works)

### Server-Side: Computing Attribution

[Section titled “Server-Side: Computing Attribution”](#server-side-computing-attribution)

When a client sends a Y.js update, the server:

1. **Extracts operation IDs** from the update — which operations were inserted and which were deleted
2. **Tags** each operation range with the authenticated userId and current timestamp
3. **Encodes** the result as a binary ContentMap
4. **Stores** it alongside the update in storage

```typescript
// This happens automatically inside the server — no configuration needed
// beyond enabling attribution in your storage implementation.
```

The server emits a `document-attribution` event on every attributed update, which you can use for real-time integrations:

```typescript
server.on(
  "document-attribution",
  ({ documentId, namespacedDocumentId, sessionId, userId, timestamp, contentMap }) => {
    // React to attribution changes in real-time
  },
);
```

### Client-Side: Querying Attribution

[Section titled “Client-Side: Querying Attribution”](#client-side-querying-attribution)

Attribution is accessed via a client RPC extension registered on the `Provider`. The extension queries the server and resolves content ranges locally:

```typescript
import { Provider } from "teleportal/providers";
import { createAttributionRpc } from "teleportal/protocols/attribution";
import { createEncryptionKey } from "teleportal/encryption-key";


const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document",
  encryptionKey: createEncryptionKey(), // encrypted by default
  rpc: {
    attribution: createAttributionRpc,
  },
});


// Activity timeline — who edited, and when?
const activity = await provider.rpc.attribution.getActivity();


// Who wrote characters 0..100 of a Y.Text?
const text = provider.doc.getText("body");
const segments = await provider.rpc.attribution.getForRange(text, 0, 100);
// → [{ from: 0, to: 45, userId: "alice", timestamp: 1700000000,
//       attributes: { insert: "alice", insertAt: 1700000000 } },
//    { from: 45, to: 100, userId: "bob", timestamp: 1700000500,
//       attributes: { insert: "bob", insertAt: 1700000500 } }]


// Point lookup by CRDT ID
const author = await provider.rpc.attribution.resolveItem(clientID, clock);
// → { userId: "alice", timestamp: 1700000000, attributes: { ... } }
```

`getForRange` works with any Y.js sequence type — `Y.Text`, `Y.Array`, `Y.XmlText`, `Y.XmlFragment`, etc. For key-value types like `Y.Map`, use `resolveItem` with the CRDT ID of the item you’re interested in.

## Custom Attributes

[Section titled “Custom Attributes”](#custom-attributes)

You can attach arbitrary metadata by providing an `attributionConfig` when creating the server. The returned attributes are stored as-is on both the insert and delete sides of the ContentMap:

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  storage: async (ctx) => storage,
  attributionConfig: {
    getAttributes: ({ context }) => ({
      source: context.source ?? "human",
      model: context.model ?? "unknown",
    }),
  },
});
```

Custom attributes are encoded, stored, and transmitted alongside standard attributes. Use cases include AI agent tagging, change source tracking, or any domain-specific metadata.

## Client API

[Section titled “Client API”](#client-api)

Each `Provider` instance represents a single document. Attribution methods are available on `provider.rpc.attribution` when the `createAttributionRpc` extension is registered — subdocuments have their own `Provider` instances with independent attribution.

### Activity Timeline

[Section titled “Activity Timeline”](#activity-timeline)

`getActivity` is the single entrypoint for “who did what, when?” — all filters compose with AND:

```typescript
const { attribution } = provider.rpc;


// All activity
await attribution.getActivity();


// Filter by user
await attribution.getActivity({ userId: "alice" });


// Time range
await attribution.getActivity({ from: hourAgo, to: now });


// Scoped to a milestone
await attribution.getActivity({ milestone: milestoneId });


// Changes between two milestones
await attribution.getActivity({ changeset: [fromId, toId] });


// Custom attribute filter
await attribution.getActivity({ attributes: { source: "ai" } });


// Combine any filters
await attribution.getActivity({ milestone: milestoneId, userId: "alice" });
```

Each entry includes an `attributes` record containing all attributes (standard and custom):

```typescript
// → [{ from: 1700000000, to: 1700000500, userId: "alice",
//       attributes: { insert: "alice", insertAt: 1700000000, source: "human" } }, ...]
```

Adjacent entries from the same user within 1 second are grouped, but only when their attributes match — entries from the same user but different custom attributes (e.g. human vs AI) stay separate.

Tip

Activity works for **encrypted documents** — it is derived from authorship metadata and timestamps, never from document content. Milestone/changeset scoping is also E2EE-safe.

### Range Attribution

[Section titled “Range Attribution”](#range-attribution)

Resolve who authored a range of content in a Y type:

```typescript
const text = provider.doc.getText("body");
const segments = await provider.rpc.attribution.getForRange(text, 0, 100);
// → [{ from: 0, to: 45, userId: "alice", timestamp: 1700000000,
//       attributes: { insert: "alice", insertAt: 1700000000, source: "human" } },
//    { from: 45, to: 100, userId: "bob", timestamp: 1700000500,
//       attributes: { insert: "bob", insertAt: 1700000500, source: "ai" } }]
```

Segments with different custom attributes are not merged, even if they have the same userId and timestamp. Runs **entirely client-side**, so it works identically for encrypted and unencrypted documents.

### Lower-Level Methods

[Section titled “Lower-Level Methods”](#lower-level-methods)

For advanced use cases, the raw ContentMap and point-lookup APIs are available:

```typescript
const { attribution } = provider.rpc;


// Raw ContentMap (fetched once, cached for subsequent calls)
const map = await attribution.getMap();
const filtered = await attribution.getMap({ userId: "alice" });
attribution.invalidateCache(); // force re-fetch on next use


// Point lookup by CRDT ID
const author = await attribution.resolveItem(clientID, clock);
// → { userId: "alice", timestamp: 1700000000, attributes: { ... } } | null


// Milestone-scoped ContentMaps (for direct set operations)
const milestoneMap = await attribution.getMilestoneContentMap(milestoneId);
const changesetMap = await attribution.getChangesetContentMap(fromId, toId);
```

```
flowchart LR
    Full[Full ContentMap] --> Intersect["Intersect with milestone IDs"]
    Milestone[Milestone Snapshot] --> Extract["Extract operation IDs"]
    Extract --> Intersect
    Intersect --> Scoped[Scoped ContentMap]
```

## Server Configuration

[Section titled “Server Configuration”](#server-configuration)

### Enabling Attribution

[Section titled “Enabling Attribution”](#enabling-attribution)

Attribution requires a storage implementation that supports it. Your storage must:

1. Accept the optional `attribution` parameter in `handleUpdate`
2. Implement the optional `retrieveAttribution` method

```typescript
import { type DocumentStorage, type EncodedContentMap } from "teleportal/storage";


class MyStorage implements DocumentStorage {
  async handleUpdate(documentId: string, update: VersionedUpdate, attribution?: EncodedContentMap) {
    // Persist the update
    await this.saveUpdate(documentId, update);


    // Persist attribution alongside the update (merge with existing)
    if (attribution) {
      await this.mergeAttribution(documentId, attribution);
    }
  }


  async retrieveAttribution(documentId: string): Promise<EncodedContentMap | null> {
    // Return the merged ContentMap for the document
    return this.loadAttribution(documentId);
  }
}
```

### Attribution RPC Handlers

[Section titled “Attribution RPC Handlers”](#attribution-rpc-handlers)

Register the attribution RPC handlers on the server to expose the query API to clients:

```typescript
import { Server } from "teleportal/server";
import { getAttributionRpcHandlers } from "teleportal/protocols/attribution";


const server = new Server({
  storage: async (ctx) => storage,
  rpcHandlers: {
    ...getAttributionRpcHandlers(),
  },
});
```

### Permission Control

[Section titled “Permission Control”](#permission-control)

Attribution RPC methods (`attributionActivity`, `attributionGet`) are covered by the server’s global `checkPermission` hook. Use the `rpcMethod` field to apply method-level authorization:

```typescript
const server = new Server({
  storage: async (ctx) => storage,
  checkPermission: async ({ context, documentId, rpcMethod }) => {
    if (rpcMethod === "attributionActivity" || rpcMethod === "attributionGet") {
      return canReadAttribution(context.userId, documentId);
    }
    // ... other permission logic
  },
  rpcHandlers: {
    ...getAttributionRpcHandlers(),
  },
});
```

When `checkPermission` returns `false` (or throws), the RPC call fails with a `403`.

## Encryption Boundary

[Section titled “Encryption Boundary”](#encryption-boundary)

Documents are end-to-end-encrypted by default, and attribution works on them without the server ever seeing content.

Every Y.js operation has a structural ID — `(clientID, clock)` — that is separate from the content it represents. With content-level encryption the CRDT structure stays in **plaintext** and only the content is encrypted into sidecars, so the server reads these IDs directly from the plaintext structure update — the same code path as unencrypted documents — and tags them with the authenticated userId and timestamp. The client does not extract or ship IDs separately. The structure reveals the shape of the CRDT (which client wrote how many operations), but never the text or data itself.

* The server holds only the encrypted content sidecars plus the plaintext structure and ContentMap (operation IDs + userId/timestamp — no document content)
* **`getActivity`** works — it is derived purely from authorship metadata
* **`getForRange`** works — it resolves content positions against the local decrypted document, entirely client-side
* **Milestone attribution** works — the client decrypts the milestone snapshot locally before extracting operation IDs

The server never sees document content in any of these flows.

## Set Operations

[Section titled “Set Operations”](#set-operations)

The attribution library provides a full set algebra over ContentMaps and ContentIds, useful for advanced use cases:

```typescript
import {
  mergeContentMaps,
  filterContentMap,
  intersectContentMap,
  excludeContentMap,
} from "teleportal/attribution";


// Merge multiple ContentMaps
const merged = mergeContentMaps([mapA, mapB, mapC]);


// Filter by attribute predicate
const byAlice = filterContentMap(contentMap, (attrs) => {
  const user = attrs.find((a) => a.name === "insert");
  return user?.val === "alice";
});


// Intersect: keep only ranges present in both
const scoped = intersectContentMap(fullMap, milestoneIds);


// Exclude: remove already-attributed ranges
const newOnly = excludeContentMap(fullMap, alreadyAttributedIds);
```

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Milestones](/docs/core-concepts/milestones/) – Scope attribution to document snapshots
* [Provider](/docs/core-concepts/provider/) – Client-side API reference
* [Server](/docs/core-concepts/server/) – Server configuration and events
* [Authentication](/docs/core-concepts/authentication/) – How userId flows into attribution

# Authentication

> Why authentication matters in collaborative editing and how Teleportal secures document access

In a collaborative editing system, authentication is not just about knowing who a user is — it determines what documents they can access, what operations they can perform, and how their actions are attributed. Unlike a traditional web app where a session cookie gates access to pages, a sync server must authorize every individual message: each keystroke, cursor move, or file upload passes through a permission check.

Teleportal provides two approaches: a built-in JWT token system with IAM-like document access patterns, or a bring-your-own-auth hook for integrating with your existing identity provider.

## Custom Authentication

[Section titled “Custom Authentication”](#custom-authentication)

You can bypass the built-in token system entirely by implementing your own logic in the `onUpgrade` and `checkPermission` hooks. This is useful when you already have session management (cookies, OAuth tokens, API keys) and don’t want to introduce a second token format.

### The onUpgrade Hook

[Section titled “The onUpgrade Hook”](#the-onupgrade-hook)

The `onUpgrade` hook runs when a WebSocket connection is first established. It receives the HTTP request and returns a context object that is attached to the connection for its lifetime:

```typescript
const handlers = getWebsocketHandlers({
  onUpgrade: async (request) => {
    // Use your existing session/auth system
    const user = await verifySession(request);
    if (!user) throw new Response("Unauthorized", { status: 401 });


    return { context: { userId: user.id, role: user.role } };
  },
});
```

### The checkPermission Hook

[Section titled “The checkPermission Hook”](#the-checkpermission-hook)

The `checkPermission` hook runs before every message is processed. It receives the connection context (from `onUpgrade`), the target document or file, and the message itself:

```typescript
const server = new Server({
  storage: async (ctx) => documentStorage,
  checkPermission: async ({ context, documentId, fileId, message, rpcMethod }) => {
    // Derive read/write intent from the MESSAGE, not the `type` argument.
    // (For inbound client messages `type` is always "write" — see the caveat below.)
    const isWrite =
      message.type === "doc" &&
      (message.payload.type === "sync-step-2" || message.payload.type === "update");
    return await yourAuthService.canAccess(context.userId, documentId, isWrite ? "write" : "read");
  },
});
```

This two-hook design separates identity verification (once, at connection time) from authorization (continuously, per message).

\`type\` is always \`write\` for inbound messages

The transport-level validator authorizes **both** the read (source) and write (sink) directions using the literal `"write"` — it does not compute per-message read/write intent. A custom `checkPermission` must **not** rely on the `type` argument to distinguish reads from writes; derive the real intent from the message itself (payload type for `doc`, method name for `rpc`), exactly as the built-in `checkPermissionWithTokenManager` does.

## Built-in JWT Token System

[Section titled “Built-in JWT Token System”](#built-in-jwt-token-system)

For applications that need a self-contained permission model, Teleportal provides a JWT-based token manager with document-level access control patterns.

### How It Works

[Section titled “How It Works”](#how-it-works)

The token lifecycle:

1. Your application server creates a token using `TokenManager`, encoding the user’s identity and document access rules
2. The client passes this token when connecting (via query parameter or Authorization header)
3. The `onUpgrade` hook verifies the token signature, expiration, and claims
4. The `checkPermission` hook checks the token’s document access patterns against each incoming message

### Token Structure

[Section titled “Token Structure”](#token-structure)

Each token contains:

```typescript
{
  userId: string;           // User identifier
  room: string;             // Room/organization identifier
  documentAccess?: [        // Document access patterns (OPTIONAL — see gotcha below)
    {
      pattern: string;      // Document pattern (`*` is the only wildcard)
      permissions: Permission[];
    }
  ];
  exp?: number;             // Expiration time (Unix timestamp) — set + enforced by verifyToken
  iat?: number;             // Issued at time (Unix timestamp)
  iss?: string;             // Issuer — set + enforced
  aud?: string;             // Audience (default: "teleportal") — set + enforced
}
```

The `room` field scopes the token to a single room or organization, preventing a token issued for one tenant from being used in another.

Fail-open when documentAccess is absent

`documentAccess` is **optional**, and `hasDocumentPermission` **fails open**: a token minted *without* a `documentAccess` claim grants **every** permission on **every** document in its room — effectively a room-admin token. This is deliberate (“an unrestricted, room-scoped token”) but is a footgun. Always attach a `documentAccess` policy (or use `createAdminToken` explicitly) unless you intend room-wide access. Note the asymmetry: `getDocumentPermissions` fails **closed** (returns `[]`) for the same missing claim.

### Permission Types

[Section titled “Permission Types”](#permission-types)

* **`read`**: View document content and receive awareness updates
* **`write`**: Modify document content
* **`comment`**: Add comments to documents
* **`suggest`**: Make suggestions for document changes
* **`admin`**: Full access to all operations (supersedes other permissions)

### Creating Tokens

[Section titled “Creating Tokens”](#creating-tokens)

```typescript
import { createTokenManager } from "teleportal/token";


const tokenManager = createTokenManager({
  secret: "your-secret-key-here",
  expiresIn: 3600, // 1 hour
  issuer: "my-collaborative-app",
});


// Token with explicit access patterns
const token = await tokenManager.createToken("user-123", "org-456", [
  { pattern: "shared/*", permissions: ["read", "comment"] },
  { pattern: "user-123/*", permissions: ["read", "write", "admin"] },
]);


// Admin token (full access to all documents in the room)
const adminToken = await tokenManager.createAdminToken("admin-789", "org-456");
```

### Verifying Tokens and Checking Permissions

[Section titled “Verifying Tokens and Checking Permissions”](#verifying-tokens-and-checking-permissions)

```typescript
const result = await tokenManager.verifyToken(token);
if (result.valid && result.payload) {
  // Check a specific permission against a document
  const canRead = tokenManager.hasDocumentPermission(result.payload, "user-123/document1", "read");


  // Get all permissions the token grants for a document
  const permissions = tokenManager.getDocumentPermissions(result.payload, "user-123/document1");
}
```

## Document Pattern Matching

[Section titled “Document Pattern Matching”](#document-pattern-matching)

The permission system uses pattern matching to map tokens to documents, similar to IAM policy resources. This avoids needing to enumerate every document in the token.

`*` is the **only** wildcard — it matches any run of characters (including none). **Every other character is matched literally.** Patterns are compiled to an anchored `RegExp` with all regex metacharacters escaped, so a pattern like `logs[prod]*` matches the literal text `logs[prod]…` and is never treated as a regex character class. This escaping is a security boundary: neither patterns nor document names can inject regex syntax.

### Exact Match

[Section titled “Exact Match”](#exact-match)

```typescript
pattern: "document1";
// Matches: "document1" only
```

### Prefix Match

[Section titled “Prefix Match”](#prefix-match)

```typescript
pattern: "user/*";
// Matches: "user/doc1", "user/doc2", "user/project/doc3"
```

### Wildcard Match

[Section titled “Wildcard Match”](#wildcard-match)

```typescript
pattern: "*";
// Matches: any document name
```

### Suffix Match

[Section titled “Suffix Match”](#suffix-match)

```typescript
pattern: "*.md";
// Matches: "readme.md", "document.md"
```

Patterns compose naturally with the permission system: a token can grant `read` to `shared/*` and `admin` to `user-123/*`, giving fine-grained control without per-document token issuance.

## DocumentAccessBuilder

[Section titled “DocumentAccessBuilder”](#documentaccessbuilder)

For complex permission scenarios, the `DocumentAccessBuilder` provides a fluent API to construct `DocumentAccess[]` arrays:

```typescript
import { DocumentAccessBuilder } from "teleportal/token";


// Basic allow/deny
const access = new DocumentAccessBuilder()
  .allow("user/*", ["read", "write"])
  .deny("private/*")
  .build();


// Permission convenience methods
const access = new DocumentAccessBuilder()
  .readOnly("public/*")
  .write("user/*")
  .fullAccess("admin/*")
  .admin("super-admin/*")
  .build();


// User-owned documents + custom patterns
const access = new DocumentAccessBuilder()
  .ownDocuments("user-123")
  .allow("shared/*", ["read", "write"])
  .allow("projects/my-project/*", ["read", "write"])
  .build();


// Complex patterns with exclusions
const access = new DocumentAccessBuilder()
  .allowAll(["read", "write"])
  .deny("private/*")
  .deny("*.secret")
  .ownDocuments("user-456", ["read", "write", "comment", "suggest", "admin"])
  .allow("projects/important-project/*", ["read", "write", "comment", "suggest"])
  .admin("system/*")
  .build();
```

## Server Integration

[Section titled “Server Integration”](#server-integration)

Here is a complete example integrating the token manager with the WebSocket server:

```typescript
import { getWebsocketHandlers } from "teleportal/websocket-server";
import { Server } from "teleportal/server";
import { createTokenManager } from "teleportal/token";


const tokenManager = createTokenManager({
  secret: "your-secret-key",
  expiresIn: 3600,
});


const server = new Server({
  storage: async (ctx) => documentStorage,
  checkPermission: async ({ context, documentId, fileId, message, type }) => {
    const token = (context as any).token;
    if (!token) return false;


    const result = await tokenManager.verifyToken(token);
    if (!result.valid || !result.payload) return false;


    // Verify room scope
    if (result.payload.room !== context.room) return false;


    // Check document permissions
    if (documentId) {
      const requiredPermission = message.type === "awareness" ? "read" : "write";
      return tokenManager.hasDocumentPermission(result.payload, documentId, requiredPermission);
    }


    return true;
  },
});


const handlers = getWebsocketHandlers({
  onUpgrade: async (request) => {
    const url = new URL(request.url);
    const authHeader = request.headers.get("authorization");
    const token =
      url.searchParams.get("token") ||
      (authHeader && /^bearer\s+/i.test(authHeader) ? authHeader.replace(/^bearer\s+/i, "") : null);


    if (!token) {
      throw new Response("No token provided", { status: 401 });
    }


    const result = await tokenManager.verifyToken(token);
    if (!result.valid || !result.payload) {
      throw new Response("Invalid token", { status: 401 });
    }


    return {
      context: {
        userId: result.payload.userId,
        room: result.payload.room,
        token,
      },
    };
  },
});
```

## Security Considerations

[Section titled “Security Considerations”](#security-considerations)

1. **Use strong secrets (≥256 bits)**: Generate cryptographically secure random secrets for signing tokens. `jose` does not enforce a minimum HMAC key length — that is on you.
2. **Algorithm is pinned to HS256**: Tokens are always signed and verified with **HS256** (symmetric HMAC-SHA-256). `verifyToken` pins `algorithms: ["HS256"]`, so HS384/HS512 tokens and unsecured `alg: "none"` tokens are rejected.
3. **Always attach a `documentAccess` policy**: A token minted without one fails **open** and grants room-wide admin access (see the danger callout above). Use `createAdminToken` only when you actually mean room-wide access.
4. **Set appropriate expiration**: Short-lived tokens limit the window of compromise. Prefer tokens that expire in minutes to hours, not days.
5. **Validate room access**: `hasDocumentPermission` ignores `payload.room` — you must compare the token’s `room` claim to the connection’s room yourself.
6. **Check permissions on every message**: The `checkPermission` hook runs per message by design — do not cache authorization results, as tokens may have been revoked. Remember its `type` argument is always `"write"`; derive read/write intent from the message.
7. **Use HTTPS/WSS**: Always use secure connections in production to prevent token interception.
8. **Rotate secrets**: Regularly rotate your JWT signing secrets.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Server](/docs/core-concepts/server/) - How the server uses authentication for permission enforcement
* [Guides: Authentication](/docs/guides/authentication/) - Step-by-step setup guide
* [Attribution](/docs/core-concepts/attribution/) - How userId from authentication flows into authorship tracking

# Milestones

> Document snapshots and version history

Milestones are document snapshots that represent the document state at a specific point in time. They provide version history and can be used to restore documents to previous states.

## Overview

[Section titled “Overview”](#overview)

A **Milestone** is a document snapshot as the client saw the document at a point in time. Milestones are:

* Stored separately from the document as a sort of version history
* Can be pulled back over the network again
* An example of the RPC system, which adds custom methods & handlers for custom operations over the transport layer in-protocol

Milestone operations live on the provider’s `rpc.milestones` namespace, which you enable by registering the `createMilestoneRpc` extension when creating the provider.

## Creating Milestones

[Section titled “Creating Milestones”](#creating-milestones)

### Manual Creation

[Section titled “Manual Creation”](#manual-creation)

Create a milestone from the current document state:

```typescript
import { Provider } from "teleportal/providers";
import { createMilestoneRpc } from "teleportal/protocols/milestone";
import { createEncryptionKey } from "teleportal/encryption-key";


const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document",
  encryptionKey: createEncryptionKey(), // encrypted by default
  rpc: {
    milestones: createMilestoneRpc,
  },
});


// Create a milestone with a name
const milestone = await provider.rpc.milestones.create("v1.0");


// Or let the server auto-generate a name
const milestone2 = await provider.rpc.milestones.create();
```

### Automatic Creation

[Section titled “Automatic Creation”](#automatic-creation)

Milestones can be created automatically based on triggers:

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  // ... other options
  milestoneTriggerConfig: {
    defaultTriggers: [
      {
        id: "hourly",
        type: "time-based",
        enabled: true,
        config: { interval: 3600000 }, // every hour
      },
      {
        id: "every-1000",
        type: "update-count",
        enabled: true,
        config: { updateCount: 1000 }, // after 1000 updates
      },
    ],
  },
});
```

> **Note**: Time-based and update-count triggers are evaluated **on the document-write path only** — there is no background timer. An idle document never accumulates milestones; a time-based trigger fires on the next write after its interval has elapsed. Automatic milestones are attributed to `{ type: "system", id: "auto" }`.

## Listing Milestones

[Section titled “Listing Milestones”](#listing-milestones)

List all milestones for a document:

```typescript
// List all milestones
const milestones = await provider.rpc.milestones.list();


// Include soft-deleted milestones, or send known ids for incremental updates
const withDeleted = await provider.rpc.milestones.list({ includeDeleted: true });
const incremental = await provider.rpc.milestones.list({ snapshotIds: existingMilestoneIds });
```

## Retrieving Milestones

[Section titled “Retrieving Milestones”](#retrieving-milestones)

Get the snapshot content for a specific milestone:

```typescript
// Get milestone snapshot
const snapshot = await provider.rpc.milestones.getSnapshot(milestoneId);


// The snapshot is a Uint8Array containing the Y.js document state (decrypted
// client-side for encrypted documents). You can apply it to a new document:
import * as Y from "yjs";


const newDoc = new Y.Doc();
Y.applyUpdateV2(newDoc, snapshot);
```

> **Note**: Milestones work on encrypted documents. Since content E2EE is the default, the client decrypts the snapshot locally — the server stores and serves only the encrypted snapshot and never sees the document content.

## Updating Milestones

[Section titled “Updating Milestones”](#updating-milestones)

Update a milestone’s name:

```typescript
await provider.rpc.milestones.updateName(milestoneId, "v1.0.1");
```

## Soft Delete and Restore

[Section titled “Soft Delete and Restore”](#soft-delete-and-restore)

Milestones support soft delete and restore:

```typescript
// Soft delete a milestone
await provider.rpc.milestones.delete(milestoneId);


// Restore a deleted milestone
await provider.rpc.milestones.restore(milestoneId);
```

## Milestone Metadata

[Section titled “Milestone Metadata”](#milestone-metadata)

Each milestone includes metadata:

```typescript
interface Milestone {
  id: string;
  name: string;
  documentId: string;
  createdAt: number;
  deletedAt?: number;
  lifecycleState?: "active" | "deleted" | "archived" | "expired";
  expiresAt?: number;
  createdBy: {
    type: "user" | "system";
    id: string;
  };
}
```

The `createdBy` field indicates who or what created the milestone:

* `{ type: "user", id: userId }` - Created by a user via the `milestoneCreate` RPC method
* `{ type: "system", id: "auto" }` - Created automatically by a trigger

> **Note**: A freshly-created milestone has `lifecycleState === undefined`, which every `getMilestones` filter and `Milestone.toString()` treats as `"active"`. The string `"active"` is not written to the wire in the common case — a milestone only carries an explicit `lifecycleState` once it has been deleted, archived, or expired.

## Server Configuration

[Section titled “Server Configuration”](#server-configuration)

To enable milestone operations on the server:

```typescript
import { Server } from "teleportal/server";
import { getMilestoneRpcHandlers } from "teleportal/protocols/milestone";
import { UnstorageDocumentStorage, UnstorageMilestoneStorage } from "teleportal/storage";


const milestoneStorage = new UnstorageMilestoneStorage(storage, {
  keyPrefix: "document-milestone",
});


const server = new Server({
  storage: async (ctx) =>
    new UnstorageDocumentStorage(storage, { keyPrefix: "doc", encrypted: ctx.encrypted }),
  rpcHandlers: {
    ...getMilestoneRpcHandlers(milestoneStorage),
  },
});
```

## Storage

[Section titled “Storage”](#storage)

Milestones are stored separately from documents using `MilestoneStorage`:

```typescript
import type { MilestoneStorage } from "teleportal/storage";


// MilestoneStorage interface (abridged)
interface MilestoneStorage {
  readonly type: "milestone-storage";


  createMilestone(ctx: {
    name: string;
    documentId: string;
    createdAt: number;
    snapshot: MilestoneSnapshot;
    createdBy: { type: "user" | "system"; id: string };
  }): Promise<string>; // returns the created milestone id


  getMilestone(documentId: string, id: string): Promise<Milestone | null>;


  getMilestones(
    documentId: string,
    options?: { includeDeleted?: boolean; lifecycleState?: Milestone["lifecycleState"] },
  ): Promise<Milestone[]>;


  deleteMilestone(documentId: string, id: string | string[], deletedBy?: string): Promise<void>;
  restoreMilestone(documentId: string, id: string | string[]): Promise<void>;
  updateMilestoneName(
    documentId: string,
    id: string,
    name: string,
    createdBy?: { type: "user" | "system"; id: string },
  ): Promise<void>;
}
```

## Use Cases

[Section titled “Use Cases”](#use-cases)

### Version History

[Section titled “Version History”](#version-history)

Create milestones at important points in the document lifecycle:

```typescript
// Create milestone when user clicks "Save"
await provider.rpc.milestones.create("Save point 1");


// Create milestone when document is published
await provider.rpc.milestones.create("Published v1.0");
```

### Document Restoration

[Section titled “Document Restoration”](#document-restoration)

Restore a document to a previous milestone:

```typescript
// Get milestone snapshot
const snapshot = await provider.rpc.milestones.getSnapshot(milestoneId);


// Apply to current document
import * as Y from "yjs";
Y.applyUpdateV2(provider.doc, snapshot);
```

### Automatic Backups

[Section titled “Automatic Backups”](#automatic-backups)

Use automatic triggers to create regular backups:

```typescript
const server = new Server({
  milestoneTriggerConfig: {
    defaultTriggers: [
      {
        id: "hourly-backup",
        type: "time-based",
        enabled: true,
        config: { interval: 3600000 }, // every hour (fires on the next write after the interval)
      },
    ],
  },
});
```

## Best Practices

[Section titled “Best Practices”](#best-practices)

1. **Name milestones meaningfully**: Use descriptive names like “v1.0”, “Published”, “Before refactor”
2. **Create milestones at important points**: Save milestones at key moments in the document lifecycle
3. **Use automatic triggers**: Set up automatic milestone creation for regular backups
4. **Clean up old milestones**: Periodically delete or archive old milestones to save storage
5. **Track createdBy**: Use the `createdBy` field to distinguish user vs system milestones

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Provider](/docs/core-concepts/provider/) - Learn how to use milestones from the client
* [Server](/docs/core-concepts/server/) - Configure milestone triggers on the server
* [Custom Storage](/docs/guides/custom-storage/) - Understand milestone storage

# Protocol

> Why the Teleportal synchronization protocol is designed the way it is

The Teleportal protocol is a binary protocol built on top of Y.js. It defines five wire message types — document sync, awareness, ack, presence, and RPC — where higher-level features like file transfers and milestones ride on top of RPC. This document explains the design decisions and rationale behind the protocol’s architecture.

## Multiplexing Multiple Documents

[Section titled “Multiplexing Multiple Documents”](#multiplexing-multiple-documents)

One of the core design principles of the Teleportal protocol is **multiplexing**: the ability to sync multiple documents over a single connection. This is why every message is tagged with a `documentId`.

**Why multiplexing?** In real-world applications, clients often need to work with multiple documents simultaneously. Without multiplexing, you’d need a separate connection for each document, which would:

* Consume more network resources
* Increase connection overhead
* Make connection management more complex
* Limit scalability

By tagging each message with a `documentId`, the protocol allows a single WebSocket or HTTP connection to handle synchronization for many documents at once. The server and client can route messages to the correct document handler based on this identifier.

## Encryption Semantics

[Section titled “Encryption Semantics”](#encryption-semantics)

**Content-level end-to-end encryption is the default.** Every message carries an **encryption flag** in its header so encrypted and unencrypted documents can share the same connection.

**Why different semantics?** When a document is encrypted:

* The payload structure changes (the content is split into a plaintext CRDT **structure update** plus an encrypted **sidecar**, rather than a raw Y.js update)
* The server still merges, diffs, and syncs the structure update exactly as it does for plaintext — but it **never** sees the content or the key (those stay client-side, encrypted with AES-GCM)
* Storage tags the document as encrypted but otherwise treats the bytes identically

By marking messages as encrypted or not, the protocol ensures that:

* The server knows how to process each message correctly
* Clients can handle encrypted and unencrypted documents in the same connection
* The protocol can evolve encryption features independently

The server enforces that all clients of a single document agree on its encryption mode — a mismatch is rejected. See [`teleportal/protocol/encryption`](/docs/advanced/protocol/) for the wire format and sidecar details.

## Document Synchronization Flow

[Section titled “Document Synchronization Flow”](#document-synchronization-flow)

The synchronization process uses a two-step handshake to efficiently determine what updates need to be exchanged. This design minimizes the amount of data transferred during initial sync.

**Why a two-step process?** When a client connects, both the client and server may have updates the other doesn’t have. The protocol uses state vectors (compact representations of document state) to determine what’s missing:

```
sequenceDiagram
    participant C as Client
    participant S as Server

    Note over C,S: Client initiates sync
    C->>S: sync-step-1 (client state vector)

    Note over S: Server compares state vectors
    S->>C: sync-step-2 (server sends missing updates)

    Note over S: Server sends its state vector
    S->>C: sync-step-1 (server state vector)

    Note over C: Client compares state vectors
    C->>S: sync-step-2 (client sends missing updates)

    Note over C,S: Both sides now synchronized
    S->>C: sync-done
```

**The flow explained:**

1. **Client → Server (sync-step-1)**: Client sends its state vector, telling the server “this is what I have”
2. **Server → Client (sync-step-2)**: Server compares state vectors and sends only the updates the client is missing
3. **Server → Client (sync-step-1)**: Server sends its state vector, telling the client “this is what I have”
4. **Client → Server (sync-step-2)**: Client compares state vectors and sends only the updates the server is missing
5. **Server → Client (sync-done)**: Server confirms synchronization is complete

This bidirectional exchange ensures both sides converge to the same state with minimal data transfer, only sending what’s actually needed.

## Connection Health: Ping/Pong

[Section titled “Connection Health: Ping/Pong”](#connection-health-pingpong)

The protocol includes **ping/pong message types** to check if the connection is still active.

**Why ping/pong?** Network connections can fail silently. A client might think it’s connected, but the connection could be dead. Without a keep-alive mechanism:

* Dead connections would go undetected
* Clients wouldn’t know when to reconnect
* Resources would be wasted maintaining broken connections

Ping/pong messages allow both client and server to:

* Detect dead connections proactively
* Trigger reconnection logic when needed
* Maintain connection health metrics

The server or client can send a ping, and the recipient must respond with a pong. If no pong is received within a timeout period, the connection is considered dead and should be closed.

## Message Reliability: ACK Messages

[Section titled “Message Reliability: ACK Messages”](#message-reliability-ack-messages)

**ACK (acknowledgment) messages** allow the client to know whether the server actually received and processed the message it was sent.

**Why ACKs?** In distributed systems, message delivery isn’t guaranteed. A message could be:

* Lost in transit
* Received but not processed due to an error
* Processed but the response lost

Without ACKs, clients have no way to know if their messages were successfully handled. ACK messages provide:

* **Reliability**: Clients can retry if no ACK is received
* **Confirmation**: Clients know their updates were applied
* **Error detection**: Servers can send negative ACKs for failed operations

This is especially important for critical operations like document updates, where you need to ensure changes are persisted.

## Protocol Extensibility: RPC Messages

[Section titled “Protocol Extensibility: RPC Messages”](#protocol-extensibility-rpc-messages)

**RPC (Remote Procedure Call) messages** allow you to extend the protocol to add your own logic for sending and receiving data over the multiplexed connection.

**Why RPC?** While document synchronization is the core use case, real applications need more:

* File uploads and downloads
* Metadata operations (like milestones)
* Custom business logic
* Integration with external services

Rather than creating separate protocols or connections for these features, RPC messages let you:

* Reuse the same connection infrastructure
* Leverage existing authentication and multiplexing
* Add custom functionality without protocol changes

**Built-in RPC methods** demonstrate this pattern:

* **Milestones**: `milestoneList`, `milestoneCreate`, `milestoneGet`, etc.
* **Files**: `fileUpload`, `fileDownload`
* **Attribution**: `attributionActivity`, `attributionGet`, etc.
* **Key registry**: key distribution methods

None of these are dedicated wire message types — they are all RPC methods (wire type `0x04`), showing how the protocol can be extended for domain-specific needs without changing the core wire format.

> **Note**: The RPC API is still in development, and more work will be done on improving the ergonomics of the API for custom RPC handlers.

## Binary Encoding for Efficiency

[Section titled “Binary Encoding for Efficiency”](#binary-encoding-for-efficiency)

Messages are **encodable and decodable from binary** for efficient transmission over the network.

**Why binary?** Text-based protocols (like JSON) are human-readable but inefficient:

* Larger payload sizes (text encoding overhead)
* Slower parsing (string parsing is CPU-intensive)
* More bandwidth usage (especially for large Y.js updates)

Binary encoding provides:

* **Size efficiency**: Compact representation of data
* **Speed**: Fast encoding/decoding operations
* **Bandwidth savings**: Critical for real-time synchronization where every byte counts

The protocol uses a structured binary format with:

* **Header**: magic (`YJS`), version, document name, encryption flag, and the 1-byte message type
* **Payload**: Type-specific binary data (Y.js updates, awareness states, presence, RPC data, etc.)

This binary format is optimized for the specific needs of document synchronization, where you’re frequently sending Y.js updates that are already in binary format.

## Design Philosophy

[Section titled “Design Philosophy”](#design-philosophy)

The Teleportal protocol is designed around a few core principles:

1. **Efficiency**: Minimize bandwidth and processing overhead
2. **Reliability**: Ensure message delivery and connection health
3. **Extensibility**: Allow custom functionality without protocol changes
4. **Simplicity**: Keep the core protocol simple while supporting complex use cases

These principles guide every design decision, from the binary encoding format to the RPC extensibility mechanism.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Server](/docs/core-concepts/server/) - Learn how the server processes protocol messages
* [Transport](/docs/core-concepts/transport/) - Understand how messages are transmitted
* [Provider](/docs/core-concepts/provider/) - See how clients use the protocol

# Provider

> The client-side API for connecting to Teleportal servers

The `Provider` is the client-side API that manages Y.js document synchronization, awareness, offline persistence, and RPC operations. It wraps a `Connection` and handles the higher-level document synchronization protocol.

## Overview

[Section titled “Overview”](#overview)

The provider system is built on two main abstractions:

* **`Connection`**: Manages the low-level network connection, handles reconnection logic, message buffering, transport selection, and connection state. The default implementation is `DirectConnection`, which accepts an ordered list of pluggable transports.
* **`Provider`**: Manages Yjs document synchronization, awareness, offline persistence, and RPC operations. It uses a `Connection` for network communication.

### Why the Split?

[Section titled “Why the Split?”](#why-the-split)

The Connection and Provider are separate classes because they have fundamentally different lifecycles and responsibilities:

**Connection** handles network concerns: selecting a transport (WebSocket, HTTP, or custom), reconnecting with exponential backoff when the network drops, buffering and batching messages while disconnected, and tracking heartbeats. It knows nothing about Y.js documents.

**Provider** handles document concerns: running the Y.js sync protocol, managing awareness (cursor/presence) state, coordinating offline persistence with IndexedDB, and exposing RPC operations like milestones and attribution. It relies on a Connection for transport but doesn’t manage the connection itself.

This separation enables **connection reuse** – multiple Providers can share a single Connection, which is how Teleportal multiplexes many documents over one WebSocket. It also enables **independent lifecycle management**: you can switch documents (destroying one Provider and creating another) without tearing down and re-establishing the network connection. The `switchDocument()` method exploits this directly. And when using a SharedWorker, the Connection lives in the worker thread while Providers remain in the main thread, a split that would be impossible if they were a single object.

## Basic Usage

[Section titled “Basic Usage”](#basic-usage)

```typescript
import { Provider } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


// Create a provider with automatic connection.
// By default creates a DirectConnection with [websocketTransport(), httpTransport()]
const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document-id",
  encryptionKey: createEncryptionKey(),
});


// Wait for document to be synced
await provider.synced;


// Access the Yjs document
const ymap = provider.doc.getMap("data");
ymap.set("key", "value");


// Listen to connection state
provider.on("update", (state) => {
  console.log("Connection state:", state.type);
});
```

## Connection and Transports

[Section titled “Connection and Transports”](#connection-and-transports)

### DirectConnection with Pluggable Transports

[Section titled “DirectConnection with Pluggable Transports”](#directconnection-with-pluggable-transports)

`DirectConnection` is the single connection class. Instead of separate connection classes for WebSocket and HTTP, it takes an ordered array of `ConnectionTransport` instances and manages transport selection, fallback, and auto-upgrade internally:

```typescript
import { DirectConnection, websocketTransport, httpTransport } from "teleportal/providers";


const connection = new DirectConnection({
  url: "wss://example.com",
  transports: [websocketTransport(), httpTransport()],
});
```

**How it works:**

* **Transport fallback**: On connect, transports are tried in order. If WebSocket fails (e.g. blocked by a corporate firewall), the HTTP transport is tried automatically.
* **Upgrade probing**: When connected on a non-preferred transport (e.g. HTTP), `DirectConnection` periodically probes the preferred transport (WebSocket). If the probe succeeds, it transparently upgrades back – no reconnection logic needed from the application.
* **Manual override**: Call `connection.switchTransport("http")` to force a specific transport and disable automatic upgrade probing.

`Provider.create()` uses this by default with `[websocketTransport({ timeout: 5000 }), httpTransport()]`:

```typescript
// These are equivalent:
const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document",
  encryptionKey: key,
});


const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document",
  encryptionKey: key,
  transports: [websocketTransport({ timeout: 5000 }), httpTransport()],
});
```

### Built-in Transport Factories

[Section titled “Built-in Transport Factories”](#built-in-transport-factories)

Two transport factories are provided:

**`websocketTransport(options?)`** – WebSocket-based transport. Supports `probe()` for upgrade detection.

```typescript
import { websocketTransport } from "teleportal/providers";


websocketTransport({
  timeout: 5000, // Connection timeout (ms)
  protocols: [], // WebSocket sub-protocols
  WebSocket: WebSocket, // WebSocket implementation override
});
```

**`httpTransport(options?)`** – HTTP/SSE-based transport. Uses Server-Sent Events for server-to-client messages and HTTP POST for client-to-server messages.

```typescript
import { httpTransport } from "teleportal/providers";


httpTransport({
  timeout: 10000, // SSE connection timeout (ms)
  fetch: fetch, // fetch implementation override
  EventSource: EventSource, // EventSource implementation override
  httpBatchingOptions: { maxBatchSize: 10, maxBatchDelay: 50 },
});
```

Note

The server-to-client direction uses SSE (Server-Sent Events), a text-based protocol, so binary messages are base64-encoded for that leg. The client-to-server direction uses HTTP POST with raw binary and has no encoding overhead.

### WebSocket-only or HTTP-only

[Section titled “WebSocket-only or HTTP-only”](#websocket-only-or-http-only)

```typescript
// WebSocket only (no fallback)
const connection = new DirectConnection({
  url: "wss://example.com",
  transports: [websocketTransport()],
});


// HTTP only
const connection = new DirectConnection({
  url: "https://example.com",
  transports: [httpTransport()],
});
```

### SharedWorker Connection

[Section titled “SharedWorker Connection”](#sharedworker-connection)

Offloads the network connection to a [SharedWorker](https://developer.mozilla.org/en-US/docs/Web/API/SharedWorker) so all open tabs share a single underlying transport. Falls back to a direct in-thread connection when `SharedWorker` is unavailable.

```typescript
import { WorkerProvider } from "teleportal/providers/worker";
import { websocketTransport, httpTransport } from "teleportal/providers";


const provider = await WorkerProvider.create({
  workerUrl: new URL("./worker.ts", import.meta.url),
  url: "wss://example.com/sync",
  transports: [websocketTransport(), httpTransport()],
  document: "my-document",
  encryptionKey: key,
});


await provider.synced;
```

See the [SharedWorker Connection guide](/docs/guides/shared-worker/) for setup details, connection pooling, and configuration options.

## Document Operations

[Section titled “Document Operations”](#document-operations)

### Accessing the Document

[Section titled “Accessing the Document”](#accessing-the-document)

```typescript
// Access the Y.js document
const ydoc = provider.doc;


// Create Y.js types
const ytext = ydoc.getText("content");
const ymap = ydoc.getMap("data");
const yarray = ydoc.getArray("items");


// Make changes
ytext.insert(0, "Hello, world!");
ymap.set("key", "value");
yarray.push([1, 2, 3]);
```

### Listening to Updates

[Section titled “Listening to Updates”](#listening-to-updates)

```typescript
// Listen to document updates
ydoc.on("update", (update, origin) => {
  console.log("Document updated");
  // Changes are automatically synced to other clients
});
```

## Awareness

[Section titled “Awareness”](#awareness)

The provider includes an `Awareness` instance for user presence and cursor information:

```typescript
// Access awareness
const awareness = provider.awareness;


// Set local state
awareness.setLocalStateField("user", {
  name: "John Doe",
  color: "#ff0000",
});


// Listen to awareness updates
awareness.on("update", ({ added, updated, removed }) => {
  console.log("Users changed:", { added, updated, removed });
});
```

## Subdocuments

[Section titled “Subdocuments”](#subdocuments)

The provider supports Y.js sub-documents and properly lets the ydoc know that it is synced:

```typescript
// Listen to subdocument events
provider.on("load-subdoc", ({ subdoc, provider: subdocProvider }) => {
  console.log("Subdocument loaded:", subdoc.guid);
  // subdocProvider is a Provider instance for the subdocument
});


// Access subdocuments
const subdocProvider = provider.subdocs.get("subdoc-guid");
if (subdocProvider) {
  await subdocProvider.synced;
}
```

## Document Switching

[Section titled “Document Switching”](#document-switching)

Efficiently switch between documents while maintaining the same connection:

```typescript
// Switch to a new document (reuses connection)
const newProvider = provider.switchDocument({
  document: "new-document-id",
  encryptionKey: key,
});


// Old provider is destroyed, new provider is ready
await newProvider.synced;
```

## Offline Persistence

[Section titled “Offline Persistence”](#offline-persistence)

The provider automatically enables offline persistence by default using IndexedDB:

```typescript
const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document-id",
  encryptionKey: createEncryptionKey(),
  enableOfflinePersistence: true, // default
  indexedDBPrefix: "my-app-", // custom prefix
});


// Document will be loaded from IndexedDB if available
await provider.loaded; // Resolves when local data is loaded
await provider.synced; // Resolves when synced with server
```

## Connection State

[Section titled “Connection State”](#connection-state)

Monitor connection state:

```typescript
// Get current connection state
const state = provider.state;


if (state.type === "connected") {
  console.log("Connected via:", state.transport);
} else if (state.type === "errored") {
  console.error("Connection error:", state.error);
}


// Check active transport
console.log("Active transport:", provider.connection.activeTransport);
console.log("Available transports:", provider.connection.availableTransports);


// Wait for connection
try {
  await provider.synced;
  console.log("Fully synced!");
} catch (error) {
  console.error("Sync failed:", error);
}
```

## Connection Sharing

[Section titled “Connection Sharing”](#connection-sharing)

Multiple providers can share the same connection:

```typescript
import { DirectConnection, websocketTransport, httpTransport } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


// Create a connection
const connection = new DirectConnection({
  url: "wss://example.com",
  transports: [websocketTransport(), httpTransport()],
});


const key = createEncryptionKey();


// Create multiple providers with the same connection
const provider1 = new Provider({
  connection,
  document: "doc-1",
  encryptionKey: key,
});


const provider2 = new Provider({
  connection,
  document: "doc-2",
  encryptionKey: key,
});
```

## Lifecycle Management

[Section titled “Lifecycle Management”](#lifecycle-management)

The provider supports explicit resource management:

```typescript
// Using explicit resource management
using provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document",
  encryptionKey: key,
});


// Provider automatically disposes when exiting scope


// Or manually destroy
provider.destroy({
  destroyConnection: true, // default: true
  destroyDoc: true, // default: true
});
```

## Events

[Section titled “Events”](#events)

The provider extends `Observable` and emits events:

```typescript
// Subdocument events
provider.on("load-subdoc", ({ subdoc, provider }) => {
  console.log("Subdocument loaded");
});


provider.on("unload-subdoc", ({ subdoc, provider }) => {
  console.log("Subdocument unloaded");
});


// Connection state events (delegated from connection)
provider.on("update", (state) => {
  console.log("Connection state changed:", state.type);
});


// Peer presence events
provider.on("peer-join", (peer) => {
  console.log("Peer joined:", peer.userId);
});


provider.on("peer-leave", (peer) => {
  console.log("Peer left:", peer.userId);
});
```

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Server](/docs/core-concepts/server/) - Learn how the server handles provider connections
* [Transport](/docs/core-concepts/transport/) - Understand the transport layer
* [SharedWorker Connection](/docs/guides/shared-worker/) - Offload connections to a SharedWorker
* [Milestones](/docs/core-concepts/milestones/) - Learn about document versioning

# Server

> Why the Teleportal server exists and how it orchestrates collaborative document synchronization

The Teleportal server is the central orchestrator that enables real-time collaboration on Y.js documents. Rather than just managing connections, it solves the fundamental challenge of **coordinating multiple clients** who need to see each other’s changes while maintaining **security**, **performance**, and **reliability**.

## Why the Server Exists

[Section titled “Why the Server Exists”](#why-the-server-exists)

At its core, the server solves a coordination problem: when multiple clients are editing the same document, they need a trusted intermediary to:

* **Route messages** between clients who are interested in the same document
* **Enforce permissions** to ensure only authorized users can access or modify documents
* **Coordinate state** across multiple server instances for horizontal scaling
* **Protect against abuse** through rate limiting and validation
* **Provide visibility** into system health and operations

```
flowchart TB
    C1[Client 1] --> Server[Server]
    C2[Client 2] --> Server
    C3[Client 3] --> Server
    Server -->|routes messages| C1
    Server -->|routes messages| C2
    Server -->|routes messages| C3
```

## Connecting Clients to Document Sessions

[Section titled “Connecting Clients to Document Sessions”](#connecting-clients-to-document-sessions)

The server’s primary role is connecting clients to **document-based sessions**. Each session represents an active collaborative document that multiple clients are working on.

### Why Sessions Exist

[Section titled “Why Sessions Exist”](#why-sessions-exist)

Sessions exist because **clients need to know about each other’s messages**. When Client A makes a change, Client B and Client C need to receive that update. The session acts as a coordination point:

* **Groups interested clients**: All clients working on the same document are part of the same session
* **Broadcasts messages**: When one client sends an update, the session ensures all other clients in that session receive it
* **Manages document state**: The session maintains the authoritative document state and coordinates synchronization

```
sequenceDiagram
    participant C1 as Client 1
    participant Server
    participant Session
    participant C2 as Client 2

    C1->>Server: Request document
    Server->>Session: Create/get session
    Session->>C1: Add & sync

    C2->>Server: Request same document
    Server->>Session: Get session
    Session->>C2: Add & sync

    C1->>Session: Send update
    Session->>C2: Broadcast update
```

### Session Lifecycle

[Section titled “Session Lifecycle”](#session-lifecycle)

The server manages sessions with a clear lifecycle that balances **resource efficiency** with **responsiveness**:

1. **Creation**: When the first client requests a document, the server creates a session and loads the document from storage
2. **Active**: The session coordinates messages between all connected clients
3. **Cleanup**: When all clients disconnect, the session waits 60 seconds before cleanup (to handle quick reconnections)
4. **Disposal**: The session is removed from memory, but the document persists in storage

This lifecycle ensures that:

* **Memory is conserved**: Sessions only exist when needed
* **Reconnections are fast**: The 60-second grace period allows clients to reconnect without reloading
* **Documents persist**: Even when no session exists, documents remain in storage

### Encryption Mode

[Section titled “Encryption Mode”](#encryption-mode)

Sessions default to **encrypted** (`encrypted: true`), matching the content-level end-to-end encryption that clients apply by default. The server never sees document content or keys — it only merges and syncs the plaintext CRDT structure update and stores the encrypted sidecars. All clients of a single document must agree on its encryption mode; the server rejects a client whose mode differs from the open session, so switching a document between plaintext and encrypted is a hard cutover, not a per-client choice.

## Permission & Access Control

[Section titled “Permission & Access Control”](#permission--access-control)

The server **gates permissions and access control** in a flexible way, allowing you to either use your own implementation or leverage Teleportal’s built-in capabilities.

### Why Permissions Matter

[Section titled “Why Permissions Matter”](#why-permissions-matter)

Without permission checks, any client could:

* Access any document
* Modify documents they shouldn’t have access to
* Delete documents or files
* View sensitive information

The server provides a flexible `checkPermission` hook that runs **before every message is processed**, allowing you to implement your own authorization logic.

```
flowchart LR
    Client[Client] -->|Message| Server[Server]
    Server -->|Check| Permission{checkPermission}
    Permission -->|Allowed| Process[Process Message]
    Permission -->|Denied| Reject[Reject Message]
    Process --> Session[Session]
    Session --> OtherClients[Other Clients]
```

### Flexible Permission Model

[Section titled “Flexible Permission Model”](#flexible-permission-model)

You can implement permissions based on:

* **User identity**: Check if the user has access to the document
* **Document ownership**: Verify ownership or sharing permissions
* **Role-based access**: Implement role-based permissions (viewer, editor, admin)
* **Custom logic**: Any business logic you need

```typescript
const server = new Server({
  storage: async (ctx) => {
    // ... storage setup
  },
  checkPermission: async ({ context, documentId, fileId, message, type, rpcMethod }) => {
    const userId = context.userId;


    // Your custom permission logic
    if (documentId) {
      return await hasDocumentAccess(userId, documentId, type);
    } else if (fileId) {
      return await hasFileAccess(userId, fileId, type);
    }


    return false;
  },
});
```

## Client Lifecycle & Synchronization

[Section titled “Client Lifecycle & Synchronization”](#client-lifecycle--synchronization)

Understanding how clients are created, added to sessions, and synchronize with other clients is crucial for building reliable applications.

### Client Creation

[Section titled “Client Creation”](#client-creation)

When a client connects (via WebSocket, HTTP, or any transport), the server:

1. **Creates a client instance** with a unique ID
2. **Applies rate limiting** (if configured) to protect against abuse
3. **Sets up message validation** to enforce permissions
4. **Registers the client** for lifecycle tracking

```
sequenceDiagram
    participant Transport
    participant Server
    participant RateLimiter
    participant Validator
    participant Client

    Transport->>Server: Connection request
    Server->>RateLimiter: Wrap transport
    RateLimiter->>Validator: Wrap with validator
    Validator->>Client: Create client instance
    Server->>Client: Register client
    Server-->>Transport: Client ready
```

### Adding Clients to Sessions

[Section titled “Adding Clients to Sessions”](#adding-clients-to-sessions)

Once a client is created, it needs to join a session to participate in document collaboration:

1. **Client requests document**: The client sends a message indicating interest in a document
2. **Server gets or creates session**: The server either retrieves an existing session or creates a new one
3. **Session adds client**: The client is added to the session’s client list
4. **Initial synchronization**: The client receives the current document state
5. **Message routing**: The client can now send and receive messages through the session

```
sequenceDiagram
    participant Client
    participant Server
    participant Session
    participant OtherClients

    Client->>Server: Request document
    Server->>Session: Get or create session
    Session->>Client: Add client & sync state

    Client->>Session: Send update
    Session->>OtherClients: Broadcast to all
```

### Why This Lifecycle Matters

[Section titled “Why This Lifecycle Matters”](#why-this-lifecycle-matters)

This lifecycle ensures:

* **Consistent state**: All clients start with the same document state
* **Efficient synchronization**: Only necessary data is transferred
* **Automatic cleanup**: When clients disconnect, they’re automatically removed from sessions
* **Race condition prevention**: The server prevents multiple sessions from being created for the same document

## Deployment Modes

[Section titled “Deployment Modes”](#deployment-modes)

The server can operate in two distinct modes, depending on your scaling needs.

### Single Instance Mode

[Section titled “Single Instance Mode”](#single-instance-mode)

In single instance mode, the server runs without a pub-sub implementation. This is ideal for:

* **Development and testing**: Simple setup with no external dependencies
* **Small deployments**: When a single server instance can handle all traffic
* **Low-latency requirements**: Direct message routing without pub-sub overhead

```
flowchart TB
    C1[Client 1] --> Server[Server Instance]
    C2[Client 2] --> Server
    C3[Client 3] --> Server
    Server --> Session[Document Session]
    Session --> C1
    Session --> C2
    Session --> C3
```

### Multi-Instance with Pub-Sub

[Section titled “Multi-Instance with Pub-Sub”](#multi-instance-with-pub-sub)

For horizontal scaling, the server can use a pub-sub implementation (like Redis) to coordinate between multiple instances. This enables:

* **Horizontal scaling**: Add more server instances as traffic grows
* **High availability**: If one instance fails, others continue serving clients
* **Geographic distribution**: Deploy instances in different regions

```
flowchart TB
    subgraph "Instance 1"
        C1[Client 1] --> S1[Server 1]
        C2[Client 2] --> S1
        S1 --> Session1[Session A]
    end

    subgraph "Instance 2"
        C3[Client 3] --> S2[Server 2]
        C4[Client 4] --> S2
        S2 --> Session2[Session B]
    end

    S1 <--> PubSub[Pub-Sub<br/>Redis/NATS/etc]
    S2 <--> PubSub

    Session1 -.replicates.-> PubSub
    Session2 -.replicates.-> PubSub
```

When a client on Instance 1 sends a message, it’s:

1. Processed by the local session
2. Broadcast to other clients on Instance 1
3. Published to pub-sub
4. Received by Instance 2
5. Broadcast to clients on Instance 2

This ensures all clients see updates regardless of which instance they’re connected to.

## Rate Limiting & Abuse Prevention

[Section titled “Rate Limiting & Abuse Prevention”](#rate-limiting--abuse-prevention)

The server can **enforce rate limiting in a flexible way** and **kick clients for abuse**, protecting your system from:

* **Message flooding**: Clients sending too many messages too quickly
* **Resource exhaustion**: Attempts to consume server resources
* **Denial of service**: Malicious clients trying to disrupt service

### Why Rate Limiting Matters

[Section titled “Why Rate Limiting Matters”](#why-rate-limiting-matters)

Without rate limiting, a single malicious client could:

* Overwhelm the server with messages
* Cause other clients to experience delays
* Consume excessive storage or bandwidth
* Degrade the experience for legitimate users

### Flexible Rate Limiting

[Section titled “Flexible Rate Limiting”](#flexible-rate-limiting)

The server supports multiple rate limiting strategies:

* **Per-user**: Track limits across all documents for a user
* **Per-document**: Track limits across all users for a document
* **Per user-document pair**: Track limits for specific user-document combinations
* **Message size limits**: Prevent clients from sending oversized messages

```
flowchart LR
    Client[Client Message] --> RateLimiter{Rate Limiter}
    RateLimiter -->|Within Limits| Process[Process Message]
    RateLimiter -->|Exceeded| Kick[Disconnect Client]
    RateLimiter -->|Size Exceeded| Kick
```

When rate limits are exceeded, the server automatically disconnects the client, protecting the system from abuse.

```typescript
const server = new Server({
  storage: async (ctx) => {
    // ... storage setup
  },
  rateLimitConfig: {
    rules: [
      {
        id: "per-user",
        maxMessages: 100,
        windowMs: 1000,
        trackBy: "user",
      },
      {
        id: "per-document",
        maxMessages: 500,
        windowMs: 10000,
        trackBy: "document",
      },
    ],
    maxMessageSize: 10 * 1024 * 1024, // 10MB
    onRateLimitExceeded: (details) => {
      // Log or alert on abuse
      console.warn("Rate limit exceeded", details);
    },
  },
});
```

## Observability: Metrics, Health, & Logging

[Section titled “Observability: Metrics, Health, & Logging”](#observability-metrics-health--logging)

The server provides comprehensive observability to help you understand what’s happening in your system.

### Why Observability Matters

[Section titled “Why Observability Matters”](#why-observability-matters)

Without visibility into your server, you can’t:

* **Detect problems**: Know when something goes wrong
* **Understand usage**: See how clients are using your system
* **Plan capacity**: Make informed decisions about scaling
* **Debug issues**: Troubleshoot problems when they occur

### Metrics & Monitoring

[Section titled “Metrics & Monitoring”](#metrics--monitoring)

The server automatically tracks Prometheus-compatible metrics:

* **Active sessions**: How many documents are currently active
* **Active clients**: How many clients are connected
* **Messages processed**: Total messages and breakdown by type
* **Message duration**: How long messages take to process
* **Rate limit events**: When and why rate limits are triggered

```
flowchart TB
    Server[Server] --> Metrics[Metrics Collector]
    Metrics --> Prometheus[Prometheus Endpoint]
    Prometheus --> Dashboard[Monitoring Dashboard]

    Server --> Health[Health Check]
    Health --> HealthEndpoint[health endpoint]

    Server --> Status[Status Endpoint]
    Status --> StatusEndpoint[status endpoint]
```

### Health Checks

[Section titled “Health Checks”](#health-checks)

The server provides health check endpoints that return:

* **Overall status**: `healthy` or `unhealthy`
* **Individual checks**: Status of storage, pub-sub, and other components
* **Uptime**: How long the server has been running

This enables:

* **Load balancer health checks**: Automatically route traffic away from unhealthy instances
* **Monitoring alerts**: Get notified when the server becomes unhealthy
* **Automated recovery**: Trigger recovery actions when health checks fail

### Logging with LogTape

[Section titled “Logging with LogTape”](#logging-with-logtape)

The server uses [**LogTape**](https://logtape.org/) for structured logging, providing:

* **Structured logs**: JSON-formatted logs with consistent fields
* **Contextual information**: Each log includes relevant context (client ID, document ID, etc.)
* **Configurable sinks**: Send logs to console, files, or external services
* **Log levels**: Control verbosity with trace, debug, info, warn, error levels

You configure LogTape globally, and the server automatically uses it:

```typescript
import { configure } from "@logtape/logtape";


configure({
  sinks: [
    // Console for development
    new ConsoleSink(),
    // Or your production logging service
  ],
});
```

The server logs important events like:

* Client connections and disconnections
* Session creation and cleanup
* Document loads and unloads
* Permission denials
* Rate limit violations
* Errors and warnings

## Event System

[Section titled “Event System”](#event-system)

The server **emits events for important lifecycle events** to keep your system in the loop on what it’s doing.

### Why Events Matter

[Section titled “Why Events Matter”](#why-events-matter)

Events enable you to:

* **Integrate with external systems**: Send webhooks, update databases, trigger workflows
* **Build custom monitoring**: Track events in your own monitoring system
* **Implement business logic**: React to server events in your application
* **Debug and audit**: Understand what happened and when

### Available Events

[Section titled “Available Events”](#available-events)

The server emits events for:

**Document Lifecycle**:

* `document-load`: When a document session is created and loaded
* `document-unload`: When a document session is disposed
* `document-delete`: When a document is deleted from storage

**Client Lifecycle**:

* `client-connect`: When a client connects to the server
* `client-disconnect`: When a client disconnects (with reason)
* `client-message`: Every message sent or received (for metrics/webhooks)

**Session Events**:

* `session-open`: When a new session is opened
* `client-join`: When a client joins a session
* `client-leave`: When a client leaves a session
* `document-message`: When a message is applied to a document

**Operational Events**:

* `rate-limit-exceeded`: When a client exceeds rate limits
* `document-size-warning`: When a document exceeds size warning threshold
* `milestone-created`: When a milestone is created
* `before-server-shutdown`: Before server shutdown starts
* `after-server-shutdown`: After server shutdown completes

```
sequenceDiagram
    participant Client
    participant Server
    participant EventSystem
    participant YourApp

    Client->>Server: Connect
    Server->>EventSystem: Emit "client-connect"
    EventSystem->>YourApp: Notify your application

    Client->>Server: Request document
    Server->>EventSystem: Emit "document-load"
    EventSystem->>YourApp: Notify your application

    Client->>Server: Send update
    Server->>EventSystem: Emit "document-message"
    EventSystem->>YourApp: Notify your application
```

### Using Events

[Section titled “Using Events”](#using-events)

You can listen to events to integrate with your systems:

```typescript
// Track client connections in your database
server.on("client-connect", async (data) => {
  await db.clients.create({
    clientId: data.clientId,
    connectedAt: new Date(),
  });
});


// Send webhooks when documents are loaded
server.on("document-load", async (data) => {
  await sendWebhook({
    event: "document.opened",
    documentId: data.documentId,
    userId: data.context.userId,
  });
});


// Alert on rate limit violations
server.on("rate-limit-exceeded", async (data) => {
  await alertingService.send({
    severity: "warning",
    message: `Rate limit exceeded for user ${data.userId}`,
  });
});
```

## Summary

[Section titled “Summary”](#summary)

The Teleportal server exists to solve the coordination problem of real-time collaboration. It:

* **Connects clients to document sessions** so they can see each other’s changes
* **Enforces permissions** flexibly to protect your documents
* **Manages client lifecycle** from connection to synchronization
* **Scales horizontally** with pub-sub coordination
* **Protects against abuse** with flexible rate limiting
* **Provides observability** through metrics, health checks, and logging
* **Emits events** to keep your system informed

Understanding these “whys” helps you build reliable, secure, and scalable collaborative applications.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Transport](/docs/core-concepts/transport/) - Learn how messages are transmitted between clients and server
* [Provider](/docs/core-concepts/provider/) - See how clients connect to the server
* [Persistent Storage](/docs/guides/persistent-storage/) - Understand how documents are persisted
* [Guides](/docs/guides/) - Step-by-step implementation guides

# Transport

> How messages are transmitted between clients and servers

The transport layer handles how messages are sent and received between clients and servers. Teleportal’s transport system is built on **async iterables**, making it composable and allowing multiple transports to be chained together.

## Why Async Iterables?

[Section titled “Why Async Iterables?”](#why-async-iterables)

Teleportal uses async iterables (`AsyncIterable<Message[]>`) as the foundation for its transport layer rather than raw callbacks, event emitters, or Web Streams. This is a deliberate architectural choice that enables three properties:

* **Composability**: Transports are just iterables and write functions, so middleware (encryption, rate limiting, logging, validation) can be layered by wrapping one transport in another. Each layer is independent and testable in isolation. You can stack them in any order without the middleware needing to know about each other.
* **Backpressure**: The pull-based async iterable model naturally pauses the producer when the consumer is slow. The consumer pulls the next batch only when it is ready, preventing unbounded memory growth from buffered messages.
* **Runtime portability**: Async iterables are a language-level primitive available in every JavaScript runtime — browsers, Bun, Deno, Node.js, and Cloudflare Workers. By building on a language standard rather than a runtime-specific abstraction, the transport layer works everywhere without polyfills.

This means that adding a new transport (say, WebTransport or a custom binary protocol) only requires implementing an async iterable source and a write function — all existing middleware automatically applies to it.

## Overview

[Section titled “Overview”](#overview)

A **Transport** combines a **Source** (for reading messages) and a **Sink** (for writing messages):

```typescript
type Source<Context> = {
  source: AsyncIterable<Message<Context>[]>;
};


type Sink<Context> = {
  write(message: Message<Context>): void | Promise<void>;
  close(): void;
};


type Transport<Context> = Source<Context> & Sink<Context>;
```

The source yields **batches** (arrays) of messages — preserving the natural batching of the underlying transport (a single WebSocket frame might carry multiple protocol messages). The write function sends a single message. The close function tears down the transport.

## Architecture

[Section titled “Architecture”](#architecture)

The transport system follows a composable architecture:

1. **Base transports** handle the actual communication (HTTP, SSE, WebSocket, PubSub)
2. **Middleware transports** wrap other transports to add functionality (encryption, rate limiting, logging, validation)
3. **Utility functions** help compose, connect, and transform transports

### Rate Limiting

[Section titled “Rate Limiting”](#rate-limiting)

Throttles messages using a token bucket algorithm:

```typescript
import { withRateLimit } from "teleportal/transports/rate-limiter";


const rateLimitedTransport = withRateLimit(transport, {
  rules: [
    {
      id: "per-user",
      maxMessages: 100,
      windowMs: 1000,
      trackBy: "user",
    },
  ],
});
```

### Message Encryption

[Section titled “Message Encryption”](#message-encryption)

Wraps a transport with content-level end-to-end encryption:

```typescript
import { getEncryptedTransport } from "teleportal/transports";


const encryptedTransport = getEncryptedTransport(handler);
```

You rarely call this directly — the `Provider` applies it automatically because content encryption is the **default**. Pass an `encryptionKey` (a `CryptoKey`) when creating the provider, or `encryptionKey: false` to opt a document out into plaintext. The encrypted transport keeps the CRDT structure update in plaintext (so the server can still merge and sync) and encrypts only the document content into sidecars.

### Message Validation

[Section titled “Message Validation”](#message-validation)

Adds authorization checks to message reading and writing:

```typescript
import { withMessageValidator } from "teleportal/transports";


const validatedTransport = withMessageValidator(transport, {
  isAuthorized: async (message, type) => {
    // Your authorization logic. Note: for server-side transports the framework
    // always passes type "write" (both directions are checked with "write"),
    // so derive read/write from the message payload, not the `type` argument.
    return true;
  },
});
```

### Logging

[Section titled “Logging”](#logging)

Logs all messages for debugging:

```typescript
import { withLogger } from "teleportal/transports";


const loggedTransport = withLogger(transport);
```

### ACK Support

[Section titled “ACK Support”](#ack-support)

Adds acknowledgment message support for reliable message delivery:

```typescript
import { withAckSink, withAckTrackingSink } from "teleportal/transports";


// Server: Send ACKs automatically
const ackSink = withAckSink(sink, {
  pubSub,
  ackTopic: "acks",
  sourceId: "server-1",
});


// Client: Track ACKs
const trackingSink = withAckTrackingSink(sink, {
  pubSub,
  ackTopic: "acks",
  sourceId: "client-1",
  ackTimeout: 10000,
});
```

## Transport Composition

[Section titled “Transport Composition”](#transport-composition)

Transports can be composed in layers:

```typescript
// Base transport
let transport = getBaseTransport();


// Add encryption
transport = getEncryptedTransport(handler);


// Add rate limiting
transport = withRateLimit(transport, { rules: myRules });


// Add logging
transport = withLogger(transport);


// Add message validation
transport = withMessageValidator(transport, {
  isAuthorized: async (message, type) => {
    // Authorization logic
    return true;
  },
});
```

## Y.js Document Transport

[Section titled “Y.js Document Transport”](#yjs-document-transport)

The YDoc transport integrates Y.js documents with the Teleportal transport system:

```typescript
import { getYTransportFromYDoc } from "teleportal/transports";


const transport = getYTransportFromYDoc({
  ydoc,
  awareness,
  document: "my-document",
  context: { clientId: "client-1" },
});


// Start synchronization
await transport.handler.start();
```

## Utility Functions

[Section titled “Utility Functions”](#utility-functions)

### Compose

[Section titled “Compose”](#compose)

Combines a Source and Sink into a Transport:

```typescript
import { compose } from "teleportal/transports";


const transport = compose(source, sink);
```

### Connect

[Section titled “Connect”](#connect)

Drains all messages from a source and writes them to a sink (one-directional):

```typescript
import { connect } from "teleportal/transports";


await connect(source, sink);
```

### Sync

[Section titled “Sync”](#sync)

Bidirectionally connects two transports — each transport’s source feeds the other’s sink:

```typescript
import { sync } from "teleportal/transports";


await sync(transportA, transportB);
```

### Transform Helpers

[Section titled “Transform Helpers”](#transform-helpers)

Transform messages flowing through a source. Each helper takes the per-item function and returns a transform `(source) => source` that operates over the batched `AsyncIterable` (a fully-filtered batch is skipped, never yielded empty):

```typescript
import { mapMessages, filterMessages, flatMapMessages } from "teleportal/transports";


// Transform each message (return null/undefined to drop it)
const mapped = mapMessages((msg) => transform(msg))(source);


// Drop messages that don't match a predicate
const filtered = filterMessages((msg) => msg.type === "doc")(source);


// Expand each message into zero or more outputs
const expanded = flatMapMessages((msg) => [msg, derivedMsg])(source);
```

### Binary Conversion

[Section titled “Binary Conversion”](#binary-conversion)

Convert between message-level and binary transports:

```typescript
import { toBinaryTransport, fromBinaryTransport } from "teleportal/transports";


// Message transport → binary (for wire transmission)
const binaryTransport = toBinaryTransport(transport, context);


// Binary transport → message (for protocol processing)
const messageTransport = fromBinaryTransport(binaryTransport, context);
```

## Best Practices

[Section titled “Best Practices”](#best-practices)

1. **Compose from bottom up**: Start with base transports, then add middleware
2. **Handle errors**: Transports can error, ensure proper error handling
3. **Clean up resources**: Call `close()` when done
4. **Use appropriate transports**: Choose transports based on your use case
5. **Rate limit client transports**: Always rate limit client-side transports to prevent abuse
6. **Validate messages**: Use message validators for authorization checks
7. **Monitor with logging**: Use logger transport during development

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Server](/docs/core-concepts/server/) - Learn how the server uses transports
* [Provider](/docs/core-concepts/provider/) - See how clients use transports
* [Advanced: Custom Transport](/docs/advanced/custom-transport/) - Implement your own transport

# Getting Started

> Build your first real-time collaborative app with Teleportal in under 10 minutes

This tutorial walks you through creating a working real-time collaborative app with Teleportal. By the end, you’ll have a server syncing Y.js documents and a client that connects, edits, and receives live updates.

## Prerequisites

[Section titled “Prerequisites”](#prerequisites)

You need a JavaScript runtime installed. Teleportal works with:

* **Bun** (recommended) – [bun.sh](https://bun.sh)
* **Node.js 24+** – [nodejs.org](https://nodejs.org)

You should also have basic familiarity with Y.js. If you haven’t used Y.js before, [What is Teleportal?](/docs/what-is-teleportal/) covers the foundations.

## Installation

[Section titled “Installation”](#installation)

Create a new project and install Teleportal:

* Bun

  ```bash
  mkdir my-teleportal-app && cd my-teleportal-app
  bun init -y
  bun add teleportal yjs crossws srvx
  ```

* npm

  ```bash
  mkdir my-teleportal-app && cd my-teleportal-app
  npm init -y
  npm install teleportal yjs crossws srvx
  ```

## Step 1: Create the Server

[Section titled “Step 1: Create the Server”](#step-1-create-the-server)

Create a file called `server.ts` with the following:

```typescript
import { serve } from "crossws/server";
import { Server } from "teleportal/server";
import { MemoryDocumentStorage } from "teleportal/storage";
import { getWebsocketHandlers } from "teleportal/websocket-server";


// Create a Teleportal server with in-memory storage
const server = new Server({
  storage: new MemoryDocumentStorage(),
});


// Set up WebSocket handlers
serve({
  websocket: getWebsocketHandlers({
    server,
    onUpgrade: async () => {
      // Extract user context from the request
      // In production, you'd verify authentication here
      return {
        context: { userId: "user-123", room: "workspace-1" },
      };
    },
  }),
  fetch: () => new Response("Not found", { status: 404 }),
});
```

Here is what each piece does:

* **`Server`** is the core of Teleportal. It manages document sessions, coordinates sync between clients, and handles the Y.js protocol.
* **`MemoryDocumentStorage`** stores documents in memory. This is fine for development – for production, swap in a persistent storage backend like SQLite or Postgres.
* **`getWebsocketHandlers`** bridges Teleportal to a WebSocket server. It handles connection upgrades, message routing, and disconnection cleanup.
* **`onUpgrade`** runs when a client connects. The returned `context` object is attached to the connection and can be used for access control. In production, you would verify a JWT or session token here.

## Step 2: Create the Client

[Section titled “Step 2: Create the Client”](#step-2-create-the-client)

Create a file called `client.ts`:

```typescript
import { Provider, websocketTransport } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


// Create a provider that connects to the server.
// End-to-end content encryption is the default, so an `encryptionKey` is
// required. Share this key with collaborators out-of-band (e.g. in the URL
// fragment) — the server never sees it.
const provider = await Provider.create({
  url: "ws://localhost:3000",
  document: "my-document",
  encryptionKey: createEncryptionKey(),
  transports: [websocketTransport()],
});


// Wait for the document to sync
await provider.synced;


// Create a Y.js text type and insert content
const ytext = provider.doc.getText("content");
ytext.insert(0, "Hello, world!");


console.log("Document updated:", ytext.toString());


// Listen for updates from other clients
provider.doc.on("update", () => {
  console.log("Document updated:", ytext.toString());
});


// Flush pending messages before cleanup
await provider.flush();
await provider.destroy();
```

Here is what each piece does:

* **`Provider.create()`** establishes a WebSocket connection to the server, performs the sync handshake, and returns a ready-to-use provider.
* **`websocketTransport()`** configures the WebSocket transport layer for real-time communication.
* **`encryptionKey`** enables end-to-end content encryption (E2EE), which is on by default. The server never sees your plaintext content. To share a document between clients, they need the same key – distribute it out-of-band (e.g. in the URL fragment). To run without encryption, pass `encryptionKey: false`.
* **`provider.synced`** resolves once the initial document state has been downloaded from the server.
* **`provider.doc`** is a standard Y.js `Y.Doc` – use it exactly as you would in any Y.js application.
* **Updates** are automatically synced to all other clients connected to the same document.
* **`provider.destroy()`** cleans up the connection when you’re done.

> **Note**: With content-level E2EE the server only ever sees the plaintext CRDT structure and encrypted content sidecars – never your text or keys. Offline persistence stores the encrypted wire representation to IndexedDB, so data at rest is encrypted too. One caveat: awareness/presence routing IDs (clientID/userId) travel in cleartext even though the awareness payload itself is encrypted.

## Step 3: Run It

[Section titled “Step 3: Run It”](#step-3-run-it)

Start the server:

* Bun

  ```bash
  bun run server.ts
  ```

* Node.js

  ```bash
  node --experimental-strip-types server.ts
  ```

The server starts listening on `ws://localhost:3000`. Then in a separate terminal, run the client:

* Bun

  ```bash
  bun run client.ts
  ```

* Node.js

  ```bash
  node --experimental-strip-types client.ts
  ```

You should see `Document updated: Hello, world!` printed as the client inserts text and the change round-trips through the server.

## Step 4: Test Collaboration

[Section titled “Step 4: Test Collaboration”](#step-4-test-collaboration)

To see real-time sync in action, open a second terminal and run the client again. Both clients connect to the same document, so edits from one appear in the other. Since both use `createEncryptionKey()` without a password, they derive the same encryption key from the document ID (`"my-document"`), allowing them to decrypt each other’s content. If you modify the client script to insert different text, you’ll see both clients converge to the same state – that’s the CRDT at work.

> **Note**: The document ID-based key derivation is suitable for getting started, but provides minimal security – anyone who knows the document ID can derive the key. For production, consider using `createEncryptionKey("password")` for password-based encryption, sharing keys via URL fragment (e.g., `#token=...`), or using the key registry for server-managed key distribution.

In a real application, the client code runs in a browser and connects to a Y.js editor binding (like [y-prosemirror](https://github.com/yjs/y-prosemirror), [y-monaco](https://github.com/nicklassandell/y-monaco), or [y-codemirror.next](https://github.com/yjs/y-codemirror.next)). The Provider works the same way regardless of the environment.

## How It Works

[Section titled “How It Works”](#how-it-works)

1. **Client connects**: The provider establishes a WebSocket connection to the server
2. **Sync protocol**: The client and server exchange state vectors to determine what updates are needed
3. **Document sync**: Missing updates are sent from server to client, and the document is synchronized
4. **Real-time updates**: When you make changes, they’re sent to the server and broadcast to all other clients
5. **Awareness**: Cursor positions and user presence are automatically synchronized

## What’s Next

[Section titled “What’s Next”](#whats-next)

Integration Guide

Decide which transport, storage, and auth strategy fits your app

Core Concepts

Understand the protocol, server architecture, and provider lifecycle

Guides

Step-by-step recipes for authentication, persistent storage, scaling, and more

# Guides

> Step-by-step guides for common Teleportal scenarios

These guides walk you through implementing common Teleportal features and patterns. Each guide includes working code examples and explanations.

WebSocket Only

Minimal server setup with WebSocket transport and in-memory storage

HTTP Transport

Using HTTP/SSE transport for environments where WebSockets are blocked

Fallback Connection

Automatic fallback from WebSocket to HTTP when WebSocket fails

SharedWorker Connection

Offload sync to a SharedWorker so multiple tabs share one connection

Authentication

JWT token-based authentication with permission management

Encryption Keys

Managing encryption keys for end-to-end encrypted documents

Persistent Storage

Using persistent storage backends instead of in-memory storage

Custom Storage

Implementing a custom storage backend

Encryption at Rest

Server-side storage encryption (separate from the default content E2EE)

File Transfers

Upload and download files through the Teleportal protocol

Custom RPC Methods

Extend the protocol with your own request/response methods

Pub/Sub

Multi-server setup with pub/sub for horizontal scaling

Rate Limiting

Rate limiting client messages to prevent abuse

Observability

Monitoring and health checks with Prometheus metrics

Cloudflare Workers

Run a sync server on Cloudflare Workers with Durable Object storage

# Authentication

> JWT token-based authentication with permission management

This guide demonstrates JWT token-based authentication for securing Teleportal connections. Both WebSocket and HTTP handlers verify tokens before allowing access.

Tokens are always signed with **HS256** (symmetric HMAC-SHA-256). `verifyToken` pins `algorithms: ["HS256"]`, so tokens signed with any other algorithm – including unsecured `alg: "none"` tokens – are rejected.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Setting up JWT token authentication using `createTokenManager`
* Using token authentication for secure connections
* Configuring permission checks with token manager
* Creating and using JWT tokens on the client side

## Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { serve } from "crossws/server";
import { Server } from "teleportal/server";
import { createTokenManager } from "teleportal/token";
import { getWebsocketHandlers } from "teleportal/websocket-server";


const tokenManager = createTokenManager({
  secret: "your-secret-key",
  expiresIn: 3600,
});


const server = new Server({
  storage: async (ctx) => {
    // Your storage implementation
    return documentStorage;
  },
  checkPermission: async ({ context, documentId, message, type }) => {
    const token = (context as any).token;
    if (!token) return false;


    const result = await tokenManager.verifyToken(token);
    if (!result.valid || !result.payload) return false;


    const payload = result.payload;
    const requiredPermission = type === "read" ? "read" : "write";
    return tokenManager.hasDocumentPermission(payload, documentId!, requiredPermission);
  },
});


serve({
  websocket: getWebsocketHandlers({
    server,
    onUpgrade: async (request) => {
      // Extract token from request
      const url = new URL(request.url);
      const authHeader = request.headers.get("authorization");
      const token =
        url.searchParams.get("token") ||
        (authHeader && /^bearer\s+/i.test(authHeader)
          ? authHeader.replace(/^bearer\s+/i, "")
          : null);


      if (!token) {
        throw new Response("No token provided", { status: 401 });
      }


      const result = await tokenManager.verifyToken(token);
      if (!result.valid || !result.payload) {
        throw new Response("Invalid token", { status: 401 });
      }


      return {
        context: {
          userId: result.payload.userId,
          room: result.payload.room,
          token,
        },
      };
    },
  }),
  fetch: () => new Response("Not found", { status: 404 }),
});
```

## Client Setup

[Section titled “Client Setup”](#client-setup)

```typescript
import { Provider } from "teleportal/providers";
import { createTokenManager } from "teleportal/token";
import { createEncryptionKey } from "teleportal/encryption-key";


// Create token manager (should match server secret)
const tokenManager = createTokenManager({
  secret: "your-secret-key",
});


// Generate token
const token = await tokenManager.createToken("user-123", "org-456", [
  { pattern: "user-123/*", permissions: ["read", "write"] },
]);


// Connect with token
const provider = await Provider.create({
  url: `wss://example.com?token=${token}`,
  document: "user-123/my-document",
  encryptionKey: createEncryptionKey(),
});


await provider.synced;
```

## Document Access Patterns

[Section titled “Document Access Patterns”](#document-access-patterns)

The `documentAccess` list carried in each token uses **literal glob matching**: `*` is the only wildcard (it matches any run of characters, including none), and every other character – including regex metacharacters like `.`, `[`, or `(` – is matched **literally**. A pattern such as `logs[prod]*` matches the literal text `logs[prod]…` and never behaves as a regex character class. A `!`-prefixed pattern is an exclusion: if any exclusion matches the document, access is denied regardless of inclusions.

Fail-open when documentAccess is absent

`hasDocumentPermission` returns `true` for **every** document and permission when a token has no `documentAccess` claim. A token minted without an access policy is effectively an admin token for its room. Always attach a `documentAccess` policy (as in the example above) – or use `createAdminToken` explicitly – unless you intend room-wide access. Note the asymmetry: `getDocumentPermissions` fails **closed** (returns `[]`) for the same missing claim.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Core Concepts: Authentication](/docs/core-concepts/authentication/) - Learn more about authentication
* [Persistent Storage](/docs/guides/persistent-storage/) - Add persistent storage

# Cloudflare Workers

> Run a Teleportal sync server on Cloudflare Workers with Durable Objects

`teleportal/cloudflare` runs a full Teleportal sync server on Cloudflare Workers, backed entirely by Durable Object storage – no KV namespace, R2 bucket, or external database required.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Hosting the Teleportal `Server` inside a Durable Object, with WebSocket and HTTP/SSE clients sharing one instance
* Every storage interface (documents, files, milestones, rate limits, key registry) implemented directly on Durable Object storage
* Handling Durable Object hibernation without dropping client state
* Deploying with `wrangler`, entirely offline-testable via `wrangler dev`

## Architecture

[Section titled “Architecture”](#architecture)

A Worker forwards all sync traffic – WebSocket upgrades **and** HTTP/SSE requests – to a single Durable Object instance, which hosts the Teleportal `Server`. Because every connection lands in the same instance, WebSocket clients and SSE/HTTP clients share one set of sessions and the in-memory PubSub, with no cross-instance coordination needed.

```
graph LR
  C1["WebSocket Client"] -->|"/api/*"| W["Worker"]
  C2["HTTP/SSE Client"] -->|"/api/*"| W
  W -->|"stub.fetch()"| DO["Durable Object<br/>Server + storage"]
```

## Storage

[Section titled “Storage”](#storage)

Every Teleportal storage interface has a direct implementation on Durable Object storage (`ctx.storage`) – there’s no adapter layer in between. Values ride structured clone, so updates, sidecars, chunks, and wrapped keys stay binary.

| Class                                 | Interface                |
| ------------------------------------- | ------------------------ |
| `DurableObjectDocumentStorage`        | `DocumentStorage`        |
| `DurableObjectFileStorage`            | `FileStorage`            |
| `DurableObjectTemporaryUploadStorage` | `TemporaryUploadStorage` |
| `DurableObjectMilestoneStorage`       | `MilestoneStorage`       |
| `DurableObjectRateLimitStorage`       | `RateLimitStorage`       |
| `DurableObjectKeyRegistryStorage`     | `KeyRegistryStorage`     |

A few implementation details worth knowing:

* The document pending log stores one update per key (`pending:{seq}`, `seq` zero-padded to 16 digits so lexicographic `list()` order equals insertion order), so appends are O(1) writes rather than a read-append-rewrite of a single array. Sequence allocation is concurrency-safe: an in-memory counter seeded once from the persisted log tail hands out numbers with no intervening `await`, so concurrent appends never collide.
* `transaction()` is an in-memory per-key mutex (`KeyedMutex`) – a Durable Object instance is the single writer for its own storage, so no TTL or advisory locks are needed. The mutex only serializes read-modify-write sequences on the same key across the `await` points where Durable Objects interleave concurrent requests.
* Milestones are scoped by `documentId`: both the meta doc and each content blob live under `milestone:{documentId}:…`, so a milestone id never resolves or hydrates under the wrong document.
* Rate-limit TTLs are stamped expiry timestamps, since Durable Object storage has no native TTL; expired state reads as absent and is deleted lazily.
* Use SQLite-backed Durable Objects (`new_sqlite_classes` in the migration config) – available on the free plan, with a 2 MiB per-value limit. A document’s compacted state and a milestone snapshot are each a single value, so this bounds document size. File chunks are 1 MiB and always fit.

## Durable Object Wiring

[Section titled “Durable Object Wiring”](#durable-object-wiring)

```typescript
import crossws from "crossws/adapters/cloudflare";
import { Server } from "teleportal/server";
import { getHTTPHandlers } from "teleportal/http";
import {
  DurableObjectDocumentStorage,
  getDurableObjectHandlers,
  getDurableObjectWebsocketHooks,
} from "teleportal/cloudflare";


export class TeleportalDurableObject {
  ctx;
  env;
  #handlers;


  constructor(state: DurableObjectState, env: Env) {
    this.ctx = state; // crossws expects the state on `.ctx`
    this.env = env;


    const server = new Server({
      storage: async () =>
        new DurableObjectDocumentStorage(state.storage, { keyPrefix: "document" }),
    });


    const getContext = () => ({ userId: "someone", room: "docs" });


    this.#handlers = getDurableObjectHandlers({
      ws: crossws({
        hooks: getDurableObjectWebsocketHooks({
          server,
          onUpgrade: async () => ({ context: getContext() }),
        }),
      }),
      http: getHTTPHandlers({ server, getContext }),
    });
  }


  fetch(request: Request) {
    return this.#handlers.fetch(this, request);
  }
  webSocketMessage(ws: WebSocket, message: ArrayBuffer | string) {
    return this.#handlers.webSocketMessage(this, ws, message);
  }
  webSocketClose(ws: WebSocket, code: number, reason: string, wasClean: boolean) {
    return this.#handlers.webSocketClose(this, ws, code, reason, wasClean);
  }
  webSocketPublish(topic: string, data: unknown, opts?: unknown) {
    return this.#handlers.webSocketPublish(this, topic, data, opts);
  }
}
```

The Worker resolves the instance and forwards everything – WebSocket upgrades pass through `stub.fetch` unchanged:

```typescript
export default {
  async fetch(request: Request, env: Env) {
    const stub = env.TELEPORTAL_DO.get(env.TELEPORTAL_DO.idFromName("teleportal"));
    return stub.fetch(request);
  },
};
```

Tip

Route WebSocket upgrades and HTTP/SSE requests under the same path prefix (e.g. `/api/*`) so both transports reach the same Durable Object instance. A WebSocket upgrade is a plain `GET /`, so if you also configure Workers `[assets]`, keep the API prefix distinct from your static asset paths or the upgrade will never reach the Worker.

## WebSockets and Hibernation

[Section titled “WebSockets and Hibernation”](#websockets-and-hibernation)

`getDurableObjectWebsocketHooks({ server, onUpgrade })` wraps `getWebsocketHandlers` from `teleportal/websocket-server` for crossws’s Cloudflare adapter (`crossws/adapters/cloudflare`), which has two quirks this package accounts for:

* **Dropped upgrade context**: the durable upgrade path drops the context returned by the upgrade hook. The wrapper stashes it per-request and re-applies it to `peer.context` before `open` runs.
* **Hibernation wake-up**: the adapter uses the WebSocket Hibernation API. When an instance is evicted and later woken by a message, the peer’s in-memory state is gone. The websocket-server hooks detect this and close the socket so the client reconnects and resyncs automatically. Open SSE connections prevent hibernation entirely.

This package never imports `crossws/adapters/cloudflare` itself, since that module only resolves inside workerd – instantiate it in your Worker code and pass it to `getDurableObjectHandlers`.

## Local Development and Testing

[Section titled “Local Development and Testing”](#local-development-and-testing)

`wrangler dev` runs on `workerd` (the same runtime Cloudflare uses in production) fully offline – no Cloudflare account needed, and Durable Object storage is emulated locally:

```bash
cd examples/cloudflare
bun run dev
```

Open `http://localhost:8787` in two tabs and type – edits sync between tabs through the Durable Object.

Because `workerd` runs locally, you can write real integration tests that spawn `wrangler dev` as a subprocess and assert against actual sync behavior (WebSocket round-trips, SSE wire format, persistence across reconnects) instead of mocking the runtime. See [`examples/cloudflare/src/integration.test.ts`](https://github.com/nperez0111/teleportal/blob/main/examples/cloudflare/src/integration.test.ts) for the pattern.

## Scaling Beyond One Instance

[Section titled “Scaling Beyond One Instance”](#scaling-beyond-one-instance)

`idFromName("teleportal")` pins the whole app to one Durable Object. To shard, derive the instance name from the room (e.g. `idFromName(room)`) in both the Worker and any place that mints client URLs – every client of a room must land on the same instance.

## Scope and Caveats

[Section titled “Scope and Caveats”](#scope-and-caveats)

* **No built-in auth in the example**: the example wires a static `{ userId, room }` context. For real deployments, use `TokenManager` (`teleportal/token`) with `tokenAuthenticatedHTTPHandler` or a token check in `onUpgrade` – `jose` runs fine on workerd. See [Authentication](/docs/guides/authentication/).
* **One Durable Object instance per app by default** – see Scaling above to shard by room.
* **2 MiB value size limit** on SQLite-backed Durable Object storage. A document’s compacted state and a milestone snapshot are each one value; file chunks (1 MiB) always fit.
* **No `crossws/adapters/cloudflare` import inside `teleportal/cloudflare`** – that module only resolves inside workerd, so the library exposes hooks with type-only crossws imports and expects the consumer to instantiate the adapter.

## Full Example

[Section titled “Full Example”](#full-example)

See [`examples/cloudflare`](https://github.com/nperez0111/teleportal/tree/main/examples/cloudflare) for a complete, deployable example: wrangler config, static assets, a bundled ProseMirror browser client, and all storages wired into RPC handlers.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Authentication](/docs/guides/authentication/) - Add real token-based auth in place of the static example context
* [Custom Storage](/docs/guides/custom-storage/) - Implement additional storage backends
* [Scaling](/docs/advanced/scaling/) - Sharding and multi-instance strategies

# Custom Storage

> How to implement a custom storage backend for Teleportal

Teleportal decouples storage from compute. The `AbstractDocumentStorage` base class handles the entire sync protocol – update merging, state vectors, content-encryption envelopes, attribution – so your custom backend only needs to persist and retrieve data.

This guide walks through implementing document, file, and milestone storage from scratch. For a deeper look at the architecture and design decisions behind the storage layer, see [Advanced: Custom Storage](/docs/advanced/custom-storage/).

## What you will learn

[Section titled “What you will learn”](#what-you-will-learn)

* Extending `AbstractDocumentStorage` with all eight required persistence methods
* Understanding the merge-on-read design (pending log + base state)
* Calling `super(encrypted)` to support both encrypted and unencrypted documents
* Overriding `transaction()` for atomic operations
* Wiring custom storage into a Teleportal `Server`
* Implementing the `FileStorage` and `MilestoneStorage` interfaces

## Implementing DocumentStorage

[Section titled “Implementing DocumentStorage”](#implementing-documentstorage)

`AbstractDocumentStorage` uses a **merge-on-read** design. Updates are appended to a pending log on write (O(1)). Reads materialize the log by batch-merging all pending updates with the base state. Your subclass implements eight abstract persistence primitives – three for the pending log, two for the base state, two for metadata, and one for cleanup.

The example below uses a hypothetical PostgreSQL client.

```typescript
import {
  AbstractDocumentStorage,
  type DocumentState,
  type PendingUpdate,
  type DocumentMetadata,
} from "teleportal/storage";
import type { IndexedSidecar } from "teleportal/protocol/encryption";


export class PostgresDocumentStorage extends AbstractDocumentStorage {
  private pg: PgClient;


  constructor(pg: PgClient, encrypted: boolean = true) {
    super(encrypted);
    this.pg = pg;
  }


  // -- Pending log (merge-on-read) --


  async appendUpdate(key: string, entry: PendingUpdate): Promise<void> {
    await this.pg.query("INSERT INTO pending_updates (document_id, data) VALUES ($1, $2)", [
      key,
      JSON.stringify(entry),
    ]);
  }


  async getPendingUpdates(key: string): Promise<{ updates: PendingUpdate[]; cursor: number }> {
    const rows = await this.pg.query(
      "SELECT data FROM pending_updates WHERE document_id = $1 ORDER BY id",
      [key],
    );
    const updates = rows.map((r: any) => JSON.parse(r.data) as PendingUpdate);
    return { updates, cursor: updates.length };
  }


  async clearPendingUpdates(key: string, upToCursor: number): Promise<void> {
    // Delete the first `upToCursor` entries
    await this.pg.query(
      `DELETE FROM pending_updates WHERE id IN (
        SELECT id FROM pending_updates WHERE document_id = $1
        ORDER BY id LIMIT $2
      )`,
      [key, upToCursor],
    );
  }


  // -- Base (compacted) state --


  async getBaseState(key: string): Promise<DocumentState | null> {
    const row = await this.pg.query("SELECT update_data, sidecars FROM documents WHERE id = $1", [
      key,
    ]);
    if (!row) return null;
    return {
      update: new Uint8Array(row.update_data),
      sidecars: JSON.parse(row.sidecars) as IndexedSidecar[],
    };
  }


  async replaceBaseState(
    key: string,
    update: Uint8Array,
    sidecars: IndexedSidecar[],
  ): Promise<void> {
    await this.pg.query(
      `INSERT INTO documents (id, update_data, sidecars)
       VALUES ($1, $2, $3)
       ON CONFLICT (id) DO UPDATE
       SET update_data = $2, sidecars = $3`,
      [key, Buffer.from(update), JSON.stringify(sidecars)],
    );
  }


  // -- Metadata --


  async writeDocumentMetadata(key: string, metadata: DocumentMetadata): Promise<void> {
    await this.pg.query(
      `INSERT INTO document_metadata (id, data)
       VALUES ($1, $2)
       ON CONFLICT (id) DO UPDATE SET data = $2`,
      [key, JSON.stringify(metadata)],
    );
  }


  async getDocumentMetadata(key: string): Promise<DocumentMetadata> {
    const row = await this.pg.query("SELECT data FROM document_metadata WHERE id = $1", [key]);
    if (!row) {
      return {
        createdAt: Date.now(),
        updatedAt: Date.now(),
        encrypted: this.encrypted,
      };
    }
    return JSON.parse(row.data) as DocumentMetadata;
  }


  // -- Cleanup --


  async deleteDocument(key: string): Promise<void> {
    await this.pg.query("DELETE FROM pending_updates WHERE document_id = $1", [key]);
    await this.pg.query("DELETE FROM documents WHERE id = $1", [key]);
    await this.pg.query("DELETE FROM document_metadata WHERE id = $1", [key]);
  }
}
```

Key points:

* **`super(encrypted)`** passes the encryption flag to the base class. You never handle encryption yourself.
* **`appendUpdate`** appends to the pending log. The base class calls this from `handleUpdate` after decoding the content-encrypted envelope.
* **`getBaseState` / `replaceBaseState`** manage the last fully-merged snapshot. The base class materializes pending updates against this on read.
* **`getDocumentMetadata`** should return sensible defaults when no row exists yet.

## Overriding transaction()

[Section titled “Overriding transaction()”](#overriding-transaction)

The base class calls `transaction(key, cb)` to serialize concurrent writes. The default implementation applies no locking. Override it if your backend supports real transactions.

```typescript
async transaction<T>(key: string, cb: () => Promise<T>): Promise<T> {
  return await this.pg.transaction(async (tx) => {
    // Acquire an advisory lock scoped to this document
    await tx.query("SELECT pg_advisory_xact_lock(hashtext($1))", [key]);
    return await cb();
  });
}
```

Other backends can use distributed locks (Redis `SETNX`), optimistic concurrency (version columns), or sequential execution (in-memory queue). The interface is intentionally flexible.

## Wiring into the server

[Section titled “Wiring into the server”](#wiring-into-the-server)

The `Server` constructor accepts a `storage` function that receives a context object. Use `ctx.encrypted` to pass the session’s encryption mode to your storage.

```typescript
import { Server } from "teleportal/server";


const pg = createPgClient(/* connection config */);


const server = new Server({
  storage: async (ctx) => new PostgresDocumentStorage(pg, ctx.encrypted),
});
```

A fresh storage instance is created per session. Wire in `ctx.encrypted` so encrypted and unencrypted documents use the same class with different configuration.

## Implementing FileStorage

[Section titled “Implementing FileStorage”](#implementing-filestorage)

`FileStorage` manages binary file data (images, attachments). Implement three methods: `getFile`, `deleteFile`, and `storeFileFromUpload`.

```typescript
import type { FileStorage, File, FileUploadResult } from "teleportal/storage";


export class S3FileStorage implements FileStorage {
  readonly type = "file-storage" as const;
  private s3: S3Client;


  constructor(s3: S3Client) {
    this.s3 = s3;
  }


  async getFile(fileId: string): Promise<File | null> {
    const obj = await this.s3.getObject(`files/${fileId}`);
    return obj ? (obj as File) : null;
  }


  async deleteFile(fileId: string): Promise<void> {
    await this.s3.deleteObject(`files/${fileId}`);
  }


  async storeFileFromUpload(uploadResult: FileUploadResult): Promise<void> {
    for (let i = 0; i < uploadResult.totalChunks; i++) {
      const chunk = await uploadResult.getChunk(i);
      await this.s3.putObject(`files/${uploadResult.fileId}/chunk/${i}`, chunk);
    }
  }
}
```

Wire it into the server via RPC handlers:

```typescript
import { getFileRpcHandlers } from "teleportal/protocols/file";


const server = new Server({
  storage: async (ctx) => new PostgresDocumentStorage(pg, ctx.encrypted),
  rpcHandlers: { ...getFileRpcHandlers(new S3FileStorage(s3Client)) },
});
```

## Implementing MilestoneStorage

[Section titled “Implementing MilestoneStorage”](#implementing-milestonestorage)

`MilestoneStorage` stores named document snapshots. Every milestone requires a `createdBy` field: `{ type: "user", id }` or `{ type: "system", id }`.

```typescript
import type { MilestoneStorage } from "teleportal/storage";
import type { Milestone, MilestoneSnapshot } from "teleportal";


export class PostgresMilestoneStorage implements MilestoneStorage {
  readonly type = "milestone-storage" as const;
  private pg: PgClient;


  constructor(pg: PgClient) {
    this.pg = pg;
  }


  async createMilestone(ctx: {
    name: string;
    documentId: string;
    createdAt: number;
    snapshot: MilestoneSnapshot;
    createdBy: { type: "user" | "system"; id: string };
  }): Promise<string> {
    const id = crypto.randomUUID();
    await this.pg.query(
      `INSERT INTO milestones (id, document_id, name, created_at, snapshot, created_by)
       VALUES ($1, $2, $3, $4, $5, $6)`,
      [id, ctx.documentId, ctx.name, ctx.createdAt, ctx.snapshot, ctx.createdBy],
    );
    return id;
  }


  async getMilestones(documentId: string): Promise<Milestone[]> {
    return await this.pg.query(
      "SELECT * FROM milestones WHERE document_id = $1 AND deleted_at IS NULL",
      [documentId],
    );
  }


  // Also implement: getMilestone, deleteMilestone, restoreMilestone, updateMilestoneName
}
```

Wire it the same way as file storage:

```typescript
import { getMilestoneRpcHandlers } from "teleportal/protocols/milestone";


const server = new Server({
  storage: async (ctx) => new PostgresDocumentStorage(pg, ctx.encrypted),
  rpcHandlers: {
    ...getFileRpcHandlers(new S3FileStorage(s3Client)),
    ...getMilestoneRpcHandlers(new PostgresMilestoneStorage(pg)),
  },
});
```

## Next steps

[Section titled “Next steps”](#next-steps)

* [Advanced: Custom Storage](/docs/advanced/custom-storage/) – Architecture deep-dive and best practices for production storage backends
* [Persistent Storage](/docs/guides/persistent-storage/) – Full reference for all storage interfaces and provided implementations
* [Server](/docs/core-concepts/server/) – How storage fits into the server lifecycle

# Encryption at Rest

> Encrypting documents in storage for security

This guide covers **storage-level encryption at rest** – encrypting all data the server persists so it is protected even if someone gains direct access to your storage backend (Redis, S3, filesystem, database).

At-rest encryption vs content E2EE

Teleportal has two independent encryption layers. **Content-level E2EE** (on by default) protects document *content* from the server itself – the client encrypts before sending, and the server never sees the plaintext content or the client’s key. **At-rest encryption** (this guide) protects everything the *server* persists – including CRDT structure, metadata, and any plaintext documents – from unauthorized access to the storage backend. The two are independent and can be used together for defense in depth.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Encrypting data at the storage layer with a server-held key
* Wrapping an unstorage driver with `createEncryptedDriver` so all values are encrypted before they hit disk/Redis/S3
* Generating, importing, and rotating the server-side at-rest key
* Keeping at-rest encryption independent of client-side content E2EE
* Combining both layers for defense in depth

## How to encrypt storage at rest

[Section titled “How to encrypt storage at rest”](#how-to-encrypt-storage-at-rest)

At-rest encryption is a **driver-level** concern. `createEncryptedDriver` (from `teleportal/storage`) wraps any unstorage driver so every value is encrypted with a server-held AES-256-GCM key before it is written, and decrypted when read. It is built on a generic `TransformDriver`, and for each value it lib0-encodes, encrypts (via `encryptUpdate`, with a random 12-byte IV per write), then base64-stores the result.

Note

Meta keys whose name ends in `$` pass through the transform **untransformed** (unencrypted). This lets internal bookkeeping keys bypass encryption where needed.

### Complete example with Redis

[Section titled “Complete example with Redis”](#complete-example-with-redis)

```typescript
import { Server } from "teleportal/server";
import { UnstorageDocumentStorage, createEncryptedDriver } from "teleportal/storage";
import { generateEncryptionKey, importEncryptionKey } from "teleportal/encryption-key";
import { createStorage } from "unstorage";
import redisDriver from "unstorage/drivers/redis";


// --- 1. Import or create the server-side at-rest key (a raw CryptoKey) ---
const AT_REST_KEY = process.env.AT_REST_KEY
  ? await importEncryptionKey(process.env.AT_REST_KEY)
  : await generateEncryptionKey(); // only for dev -- persist this in production


// --- 2. Wrap the underlying driver with encryption ---
const encryptedStorage = createStorage({
  driver: createEncryptedDriver(
    redisDriver({
      base: "teleportal:",
      url: process.env.REDIS_URL ?? "redis://localhost:6379",
    }),
    AT_REST_KEY,
  ),
});


// --- 3. Use the encrypted storage with the server ---
const server = new Server({
  storage: async (ctx) => {
    return new UnstorageDocumentStorage(encryptedStorage, {
      keyPrefix: "doc",
      encrypted: ctx.encrypted,
    });
  },
});
```

Every value written to Redis is now AES-256-GCM encrypted with a unique IV. If an attacker accesses Redis directly, they see only ciphertext.

The second argument to `createEncryptedDriver` can be a `CryptoKey`, a `Promise<CryptoKey>`, or a `(key: string) => CryptoKey | Promise<CryptoKey>` function if you want to vary the key per storage key.

## Key management

[Section titled “Key management”](#key-management)

The at-rest key is a symmetric AES-256-GCM key that the **server** holds. It is completely separate from the client-side content E2EE key that clients pass to `Provider.create`.

### Generate, export, and load a key

[Section titled “Generate, export, and load a key”](#generate-export-and-load-a-key)

```typescript
import {
  generateEncryptionKey,
  exportEncryptionKey,
  importEncryptionKey,
} from "teleportal/encryption-key";


// Generate a new random key and export it as a JWK string for storage
const key = await generateEncryptionKey();
const keyString = await exportEncryptionKey(key);
// Store keyString in a secrets manager or environment variable -- treat it as a secret.


// In production, load from the environment
const atRestKey = await importEncryptionKey(process.env.AT_REST_KEY!);
```

### Key rotation

[Section titled “Key rotation”](#key-rotation)

To rotate the at-rest key, re-encrypt existing data with the new key:

1. Store the new key alongside the old one.
2. On reads, try the new key first; fall back to the old key if decryption fails.
3. On writes, always use the new key.
4. Run a background migration to re-encrypt all existing values, then remove the old key.

Tip

Rotating the at-rest key does not affect clients or require any client-side changes. It operates entirely at the server’s storage layer.

## The `ctx.encrypted` flag

[Section titled “The ctx.encrypted flag”](#the-ctxencrypted-flag)

The `encrypted` option on `UnstorageDocumentStorage` reflects the **content encryption mode** of the document – the client-side E2EE feature. It tags document metadata so the server knows whether the stored content sidecars are encrypted.

```typescript
const server = new Server({
  storage: async (ctx) => {
    return new UnstorageDocumentStorage(encryptedStorage, {
      keyPrefix: "doc",
      // ctx.encrypted is true when the client connected with an encryptionKey.
      // It only tags metadata -- it does NOT control at-rest encryption.
      encrypted: ctx.encrypted,
    });
  },
});
```

This flag is **unrelated** to the at-rest encryption key used by the storage wrapper. Wire `ctx.encrypted` through so metadata stays accurate regardless of your at-rest configuration.

## Combining with content E2EE

[Section titled “Combining with content E2EE”](#combining-with-content-e2ee)

For maximum security, use both layers together. No code changes are needed beyond what is shown above – content E2EE is on by default, and the encrypted driver is transparent to `UnstorageDocumentStorage`.

On the **client**, `Provider.create` encrypts content before it leaves the browser:

```typescript
import { Provider } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey: createEncryptionKey(), // never sent to server
});
```

On the **server**, use the `createEncryptedDriver` wrapper from the example above. With both layers active, document content is encrypted by the client before it leaves the browser, and then the server encrypts all persisted bytes (CRDT structure, metadata, encrypted sidecars) again before writing to storage. An attacker who compromises the storage backend sees only AES-256-GCM ciphertext. An attacker who compromises the server process sees encrypted content sidecars but never the plaintext content.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Persistent Storage](/docs/guides/persistent-storage/) – Storage backend configuration
* [Custom Storage](/docs/guides/custom-storage/) – Implement your own storage backend
* [Encryption Keys](/docs/guides/encryption-keys/) – Client-side key creation, sharing, and distribution

# Encryption Keys

> Create, share, and manage encryption keys for end-to-end encrypted documents

Content-level end-to-end encryption is on by default in Teleportal. Every `Provider` requires an `encryptionKey` – a `KeyResolver` produced by `createEncryptionKey()`, `passwordKey()`, or `registryKey()`. A resolver describes *how* the provider obtains its `CryptoKey`; `Provider.create` resolves it after the connection is ready but before the provider is built. The key never reaches the server; all encryption and decryption happens client-side using AES-256-GCM via the Web Crypto API.

This guide covers creating, sharing, storing, and distributing encryption keys.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Creating and exporting encryption keys
* Sharing keys via URL fragments
* Importing keys from strings
* Key wrapping for secure storage
* Three tiers of key distribution (direct, password, registry)

## Creating an Encryption Key

[Section titled “Creating an Encryption Key”](#creating-an-encryption-key)

`createEncryptionKey()` returns a `KeyResolver`. Called with no argument it delegates to `simpleEncryption()`, which derives the document’s AES-256-GCM key from the **document ID** via PBKDF2. Passing a password delegates to `passwordKey(password)` instead (see [Password-Derived Keys](#password-derived-keys)):

```typescript
import { createEncryptionKey } from "teleportal/encryption-key";
import { Provider } from "teleportal/providers";


const encryptionKey = createEncryptionKey();


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey,
});
```

Caution

Because `createEncryptionKey()` with no password derives the key from the document ID alone, anyone who knows the document ID can decrypt. Only use it when the ID itself is a secret (for example, an unguessable UUID). For an independent random key, use `generateEncryptionKey()` and distribute the exported key out-of-band.

Note

To deliberately run a plaintext document without encryption, pass `encryptionKey: false`. Omitting the key entirely throws an error.

## Exporting and Importing Keys

[Section titled “Exporting and Importing Keys”](#exporting-and-importing-keys)

`exportEncryptionKey` / `importEncryptionKey` operate on a raw `CryptoKey`, not a resolver. Generate a random extractable key with `generateEncryptionKey()`, export it to a JWK string for storage or transfer, then import it back later. An imported `CryptoKey` can be passed directly as the `encryptionKey`:

```typescript
import {
  generateEncryptionKey,
  exportEncryptionKey,
  importEncryptionKey,
} from "teleportal/encryption-key";


// Generate a random key and export it to a storable string
const key = await generateEncryptionKey();
const keyString = await exportEncryptionKey(key);


// Store the keyString in localStorage, a database, etc.
localStorage.setItem("doc-key", keyString);


// Import it back later (returns a CryptoKey, usable as an encryptionKey)
const stored = localStorage.getItem("doc-key");
const importedKey = await importEncryptionKey(stored);
```

## Sharing Keys via URL Fragments

[Section titled “Sharing Keys via URL Fragments”](#sharing-keys-via-url-fragments)

The safest way to share a key in a browser is via the URL fragment (the part after `#`). Browsers never send the fragment to the server, so the key stays client-side:

```typescript
import {
  generateEncryptionKey,
  exportEncryptionKey,
  importEncryptionKey,
  keyToUrlFragment,
  keyFromUrlFragment,
} from "teleportal/encryption-key";


// Sharer: create a shareable link
const key = await generateEncryptionKey();
const shareUrl = `https://app.example.com/doc/abc123#${keyToUrlFragment(await exportEncryptionKey(key))}`;
// e.g. https://app.example.com/doc/abc123#token=<key>


// Recipient: extract the key from the URL
const keyString = keyFromUrlFragment(location.hash); // string | null
if (keyString) {
  const sharedKey = await importEncryptionKey(keyString);
  // Use sharedKey with Provider.create
}
```

`keyFromUrlFragment` accepts the raw `location.hash` (with or without a leading `#`) and returns `null` when no `token` parameter is present.

## Password-Derived Keys

[Section titled “Password-Derived Keys”](#password-derived-keys)

For simple multi-user scenarios, derive a per-document key from a shared passphrase. No server involvement is needed – anyone with the passphrase can decrypt:

```typescript
import { passwordKey } from "teleportal/encryption-key";


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey: passwordKey("shared-passphrase"),
});
```

The `passwordKey` resolver uses PBKDF2 internally to derive a unique AES-256-GCM key for each document from the passphrase.

## Key Registry (Server-Managed Distribution)

[Section titled “Key Registry (Server-Managed Distribution)”](#key-registry-server-managed-distribution)

For multi-user applications that need per-user access control and key revocation, use the key registry protocol. The server stores wrapped (encrypted) copies of the document key for each user. Clients unwrap their copy locally – the server never sees the plaintext key.

### Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { Server } from "teleportal/server";
import {
  getKeyRegistryRpcHandlers,
  getKeyRegistryHandlers,
} from "teleportal/protocols/key-registry";


const keyRegistryStorage = new InMemoryKeyRegistryStorage();
const MASTER_SECRET = new TextEncoder().encode(process.env.MASTER_SECRET!);


const server = new Server({
  storage: async (ctx) => new MemoryDocumentStorage(ctx.encrypted),
  rpcHandlers: {
    ...getKeyRegistryRpcHandlers(keyRegistryStorage),
  },
});


// HTTP handlers for key management (called by your app server, not clients)
const keyHandlers = getKeyRegistryHandlers({
  storage: keyRegistryStorage,
  masterSecret: MASTER_SECRET,
});
```

### Client Setup

[Section titled “Client Setup”](#client-setup)

The `registryKey` resolver fetches the wrapped key via RPC and unwraps it locally using the wrapping key embedded in the user’s JWT. The unwrapped key is cached **per document** (a failed fetch or unwrap does not poison the cache). On key rotation, the key-registry extension invalidates only that document’s cached key, so the next access re-fetches the new wrapped key:

```typescript
import { Provider } from "teleportal/providers";
import { registryKey, importWrappingKey } from "teleportal/encryption-key";
import { createKeyRegistryRpc } from "teleportal/protocols/key-registry";


// wrappingKey comes from the JWT issued by your app server
const wrappingKey = await importWrappingKey(tokenPayload.wrappingKey);


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey: registryKey({ wrappingKey }),
  rpc: { keys: createKeyRegistryRpc },
});
```

## Choosing a Key Distribution Tier

[Section titled “Choosing a Key Distribution Tier”](#choosing-a-key-distribution-tier)

| Tier           | Mechanism                                     | Per-User Revocation | Server Involvement | Best For                            |
| -------------- | --------------------------------------------- | ------------------- | ------------------ | ----------------------------------- |
| **Direct key** | `encryptionKey: key` or URL fragment          | No                  | None               | Quick sharing, demos                |
| **Password**   | `encryptionKey: passwordKey("...")`           | No                  | None               | Shared workspaces, simple apps      |
| **Registry**   | `encryptionKey: registryKey({ wrappingKey })` | Yes                 | Full               | Multi-user apps with access control |

All three tiers use the same `encryptionKey` option on `Provider.create`. You can start with direct keys and migrate to the registry later without changing the rest of your code.

## Best Practices

[Section titled “Best Practices”](#best-practices)

* **Never send keys to your server.** Use URL fragments, out-of-band channels, or the key registry for distribution.
* **Store exported keys securely.** If you persist a key string in localStorage or a database, treat it as a secret.
* **Use the key registry for production multi-user apps.** It provides per-user revocation, key rotation, and audit via generation counters.
* **Rotate keys when revoking access.** Removing a user’s wrapped key prevents future access, but rotating the document key ensures they cannot decrypt new content even if they cached the old key.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [File Transfers](/docs/guides/file-transfers/) - Upload and download encrypted files
* [Authentication](/docs/guides/authentication/) - Token-based authentication setup
* [Custom RPC Methods](/docs/guides/rpc-extensions/) - Extend the protocol with custom operations

# Fallback Connection

> Automatic fallback from WebSocket to HTTP when WebSocket fails

This guide demonstrates a Teleportal server that supports both WebSocket and HTTP transports, with the client automatically falling back to HTTP if WebSocket connection fails.

This is the recommended way to set up a Teleportal server, as it provides the best compatibility with different client environments.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Setting up a Teleportal server that supports both WebSocket and HTTP transports
* Client-side automatic fallback from WebSocket to HTTP transport
* Using the `connectionType` property to determine which transport is active

## Server Setup

[Section titled “Server Setup”](#server-setup)

The server supports both WebSocket and HTTP:

```typescript
import { serve } from "crossws/server";
import { Server } from "teleportal/server";
import { getWebsocketHandlers } from "teleportal/websocket-server";
import { getHTTPHandlers } from "teleportal/http";


const server = new Server({
  storage: async (ctx) => {
    // Your storage implementation
    return documentStorage;
  },
});


// WebSocket handlers
const wsHandlers = getWebsocketHandlers({
  server,
  onUpgrade: async () => {
    return {
      context: { userId: "user-123" },
    };
  },
});


// HTTP handlers
const httpHandlers = getHTTPHandlers({
  server,
  getContext: async (request) => {
    return { userId: "user-123" };
  },
});


serve({
  websocket: wsHandlers,
  fetch: httpHandlers,
});
```

## Client Setup

[Section titled “Client Setup”](#client-setup)

The `Provider` automatically uses fallback connection:

```typescript
import { Provider } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


// Provider automatically tries WebSocket first, falls back to HTTP
const provider = await Provider.create({
  url: "wss://example.com", // Tries WebSocket first
  document: "my-document",
  encryptionKey: createEncryptionKey(),
});


await provider.synced;
```

## How It Works

[Section titled “How It Works”](#how-it-works)

1. **Initial Attempt**: Provider tries to connect via WebSocket
2. **Fallback**: If WebSocket fails, automatically falls back to HTTP/SSE
3. **Reconnection**: Provider continues to attempt WebSocket in the background
4. **Upgrade**: If WebSocket becomes available, connection is upgraded

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [WebSocket Only](/docs/guides/websocket-only/) - WebSocket-only setup
* [HTTP Transport](/docs/guides/http-transport/) - HTTP-only setup

# File Transfers

> Upload and download files with Merkle tree integrity verification

Teleportal supports file uploads and downloads alongside document sync, using chunked streaming with Merkle tree integrity verification. Files are transferred as chunks (default 1MB) and can be end-to-end encrypted with the same key used for document content.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Setting up FileStorage and TemporaryUploadStorage on the server
* Registering file RPC handlers
* Uploading and downloading files from the client
* Integrity verification via Merkle proofs
* Encrypted file transfers
* Persistent chunk caching, retransmission, and progress events

## Server Setup

[Section titled “Server Setup”](#server-setup)

File transfers require a `FileStorage` and a `TemporaryUploadStorage` (for in-progress uploads). Register the file RPC handlers with `getFileRpcHandlers()`:

```typescript
import { Server } from "teleportal/server";
import {
  MemoryDocumentStorage,
  InMemoryFileStorage,
  InMemoryTemporaryUploadStorage,
} from "teleportal/storage";
import { getFileRpcHandlers } from "teleportal/protocols/file";


const fileStorage = new InMemoryFileStorage();
fileStorage.temporaryUploadStorage = new InMemoryTemporaryUploadStorage();


const server = new Server({
  storage: async (ctx) => new MemoryDocumentStorage(ctx.encrypted),
  rpcHandlers: {
    ...getFileRpcHandlers(fileStorage),
  },
});
```

### Configuring Chunk Size

[Section titled “Configuring Chunk Size”](#configuring-chunk-size)

The server controls the wire chunk size and communicates it to clients during upload initialization. The default is 1MB; adjust it to balance throughput and memory:

```typescript
const server = new Server({
  storage: async (ctx) => new MemoryDocumentStorage(ctx.encrypted),
  rpcHandlers: {
    ...getFileRpcHandlers(fileStorage, { chunkSize: 256 * 1024 }), // 256KB chunks
  },
});
```

### Permission Checking

[Section titled “Permission Checking”](#permission-checking)

Add upload and download permission checks to control who can transfer files:

```typescript
import { getFileRpcHandlers, type FileHandlerOptions } from "teleportal/protocols/file";


const options: FileHandlerOptions = {
  async checkUploadPermission(fileId, metadata, context) {
    const allowed = await checkAccess(context.userId, context.documentId, "write");
    return allowed ? { allowed: true } : { allowed: false, reason: "No write access" };
  },
  async checkDownloadPermission(fileId, context) {
    const allowed = await checkAccess(context.userId, context.documentId, "read");
    return allowed ? { allowed: true } : { allowed: false, reason: "No read access" };
  },
};


const server = new Server({
  storage: async (ctx) => new MemoryDocumentStorage(ctx.encrypted),
  rpcHandlers: {
    ...getFileRpcHandlers(fileStorage, options),
  },
});
```

## Client Setup

[Section titled “Client Setup”](#client-setup)

Register the file RPC extension on the provider. When encryption is enabled (the default), pass the same `encryptionKey` to `createFileRpc` so files are encrypted with the same key as the document:

```typescript
import { Provider } from "teleportal/providers";
import { createFileRpc } from "teleportal/protocols/file";
import { createEncryptionKey } from "teleportal/encryption-key";


const encryptionKey = createEncryptionKey();


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey,
  rpc: {
    file: () => createFileRpc({ encryptionKey }),
  },
});
```

### Uploading a File

[Section titled “Uploading a File”](#uploading-a-file)

Pass a `File` object (from an `<input>` element or the File API) to `upload()`:

```typescript
const input = document.querySelector<HTMLInputElement>("#file-input");
input.addEventListener("change", async () => {
  const file = input.files?.[0];
  if (!file) return;


  const fileId = await provider.rpc.file.upload(file);
  console.log("Uploaded:", fileId);
});
```

### Downloading a File

[Section titled “Downloading a File”](#downloading-a-file)

Retrieve a file by its ID. `download()` returns a native `File` object (decrypted client-side when encryption is enabled):

```typescript
const file = await provider.rpc.file.download(fileId);
// file is a File: file.name, file.size, file.type, await file.arrayBuffer(), etc.
```

`download()` accepts an optional second argument to override the default per-operation behavior – `{ timeout }` (download timeout in ms, default `60000`), `{ encryptionKey }` to override the extension’s key, and `{ cache: false }` to bypass the persistent cache:

```typescript
const file = await provider.rpc.file.download(fileId, { timeout: 10_000, cache: false });
```

### Error Handling

[Section titled “Error Handling”](#error-handling)

File operations throw `RpcOperationError` on failure:

```typescript
import { RpcOperationError } from "teleportal/rpc";


try {
  await provider.rpc.file.upload(myFile);
} catch (error) {
  if (error instanceof RpcOperationError) {
    console.error(error.protocol); // "file"
    console.error(error.operation); // "upload"
  }
}
```

## How It Works

[Section titled “How It Works”](#how-it-works)

### Chunked Transfer

[Section titled “Chunked Transfer”](#chunked-transfer)

Files are split into chunks (sized by the server, default 1MB) and transferred over the RPC layer as an ordered sequence of chunk messages. Chunking lets each chunk be encrypted, hashed, verified, and acknowledged independently, and makes uploads resumable and content-addressed.

**Upload flow:** The client encrypts the whole file and folds its Merkle root up front — the root is the content ID, which lets the server answer “already have it” (dedup) or “here are the chunks I’m missing” (resume) in a single round-trip. The client then streams the missing chunks (pipelined, not strictly one-ACK-at-a-time) and resolves once every sent chunk is acknowledged. After the last chunk lands, the server rebuilds the Merkle tree, verifies it against the claimed content ID, and moves the file from temporary to durable storage.

**Download flow:** The client sends a `fileDownload` request. The server streams chunks back, each with a Merkle proof. The client verifies each chunk against the root hash before accepting it, then assembles the file once all chunks arrive.

Caution

Transfer today is *pipelined*, not constant-memory: the client holds the file’s chunks in memory for the duration of an upload (freed incrementally as chunks are acknowledged) or a download (until the file is assembled), and the server loads a file’s chunks from storage to serve a download. Plan for roughly one file’s worth of memory per concurrent transfer, and keep files within the 1GB limit below. Fully bounded-memory streaming (a `ReadableStream`/`WritableStream` end-to-end path) is not yet implemented.

### Merkle Tree Verification

[Section titled “Merkle Tree Verification”](#merkle-tree-verification)

Every file is content-addressed using a Merkle tree built from its chunks. The tree hashes the **ciphertext** chunks (for encrypted files), so on download the client verifies each chunk’s Merkle proof against the root hash **before decrypting it** – verification and decryption run concurrently, but a chunk that fails its proof is rejected without its plaintext ever being trusted. This guarantees that no chunk has been tampered with or corrupted in transit or at rest.

On upload, the server rebuilds the Merkle tree from the received chunks and derives the authoritative content ID (the upload stream itself omits per-chunk proofs – the server recomputes and verifies the root at completion). The download path is the integrity boundary where per-chunk proofs are checked.

Tip

The maximum supported file size is 1GB. For encrypted files, each plaintext chunk is slightly smaller than the configured chunk size due to AES-GCM overhead (28 bytes per chunk).

### Encrypted File Transfers

[Section titled “Encrypted File Transfers”](#encrypted-file-transfers)

When you pass an `encryptionKey` to `createFileRpc`, files are encrypted client-side before upload and decrypted client-side after download, using the same AES-256-GCM key that protects the document. The server never sees plaintext file content.

### Caching

[Section titled “Caching”](#caching)

Pass a `FileCache` (such as the IndexedDB-backed `IdbFileCache`) to persist chunks across reloads:

```typescript
import { createFileRpc } from "teleportal/protocols/file";
import { IdbFileCache } from "teleportal/storage";


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey,
  rpc: {
    file: () => createFileRpc({ encryptionKey, cache: new IdbFileCache() }),
  },
});
```

Uploads are cached optimistically (keyed by content ID) as soon as the server accepts the request. Downloads are served straight from the cache when every chunk is present – with no server round-trip – while a partial cache entry falls through to the server. Chunks are cached in their verified ciphertext form (before decryption). Pass `{ cache: false }` to `upload()` or `download()` to bypass the cache for a single operation.

### Retransmission

[Section titled “Retransmission”](#retransmission)

Uploads are ACK-driven: the client resolves once every streamed chunk is positively acknowledged. The server can **nack** a chunk (ACK with `retryAfter`) to shed load, and the client runs a single background retransmit loop with exponential backoff (starting at `max(retryAfter, 200)`ms, doubling up to 10s, for up to 8 rounds), resending only still-unacked chunks. File chunks also get their own rate-limit budget on the server (they are recognized as file-transfer messages).

### Progress Events

[Section titled “Progress Events”](#progress-events)

Subscribe to lightweight, numbers-only progress snapshots via `onFileTransferProgress`:

```typescript
import { onFileTransferProgress } from "teleportal/protocols/file";


const unsubscribe = onFileTransferProgress((p) => {
  // p: { fileId, document, direction, chunksTransferred, totalChunks?, bytesTransferred, status, error? }
  console.log(`${p.direction} ${p.fileId}: ${p.chunksTransferred}/${p.totalChunks} (${p.status})`);
});
```

`status` is `"active"`, `"complete"`, or `"error"`; `bytesTransferred` is an approximate plaintext byte count.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Encryption Keys](/docs/guides/encryption-keys/) - Key creation, sharing, and management
* [Custom RPC Methods](/docs/guides/rpc-extensions/) - Add your own RPC operations
* [Custom Storage](/docs/guides/custom-storage/) - Implement FileStorage for S3, PostgreSQL, etc.

# HTTP Transport

> Using HTTP/SSE transport for environments where WebSockets are blocked

HTTP/SSE is the right transport when your deployment environment prevents WebSocket upgrades. This happens more often than you might expect: corporate proxies and firewalls that strip `Upgrade` headers, load balancers configured without WebSocket passthrough, edge runtimes like Cloudflare Workers (without Durable Objects), and restrictive CSP policies. In these scenarios, Teleportal uses standard HTTP POST requests paired with Server-Sent Events (SSE) to provide the same real-time synchronization you get with WebSockets.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Setting up a Teleportal server with HTTP-only transport using `getHTTPHandlers`
* Extracting authentication context from HTTP headers
* Forcing the client to use HTTP/SSE instead of the default WebSocket-first fallback
* The SSE + POST architecture that enables bidirectional communication over plain HTTP
* Using the stateless `POST /message` endpoint for one-off operations

## Server Setup

[Section titled “Server Setup”](#server-setup)

`getHTTPHandlers` creates a unified fetch handler that routes to all three HTTP endpoints. It works with any Fetch API-compatible runtime (Bun, Node 18+, Cloudflare Workers, Deno).

```typescript
import { Server } from "teleportal/server";
import { getHTTPHandlers } from "teleportal/http";
import { MemoryDocumentStorage } from "teleportal/storage";


const server = new Server({
  storage: new MemoryDocumentStorage(),
});


const handler = getHTTPHandlers({
  server,
  getContext: async (request) => {
    // Extract user identity from the request.
    // In production, verify a JWT or session cookie here.
    const userId = request.headers.get("x-user-id") ?? "anonymous";
    return { userId };
  },
});


// Bun
Bun.serve({ fetch: handler });


// Cloudflare Workers
// export default { fetch: handler };


// Node.js (with a Fetch API adapter)
// app.all("*", (req, res) => handler(req).then(r => res.send(r)));
```

Documents are encrypted by default. If a client connects to a document named `my-doc`, the server opens an encrypted session. To opt a specific document into plaintext, append `:plaintext` to the document name in the query string: `?documents=my-doc:plaintext`.

## Client Setup

[Section titled “Client Setup”](#client-setup)

By default, `Provider.create` uses a `DirectConnection` with `[websocketTransport(), httpTransport()]` that tries WebSocket first and falls back to HTTP. To force HTTP-only, pass `transports: [httpTransport()]`:

```typescript
import { Provider, httpTransport } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


const provider = await Provider.create({
  url: "https://example.com",
  document: "my-document",
  encryptionKey: createEncryptionKey(),
  // Force HTTP/SSE only -- skip the WebSocket attempt entirely
  transports: [httpTransport()],
});


await provider.synced;


// The Yjs document is ready to use
const text = provider.doc.getText("content");
text.insert(0, "Hello from HTTP/SSE!");
```

The `httpTransport` function accepts options for customizing the SSE connection timeout, providing a custom `fetch` or `EventSource` implementation, and tuning HTTP message batching:

```typescript
import { httpTransport } from "teleportal/providers";


const transport = httpTransport({
  timeout: 15000, // SSE connection timeout (default: 10000ms)
  fetch: customFetch, // Custom fetch implementation
  EventSource: customSSE, // Custom EventSource implementation
  httpBatchingOptions: {
    maxBatchSize: 10, // Max messages per HTTP POST
    maxBatchDelay: 50, // Max ms to wait before flushing a batch
  },
});
```

## How It Works

[Section titled “How It Works”](#how-it-works)

The HTTP transport uses two complementary channels to achieve bidirectional communication over plain HTTP:

### Server-to-client: GET /sse

[Section titled “Server-to-client: GET /sse”](#server-to-client-get-sse)

The client opens a long-lived SSE connection by sending `GET /sse`. The server subscribes to a PubSub topic keyed by the client ID (`client/{clientId}`) and streams messages to the client as SSE events. This connection stays open for the lifetime of the session.

### Client-to-server: POST /sse

[Section titled “Client-to-server: POST /sse”](#client-to-server-post-sse)

When the client needs to send data (document updates, awareness changes), it sends an `HTTP POST` to `/sse` with the `x-teleportal-client-id` header. The server publishes these messages to the same PubSub topic, where the SSE reader picks them up, processes them through the server, and streams any responses back over the open SSE connection.

### Stateless alternative: POST /message

[Section titled “Stateless alternative: POST /message”](#stateless-alternative-post-message)

For one-off operations that do not need a persistent connection, `POST /message` provides a simple request-response pattern. Each request gets a fresh client ID, processes messages, and returns the response inline. This is useful for operations like fetching a document snapshot without maintaining a long-lived connection.

### ACK system

[Section titled “ACK system”](#ack-system)

The SSE writer endpoint uses an acknowledgment system to ensure reliable delivery. After the SSE reader delivers messages to the client, it publishes an ACK to `ack/{clientId}`. The writer waits for this ACK before returning a success response. If the ACK does not arrive within the configured timeout (default: 5 seconds), the writer returns a `504 Gateway Timeout`.

### Performance trade-offs

[Section titled “Performance trade-offs”](#performance-trade-offs)

The server-to-client SSE channel encodes binary data as base64 (SSE is a text protocol), adding \~33% overhead compared to WebSocket’s native binary frames. The client-to-server HTTP POST channel sends raw binary with no encoding overhead. Message batching (multiple messages per POST) mitigates the per-request overhead of HTTP, but for high-throughput scenarios with frequent small updates, WebSocket remains the better choice. Use HTTP when compatibility is the priority.

## Dual Transport Setup

[Section titled “Dual Transport Setup”](#dual-transport-setup)

In most production deployments, you want to support both WebSocket and HTTP so that clients can use the best available transport. Set up both on the same server and let the client’s `DirectConnection` handle transport selection automatically.

```typescript
import { Server } from "teleportal/server";
import { getHTTPHandlers } from "teleportal/http";
import { getWebsocketHandlers } from "teleportal/websocket-server";
import { MemoryDocumentStorage } from "teleportal/storage";


const server = new Server({
  storage: new MemoryDocumentStorage(),
});


const getContext = async (request: Request) => {
  const userId = request.headers.get("x-user-id") ?? "anonymous";
  return { userId };
};


// HTTP/SSE handler (also serves as the fetch handler for non-WS requests)
const httpHandler = getHTTPHandlers({ server, getContext });


// WebSocket handler
const wsHandler = getWebsocketHandlers({
  server,
  onUpgrade: async (request) => {
    const userId = request.headers.get("x-user-id") ?? "anonymous";
    return { context: { userId } };
  },
});


Bun.serve({
  websocket: wsHandler,
  fetch: httpHandler,
});
```

The client does not need any changes – `Provider.create` tries WebSocket first and falls back to HTTP automatically:

```typescript
import { Provider } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


// Tries WebSocket, falls back to HTTP if it fails
const provider = await Provider.create({
  url: "https://example.com",
  document: "my-document",
  encryptionKey: createEncryptionKey(),
});
```

See the [Fallback Connection](/docs/guides/fallback-connection/) guide for more details on how the automatic fallback works, including upgrade probing and transport switching.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Fallback Connection](/docs/guides/fallback-connection/) – Automatic WebSocket-to-HTTP fallback with upgrade probing
* [WebSocket Only](/docs/guides/websocket-only/) – Minimal server setup when you only need WebSocket
* [Authentication](/docs/guides/authentication/) – Add token-based authentication to your server

# Observability

> Monitoring, structured logging, and status endpoints with Prometheus metrics and LogTape

This guide demonstrates the built-in observability capabilities provided by Teleportal, including Prometheus metrics endpoints, structured logging with LogTape, and server event hooks for custom monitoring.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Accessing the `/health` endpoint for server health checks
* Accessing the `/metrics` endpoint for Prometheus-compatible metrics
* Accessing the `/status` endpoint for server status information
* Configuring structured logging with LogTape
* Listening to server events for custom monitoring workflows

## Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { Server } from "teleportal/server";
import { getMetricsHandler, getHealthHandler, getStatusHandler } from "teleportal/http";


const server = new Server({
  storage: async (ctx) => {
    // Your storage implementation
    return documentStorage;
  },
});


// Expose observability endpoints
app.get("/metrics", getMetricsHandler(server));
app.get("/health", getHealthHandler(server));
app.get("/status", getStatusHandler(server));
```

## Metrics Endpoint

[Section titled “Metrics Endpoint”](#metrics-endpoint)

The `/metrics` endpoint returns Prometheus-formatted metrics:

```bash
curl http://localhost:3000/metrics
```

Metrics include:

* `teleportal_sessions_active`: Current number of active document sessions (gauge)
* `teleportal_clients_active`: Current number of active clients (gauge)
* `teleportal_documents_opened_total`: Total documents opened (counter)
* `teleportal_messages_total`: Total messages processed, labeled by `type` (counter)
* `teleportal_messages_total_all`: Total messages processed across all types, single series (counter)
* `teleportal_message_duration_seconds`: Message processing duration, labeled by `type` (histogram)

Note

`clients_active` and `sessions_active` are **signed gauges** – they reflect live inc/dec state and are not clamped, so a transient negative value is a genuine diagnostic signal of an inc/dec imbalance rather than an error to hide.

Metric cardinality

Several metrics are labeled by `documentId` and/or `userId` (for example `teleportal_document_size_bytes` and the `rate_limit_*` counters) and are never evicted. Over a long-lived process with many distinct documents or users, the series set grows unboundedly. Keep an eye on cardinality; there is currently no automatic eviction of per-document/per-user series.

## Health Endpoint

[Section titled “Health Endpoint”](#health-endpoint)

The `/health` endpoint returns server health status:

```bash
curl http://localhost:3000/health
```

Response:

```json
{
  "status": "healthy",
  "timestamp": "2024-01-01T00:00:00.000Z",
  "checks": {},
  "uptime": 3600
}
```

The endpoint always returns `status: "healthy"` with an empty `checks` object and the process uptime in seconds. The `monitoring` module defines only the `HealthStatus` and `StatusData` **types** – it does not implement any dependency health probes. `checks` is a placeholder you can populate by extending the server if you need liveness signals for downstream dependencies (storage, PubSub, etc.).

## Status Endpoint

[Section titled “Status Endpoint”](#status-endpoint)

The `/status` endpoint returns detailed operational status:

```bash
curl http://localhost:3000/status
```

Response:

```json
{
  "nodeId": "node-123",
  "activeClients": 10,
  "activeSessions": 5,
  "pendingSessions": 0,
  "totalMessagesProcessed": 1000,
  "totalDocumentsOpened": 50,
  "messageTypeBreakdown": {
    "doc": 500,
    "awareness": 300,
    "rpc": 200
  },
  "rateLimitExceededTotal": 0,
  "rateLimitBreakdown": {},
  "rateLimitTopOffenders": [],
  "rateLimitRecentEvents": [],
  "totalDocumentSizeBytes": 1048576,
  "documentsOverWarningThreshold": 0,
  "documentsOverLimit": 0,
  "uptime": 3600,
  "timestamp": "2024-01-01T00:00:00.000Z"
}
```

The `messageTypeBreakdown` is keyed by wire message type (`doc`, `awareness`, `presence`, `rpc`, `ack`) – milestone and file operations are carried over `rpc`, not distinct message types.

## Structured Logging with LogTape

[Section titled “Structured Logging with LogTape”](#structured-logging-with-logtape)

Teleportal uses [LogTape](https://logtape.org/) for structured logging. Following LogTape’s library guidelines, Teleportal does **not** configure any sinks itself – your application must set up LogTape before starting the server.

### Configuring LogTape

[Section titled “Configuring LogTape”](#configuring-logtape)

```typescript
import { configure, getConsoleSink, getFileSink } from "@logtape/logtape";


await configure({
  sinks: {
    console: getConsoleSink(),
    file: getFileSink("teleportal.log"),
  },
  filters: {},
  loggers: [
    {
      category: ["teleportal"],
      sinks: ["console", "file"],
      lowestLevel: "info",
    },
  ],
});
```

### Log Categories

[Section titled “Log Categories”](#log-categories)

Teleportal emits all of its logs under a single category:

* `["teleportal", "server"]` – every wide event (one structured log line per logical operation): connections, errors, message processing, and session lifecycle. Each wide event carries the environment context `{ service: "teleportal" }` merged in.

Because LogTape uses hierarchical categories, configuring the `["teleportal"]` category (as shown above) captures these logs. Per LogTape’s library guidelines, Teleportal configures no sinks of its own – your application owns sink and level configuration.

### What gets logged

[Section titled “What gets logged”](#what-gets-logged)

The server emits **wide events** – single structured log lines that capture the full context of an operation. Each wide event includes environment context (service name, node ID) merged with operation-specific fields such as document IDs, client IDs, message types, error details, and timing information.

## Server Events

[Section titled “Server Events”](#server-events)

The `Server` extends `Observable` and emits typed events you can listen to for custom monitoring, alerting, or integration with external systems.

### Listening to Events

[Section titled “Listening to Events”](#listening-to-events)

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  /* ... */
});


server.on("document-load", (data) => {
  console.log("Document loaded:", data.documentId);
  console.log("Encrypted:", data.encrypted);
});


server.on("client-connect", (data) => {
  console.log("Client connected:", data.clientId);
});


server.on("client-disconnect", (data) => {
  console.log("Client disconnected:", data.clientId);
  console.log("Reason:", data.reason);
});


server.on("document-size-warning", (data) => {
  // Trigger an alert when a document grows too large
  alerting.warn(`Document ${data.documentId} is ${data.sizeBytes} bytes`);
});
```

### Available Events

[Section titled “Available Events”](#available-events)

**Document events:** `document-load`, `document-unload`, `milestone-created`, `milestone-deleted`, `milestone-restored`, `document-delete`, `document-size-warning`, `document-size-limit-exceeded`

**Client events:** `client-connect`, `client-disconnect`, `client-message`

**Lifecycle events:** `before-server-shutdown`, `after-server-shutdown`

See the [Server documentation](/docs/core-concepts/server/) for the full event list and payload types.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Server](/docs/core-concepts/server/) - Server configuration, events, and monitoring details

# Persistent Storage

> Using persistent storage backends instead of in-memory storage

This guide demonstrates using persistent storage instead of in-memory storage. Documents are persisted across server restarts using the unstorage abstraction layer.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Using `UnstorageDocumentStorage` for persistent document storage
* Configuring unstorage with different drivers (memory, SQLite, Redis, etc.)
* Replacing in-memory storage with a persistent storage backend

## Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { createStorage } from "unstorage";
import { UnstorageDocumentStorage } from "teleportal/storage";
import redisDriver from "unstorage/drivers/redis";
import { Server } from "teleportal/server";


// Create unstorage instance with Redis driver
const storage = createStorage({
  driver: redisDriver({
    base: "teleportal:",
    url: "redis://localhost:6379",
  }),
});


const server = new Server({
  storage: async (ctx) => {
    return new UnstorageDocumentStorage(storage, {
      keyPrefix: "doc",
      encrypted: ctx.encrypted,
    });
  },
});
```

## First-Party Storage Adapters

[Section titled “First-Party Storage Adapters”](#first-party-storage-adapters)

Teleportal ships first-party storage adapters for PostgreSQL and S3 that extend `AbstractDocumentStorage` directly, offering better performance and tighter integration than generic unstorage drivers.

### PostgreSQL (first-party)

[Section titled “PostgreSQL (first-party)”](#postgresql-first-party)

```typescript
import postgres from "postgres";
import { PostgresDocumentStorage, ensureSchema } from "teleportal/storage/postgres";


const sql = postgres("postgres://localhost/mydb", { max: 10 });
await ensureSchema(sql); // create tables once at startup


const server = new Server({
  storage: async (ctx) => {
    return new PostgresDocumentStorage(sql, { encrypted: ctx.encrypted });
  },
});
```

### S3 (first-party, for file storage)

[Section titled “S3 (first-party, for file storage)”](#s3-first-party-for-file-storage)

`S3FileStorage` and `S3TemporaryUploadStorage` each take an `S3Config` (or a shared `S3Http` client) plus options. Pass the temporary upload store to the file store via the `temporaryUploadStorage` option. Works with AWS S3, Cloudflare R2, and MinIO:

```typescript
import { S3FileStorage, S3Http, S3TemporaryUploadStorage } from "teleportal/storage/s3";


const s3 = new S3Http({
  endpoint: "https://<account>.r2.cloudflarestorage.com",
  bucket: "my-bucket",
  region: "auto",
  accessKeyId: process.env.S3_ACCESS_KEY_ID!,
  secretAccessKey: process.env.S3_SECRET_ACCESS_KEY!,
});


const temp = new S3TemporaryUploadStorage(s3);
const fileStorage = new S3FileStorage(s3, { temporaryUploadStorage: temp });
```

## Alternative Storage Backends (via unstorage)

[Section titled “Alternative Storage Backends (via unstorage)”](#alternative-storage-backends-via-unstorage)

For other backends, use the unstorage abstraction layer with any of its drivers.

### SQLite

[Section titled “SQLite”](#sqlite)

```typescript
import sqliteDriver from "unstorage/drivers/sqlite";


const storage = createStorage({
  driver: sqliteDriver({
    db: "./teleportal.db",
  }),
});
```

### S3 (via unstorage)

[Section titled “S3 (via unstorage)”](#s3-via-unstorage)

```typescript
import s3Driver from "unstorage/drivers/s3";


const storage = createStorage({
  driver: s3Driver({
    bucket: "my-bucket",
    region: "us-east-1",
  }),
});
```

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Encryption at Rest](/docs/guides/encryption-at-rest/) - Encrypt documents in storage
* [Custom Storage](/docs/guides/custom-storage/) - Implement your own storage backend

# Pub/Sub

> Multi-server setup with pub/sub for horizontal scaling

This guide demonstrates a multi-server setup using pub/sub for horizontal scaling. Multiple server instances share a pub/sub backend to synchronize document updates across nodes.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

In a typical websocket-only setup, each client is connected via the websocket to a single server. But, what happens if you have two clients who access the same document? In a multi-node setup, it is possible for one client to be connected to one server and the other client to be connected to another server. This can lead to inconsistent state and conflicts, since the system is operating in a sort of split-brain scenario.

This example demonstrates how each server instance can coordinate with each other over a pub/sub interface to ensure that all clients see the same state. Specifically achieving the following:

* Each server instance has a unique `nodeId`
* Messages are published to PubSub topics
* Each server filters out its own messages
* Document updates are replicated across all server instances

## Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { Server } from "teleportal/server";
import { RedisPubSub } from "teleportal/transports/redis";
import { UnstorageDocumentStorage } from "teleportal/storage";


const server = new Server({
  storage: async (ctx) => {
    // Shared storage backend
    return new UnstorageDocumentStorage(storage, {
      keyPrefix: "doc",
      encrypted: ctx.encrypted,
    });
  },
  pubSub: new RedisPubSub({
    path: "redis://localhost:6379",
  }),
  nodeId: process.env.NODE_ID || `node-${uuidv4()}`,
});
```

## Running Multiple Instances

[Section titled “Running Multiple Instances”](#running-multiple-instances)

```bash
# Instance 1
NODE_ID=node-1 PORT=3000 node server.js


# Instance 2
NODE_ID=node-2 PORT=3001 node server.js


# Instance 3
NODE_ID=node-3 PORT=3002 node server.js
```

## How It Works

[Section titled “How It Works”](#how-it-works)

Anytime that a document update (or other broadcast message) is applied to a server, the server will publish that message to the PubSub topic to broadcast to all other servers that may be listening. Each server will then apply/broadcast that message to its own clients.

In this example, we are using Redis as the PubSub backend, but you can use any PubSub backend that implements the PubSub interface (literally publish/subscribe with callbacks).

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Scaling](/docs/advanced/scaling/) - Learn more about scaling strategies
* [Server](/docs/core-concepts/server/) - Server architecture

# Rate Limiting

> Rate limiting client messages to prevent abuse

This guide demonstrates rate limiting client messages to prevent abuse. The server can track rate limits per user, per document, or per user-document pair.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Configuring rate limiting on the server
* Tracking rate limits by user, document, or user-document pair
* Using persistent rate limit storage for multi-node deployments
* Handling rate limit exceeded events

## Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { Server } from "teleportal/server";
import { createStorage } from "unstorage";
import { UnstorageDocumentStorage, UnstorageRateLimitStorage } from "teleportal/storage";


const storage = createStorage();


const server = new Server({
  storage: new UnstorageDocumentStorage(storage),
  rateLimitConfig: {
    // Multiple rate limit rules
    rules: [
      // Track by user (across all documents)
      {
        id: "per-user",
        maxMessages: 100, // 100 messages per window
        windowMs: 1000, // 1 second window
        trackBy: "user",
      },
      // Track by document (across all users)
      {
        id: "per-document",
        maxMessages: 500, // 500 messages per window per document
        windowMs: 10000, // 10 second window
        trackBy: "document",
      },
      // Track by user-document pair
      {
        id: "user-document",
        maxMessages: 100,
        windowMs: 1000,
        trackBy: "user-document",
      },
    ],
    // Use persistent storage for multi-node deployments
    rateLimitStorage: new UnstorageRateLimitStorage(storage),
    // Callback when rate limit is exceeded
    onRateLimitExceeded: (details) => {
      console.warn("Rate limit exceeded", details);
    },
    // Maximum message size
    maxMessageSize: 10 * 1024 * 1024, // 10MB
    // Callback when message size is exceeded
    onMessageSizeExceeded: (details) => {
      console.warn("Message size exceeded", details);
    },
  },
});
```

## Rate Limit Tracking Modes

[Section titled “Rate Limit Tracking Modes”](#rate-limit-tracking-modes)

* **`"user"`**: Track rate limits per user ID. All connections from the same user share the same limit.
* **`"document"`**: Track rate limits per document ID. All users editing the same document share the same limit.
* **`"user-document"`**: Track rate limits per user-document pair. Each user has separate limits for each document.
* **`"transport"`**: Track rate limits per transport instance (in-memory only, not shared).

## How the Token Bucket Works

[Section titled “How the Token Bucket Works”](#how-the-token-bucket-works)

Rate limiting uses a **token bucket algorithm** and is **inbound-only**: only messages arriving *from* clients are limited. Server-originated broadcasts are passed through untouched (silently dropping a doc update would permanently diverge a client until a full resync; egress is already bounded because every broadcast originates from a rate-limited ingress message).

Each tracking key (user, document, or user-document pair) gets a bucket that holds up to `maxMessages` tokens. Every incoming message consumes one token. Tokens refill at a steady rate of `maxMessages` per `windowMs`, so short bursts (up to `maxMessages` at once) are allowed while an average rate is enforced over time.

When a message arrives and the bucket is empty, it is **held (delayed), not immediately dropped** (default behavior). The message waits until its bucket refills, up to `maxDelayMs` (default 1000ms). Because each connection’s inbound stream is consumed sequentially, holding naturally throttles the sender losslessly. Only a wait that would exceed the budget causes the message to be dropped – and a dropped message fires `onRateLimitExceeded` and is nacked (an ACK carrying `retryAfter`) so the client can retransmit, never thrown. Set `maxDelayMs: 0` for legacy drop-only behavior.

When multiple rules apply, **all rules must pass**, and a message rejected by a later rule **refunds** the tokens it already consumed from earlier rules, so a client’s retransmit isn’t double-charged.

ACK messages are automatically excluded from rate limiting since they are protocol-level acknowledgments, not user-initiated traffic. Rate limiting is also applied **before** permission checking – messages that exceed the limit are held or nacked without incurring the cost of a permission check.

## Dynamic Rate Limits

[Section titled “Dynamic Rate Limits”](#dynamic-rate-limits)

Both `maxMessages` and `windowMs` on each rule accept either a static number or a function that receives the current message and returns a number. This lets you vary limits based on user role, document type, or any other context.

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  // ...
  rateLimitConfig: {
    rules: [
      {
        id: "per-user",
        maxMessages: (msg) => {
          if (msg.context.role === "premium") return 500;
          return 100;
        },
        windowMs: (msg) => {
          // Tighter window for public documents
          if (msg.document?.startsWith("public/")) return 500;
          return 1000;
        },
        trackBy: "user",
      },
    ],
  },
});
```

## Skipping Rate Limits

[Section titled “Skipping Rate Limits”](#skipping-rate-limits)

Use `shouldSkipRateLimit` to bypass rate limiting entirely for certain messages. This is useful for admin users, internal service accounts, or specific message types that should never be throttled.

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  // ...
  rateLimitConfig: {
    rules: [
      {
        id: "per-user",
        maxMessages: 100,
        windowMs: 1000,
        trackBy: "user",
      },
    ],
    shouldSkipRateLimit: async (msg) => {
      return msg.context.role === "admin";
    },
  },
});
```

## Alerting on Rate Limit Exceeded

[Section titled “Alerting on Rate Limit Exceeded”](#alerting-on-rate-limit-exceeded)

The `onRateLimitExceeded` callback fires every time a message is dropped due to rate limiting (after exhausting its allowed hold time). It receives structured details about the event, making it straightforward to integrate with logging and alerting systems. Note that `resetAt` is the timestamp when the **next token** refills – not the end of a full window – so a nacked sender only waits the fractional remainder of a single token before retrying.

```typescript
import { Server } from "teleportal/server";


const server = new Server({
  // ...
  rateLimitConfig: {
    rules: [
      {
        id: "per-user",
        maxMessages: 100,
        windowMs: 1000,
        trackBy: "user",
      },
    ],
    onRateLimitExceeded: (details) => {
      logger.warn("Rate limit exceeded", {
        ruleId: details.ruleId,
        userId: details.userId,
        documentId: details.documentId,
        currentCount: details.currentCount,
        maxMessages: details.maxMessages,
        resetAt: new Date(details.resetAt).toISOString(),
      });
      // Send to your alerting service
      alertService.notify("rate-limit-exceeded", details);
    },
    onMessageSizeExceeded: (details) => {
      logger.warn("Message size exceeded", {
        size: details.size,
        maxSize: details.maxSize,
      });
    },
  },
});
```

## Multi-Node Deployments

[Section titled “Multi-Node Deployments”](#multi-node-deployments)

For multi-node deployments, use persistent rate limit storage so that limits are shared across all server instances:

```typescript
import { RedisRateLimitStorage } from "teleportal/transports/redis";


const rateLimitStorage = new RedisRateLimitStorage(redisClient);


const server = new Server({
  // ... other options
  rateLimitConfig: {
    rateLimitStorage, // Shared across server instances
    // ... other options
  },
});
```

Without persistent storage, each server instance tracks limits independently (in-memory per transport), so a user could exceed the intended limit by connecting to multiple nodes.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Server](/docs/core-concepts/server/) - Learn more about server configuration
* [Scaling](/docs/advanced/scaling/) - Multi-node deployment strategies

# Custom RPC Methods

> Extend the Teleportal protocol with custom request/response operations

Teleportal’s RPC framework lets you define custom request/response methods that run over the existing sync connection. This extends the protocol without introducing new message types or transports – your custom methods share the same connection, authentication, and encryption as document sync.

Several built-in features use this system: file transfers, milestones, attribution queries, and the key registry are all implemented as RPC protocols.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Defining a typed RPC method contract
* Implementing server-side handlers
* Creating a client extension
* Wiring everything together with the Server and Provider

## Define the Contract

[Section titled “Define the Contract”](#define-the-contract)

Start by defining your method’s wire name, request type, and response type using `defineMethod`. Group related methods into a protocol with `defineProtocol`:

comments/methods.ts

```typescript
import { defineMethod, defineProtocol } from "teleportal/rpc";


interface Comment {
  id: string;
  text: string;
  userId: string;
  createdAt: number;
}


export const commentList = defineMethod<
  "commentList",
  { cursor?: string; limit?: number },
  { comments: Comment[]; nextCursor?: string }
>("commentList");


export const commentCreate = defineMethod<
  "commentCreate",
  { text: string; parentId?: string },
  { comment: Comment }
>("commentCreate");


export const commentProtocol = defineProtocol("comments", {
  list: commentList,
  create: commentCreate,
});
```

The first type parameter is the wire name (the string sent over the connection), followed by the request and response types. `defineProtocol` groups methods under ergonomic keys (`list`, `create`) while preserving their wire names.

## Implement Server Handlers

[Section titled “Implement Server Handlers”](#implement-server-handlers)

Use `createHandlers` to implement the protocol. Each handler receives a dependency bag (first argument) and returns an async function that processes the request:

comments/server.ts

```typescript
import { createHandlers, ok, err } from "teleportal/rpc";
import { commentProtocol } from "./methods";


export function getCommentRpcHandlers(db: CommentDB) {
  return createHandlers(
    commentProtocol,
    { db },
    {
      list:
        ({ db }) =>
        async (payload, ctx) => {
          const result = await db.comments.find({
            documentId: ctx.documentId,
            cursor: payload.cursor,
            limit: payload.limit ?? 50,
          });
          return ok({ comments: result.items, nextCursor: result.nextCursor });
        },


      create:
        ({ db }) =>
        async (payload, ctx) => {
          if (!ctx.userId) return err(401, "Authentication required");
          const comment = await db.comments.insert({
            documentId: ctx.documentId,
            text: payload.text,
            userId: ctx.userId,
          });
          return ok({ comment });
        },
    },
  );
}
```

Handlers return `ok(value)` for success or `err(statusCode, details)` for errors. Unexpected exceptions are caught automatically and translated to `err(500, message)`.

Tip

The `ctx` parameter gives you access to `documentId`, `userId`, and other session context. Use it for authorization checks and document-scoped queries.

## Create the Client Extension

[Section titled “Create the Client Extension”](#create-the-client-extension)

For simple protocols, auto-generate the client with `createClientExtension`:

comments/client.ts

```typescript
import { createClientExtension } from "teleportal/rpc";
import { commentProtocol } from "./methods";


export const createCommentRpc = createClientExtension(commentProtocol);
```

For a more ergonomic API, provide a custom `build` function that transforms the raw RPC calls:

```typescript
export const createCommentRpc = createClientExtension(commentProtocol, {
  build(methods) {
    return {
      async list(cursor?: string) {
        const response = await methods.list({ cursor, limit: 50 });
        return response.comments;
      },
      async create(text: string) {
        const response = await methods.create({ text });
        return response.comment;
      },
    };
  },
});
```

## Wire It Up

[Section titled “Wire It Up”](#wire-it-up)

Register the handlers on the server and the client extension on the provider:

```typescript
// Server
import { Server } from "teleportal/server";
import { MemoryDocumentStorage } from "teleportal/storage";
import { getCommentRpcHandlers } from "./comments/server";


const server = new Server({
  storage: async (ctx) => new MemoryDocumentStorage(ctx.encrypted),
  rpcHandlers: {
    ...getCommentRpcHandlers(db),
  },
});


// Client
import { Provider } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";
import { createCommentRpc } from "./comments/client";


const encryptionKey = createEncryptionKey();


const provider = await Provider.create({
  url: "wss://example.com/sync",
  document: "my-doc",
  encryptionKey,
  rpc: { comments: createCommentRpc },
});


// Use the RPC methods
const comments = await provider.rpc.comments.list({ limit: 10 });
const newComment = await provider.rpc.comments.create({ text: "Hello!" });
```

## Error Handling

[Section titled “Error Handling”](#error-handling)

Client-side RPC failures throw `RpcOperationError`, which includes the protocol and operation name:

```typescript
import { RpcOperationError } from "teleportal/rpc";


try {
  await provider.rpc.comments.create({ text: "" });
} catch (error) {
  if (error instanceof RpcOperationError) {
    console.error(error.protocol); // "comments"
    console.error(error.operation); // "create"
  }
}
```

## Built-in RPC Protocols

[Section titled “Built-in RPC Protocols”](#built-in-rpc-protocols)

These built-in protocols demonstrate the same pattern at production scale:

* **File protocol** (`teleportal/protocols/file`) – Chunked file upload/download with Merkle tree verification. Uses the `"multipart"` method kind for streaming uploads.
* **Milestone protocol** (`teleportal/protocols/milestone`) – Create, list, get, and delete document snapshots.
* **Attribution protocol** (`teleportal/protocols/attribution`) – Query authorship data for document content.
* **Key registry protocol** (`teleportal/protocols/key-registry`) – Server-mediated encryption key distribution with wrapping, rotation, and revocation.

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [File Transfers](/docs/guides/file-transfers/) - File upload/download using the file RPC protocol
* [Encryption Keys](/docs/guides/encryption-keys/) - Key management and distribution
* [Observability](/docs/guides/observability/) - Monitor RPC calls with metrics and logging

# SharedWorker Connection

> Offload the network connection to a SharedWorker so all tabs share a single transport

When running in a browser that supports [`SharedWorker`](https://developer.mozilla.org/en-US/docs/Web/API/SharedWorker), Teleportal can offload the network connection to a shared worker. All open tabs then share a single underlying WebSocket (or HTTP) connection to the sync server, reducing resource usage and keeping state consistent across tabs.

This is opt-in: pass a `workerUrl` to `createConnection()` or use `WorkerProvider.create()`. When `SharedWorker` is unavailable (Node.js, older browsers, `file://` origins, restrictive CSP), the system transparently falls back to a direct in-thread connection.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Offloading the network connection to a SharedWorker
* Connection pooling across browser tabs
* Automatic fallback when SharedWorker is unavailable
* Grace period to survive page reloads without reconnection

## Quick Start

[Section titled “Quick Start”](#quick-start)

The simplest way to use SharedWorker support is through `WorkerProvider`:

```typescript
import { WorkerProvider } from "teleportal/providers/worker";
import { websocketTransport, httpTransport } from "teleportal/providers";


const provider = await WorkerProvider.create({
  // Point to your worker script (see "Worker Script" below)
  workerUrl: new URL("./worker.ts", import.meta.url),


  // Server URL
  url: "wss://example.com/sync",


  // Transports used as fallback if SharedWorker is unavailable
  transports: [websocketTransport(), httpTransport()],


  // Document to sync
  document: "my-document",
  encryptionKey: createEncryptionKey(),
});


await provider.synced;


// Use exactly like a regular Provider
provider.doc.getText("content").insert(0, "Hello from SharedWorker!");
```

## Worker Script

[Section titled “Worker Script”](#worker-script)

The worker script runs inside the SharedWorker and maps serializable transport descriptors to real transport instances. Teleportal ships a default entry point at `teleportal/providers/worker/connection-worker`, or you can write your own:

worker.ts

```typescript
import { ConnectionWorkerManager } from "teleportal/providers/worker";
import { websocketTransport } from "teleportal/providers";
import { httpTransport } from "teleportal/providers";


const manager = new ConnectionWorkerManager((options) => {
  if (options.transports && options.transports.length > 0) {
    return options.transports.map((desc) => {
      switch (desc.type) {
        case "websocket":
          return websocketTransport(desc.options);
        case "http":
          return httpTransport(desc.options);
        default:
          throw new Error(`Unknown transport type: ${desc.type}`);
      }
    });
  }
  // Default transports when none are specified
  return [websocketTransport({ timeout: 5000 }), httpTransport()];
});


declare const self: { onconnect: ((event: MessageEvent) => void) | null };
self.onconnect = (event: MessageEvent) => {
  manager.addPort(event.ports[0]);
};
```

Tip

The worker script must be bundled separately from your main application. Most bundlers (Vite, esbuild, webpack) handle `new URL("./worker.ts", import.meta.url)` as a separate entry point automatically.

## Using `createConnection` Directly

[Section titled “Using createConnection Directly”](#using-createconnection-directly)

For more control, use `createConnection` to get a `Connection` instance and pair it with a `Provider` yourself:

```typescript
import { createConnection } from "teleportal/providers/worker";
import { Provider } from "teleportal/providers";
import { websocketTransport, httpTransport } from "teleportal/providers";


const connection = createConnection({
  workerUrl: new URL("./worker.ts", import.meta.url),
  url: "wss://example.com/sync",
  token: { token: jwt },
  transports: [websocketTransport(), httpTransport()],


  // Optional: descriptors forwarded to the worker
  workerTransports: [
    { type: "websocket", options: { timeout: 5000 } },
    { type: "http", options: {} },
  ],


  // Called if the SharedWorker crashes or heartbeat times out
  onWorkerDeath: () => {
    console.warn("SharedWorker died — consider reloading");
  },
});


const provider = new Provider({
  connection,
  document: "my-document",
  encryptionKey: createEncryptionKey(),
});


await provider.synced;
```

## How Connection Pooling Works

[Section titled “How Connection Pooling Works”](#how-connection-pooling-works)

The `ConnectionWorkerManager` inside the worker pools connections using a **connection key** derived from the URL and authentication token:

```plaintext
key = url + "::" + token
```

* **Same user, same server**: all tabs share one WebSocket connection.
* **Different tokens** (different users or auth sessions): separate connections, keeping attribution isolated.
* **Page reload**: the grace period (default 5 seconds) keeps the connection alive so the reloaded tab reconnects instantly without a server handshake.

### Custom Pooling Key

[Section titled “Custom Pooling Key”](#custom-pooling-key)

Override the default key function to control which tabs share connections:

```typescript
const manager = new ConnectionWorkerManager(transportFactory, {
  getConnectionKey: (options) => {
    // Key on user ID so token refreshes don't create new connections
    const claims = decodeJwt(options.token);
    return `${options.url}::${claims.sub}`;
  },
  gracePeriodMs: 10_000, // Keep connection alive 10s after last tab leaves
});
```

## Architecture

[Section titled “Architecture”](#architecture)

```
graph TD
  subgraph Tabs["Browser Tabs"]
    A["Tab A<br/>WorkerConnection"]
    B["Tab B<br/>WorkerConnection"]
    C["Tab C<br/>WorkerConnection"]
  end

  A -- MessagePort --> CWM
  B -- MessagePort --> CWM
  C -- MessagePort --> CWM

  subgraph SW["SharedWorker"]
    CWM["ConnectionWorkerManager"]
    MC1["ManagedConnection<br/><i>key: url+tokenA</i>"]
    MC2["ManagedConnection<br/><i>key: url+tokenB</i>"]
    DC1["DirectConnection"]
    DC2["DirectConnection"]
    CWM --> MC1
    CWM --> MC2
    MC1 --> DC1
    MC2 --> DC2
  end

  DC1 -- "WebSocket / HTTP" --> S["Sync Server"]
  DC2 -- "WebSocket / HTTP" --> S
```

Each `ManagedConnection` wraps a single `DirectConnection` and handles:

* **Event forwarding** to all attached ports
* **Online/offline reconciliation** (connection stays online if *any* tab is online)
* **Grace period cleanup** when the last tab disconnects

## Heartbeat and Liveness

[Section titled “Heartbeat and Liveness”](#heartbeat-and-liveness)

Each tab runs a heartbeat loop (default: 5-second interval, 2 max missed pings) to detect if the SharedWorker has crashed:

1. The main thread sends a `heartbeat` message every interval
2. The worker replies with `heartbeat-ack`
3. If more than `maxMisses` consecutive acks are missed, the worker is declared dead
4. The `onWorkerDeath` callback fires, allowing you to fall back or notify the user

Note

Heartbeat detection is per-tab. If one tab’s port breaks but the worker is alive, only that tab detects the failure. Other tabs continue normally.

## Online/Offline Handling

[Section titled “Online/Offline Handling”](#onlineoffline-handling)

Each tab forwards browser `online`/`offline` events to the worker. The worker uses an **any-tab-online** policy:

* If **any** tab reports online, the connection stays up and reconnection is allowed
* Only when **all** tabs report offline does the connection go offline
* During the grace period (no tabs attached), the connection defaults to online

This prevents a single backgrounded tab from taking the shared connection offline.

## Fallback Behavior

[Section titled “Fallback Behavior”](#fallback-behavior)

`createConnection` handles fallback transparently:

1. If `workerUrl` is provided and `SharedWorker` is available, creates a `WorkerConnection`
2. If `SharedWorker` construction fails (CSP, `file://` origin, etc.), falls back to `DirectConnection`
3. If `workerUrl` is omitted, always creates a `DirectConnection`

The returned `Connection` implements the same interface in all cases. Your `Provider` code does not need to know whether it is backed by a SharedWorker or a direct connection.

## Configuration Reference

[Section titled “Configuration Reference”](#configuration-reference)

### `createConnection` Options

[Section titled “createConnection Options”](#createconnection-options)

| Option                  | Type                    | Default    | Description                                                      |
| ----------------------- | ----------------------- | ---------- | ---------------------------------------------------------------- |
| `workerUrl`             | `string \| URL`         | —          | URL of the SharedWorker script. Omit to use a direct connection. |
| `url`                   | `string`                | (required) | Sync server URL.                                                 |
| `token`                 | `TokenOptions`          | —          | Authentication token.                                            |
| `transports`            | `ConnectionTransport[]` | (required) | Transport instances for the direct fallback path.                |
| `workerTransports`      | `TransportDescriptor[]` | —          | Serializable transport descriptors forwarded to the worker.      |
| `onWorkerDeath`         | `() => void`            | —          | Called if the SharedWorker crashes or heartbeat times out.       |
| `connect`               | `boolean`               | `true`     | Auto-connect on creation.                                        |
| `maxReconnectAttempts`  | `number`                | `10`       | Max reconnection attempts.                                       |
| `initialReconnectDelay` | `number`                | `100`      | Initial backoff delay (ms).                                      |
| `maxBackoffTime`        | `number`                | `30000`    | Maximum backoff delay (ms).                                      |

### `ConnectionWorkerManager` Options

[Section titled “ConnectionWorkerManager Options”](#connectionworkermanager-options)

| Option             | Type                  | Default     | Description                                                         |
| ------------------ | --------------------- | ----------- | ------------------------------------------------------------------- |
| `gracePeriodMs`    | `number`              | `5000`      | How long to keep a connection alive after the last tab disconnects. |
| `getConnectionKey` | `(options) => string` | URL + token | Custom function to determine which tabs share a connection.         |

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Fallback Connection](/docs/guides/fallback-connection/) - Automatic WebSocket-to-HTTP fallback
* [WebSocket Only](/docs/guides/websocket-only/) - Minimal WebSocket setup
* [Performance](/docs/advanced/performance/) - Performance optimization strategies

# WebSocket Only

> Minimal server setup with WebSocket transport

This guide demonstrates a minimal Teleportal server setup using only WebSocket connections. The server uses in-memory storage and accepts all WebSocket upgrade requests.

## What it demonstrates

[Section titled “What it demonstrates”](#what-it-demonstrates)

* Setting up a basic Teleportal server with WebSocket transport only
* Using in-memory `MemoryDocumentStorage` for document storage
* Handling WebSocket upgrades with context extraction
* Client connection using the `Provider` API with WebSocket transport

## Server Setup

[Section titled “Server Setup”](#server-setup)

```typescript
import { serve } from "crossws/server";
import { Server } from "teleportal/server";
import { MemoryDocumentStorage } from "teleportal/storage";
import { getWebsocketHandlers } from "teleportal/websocket-server";


// Create a Teleportal server with in-memory storage
const server = new Server({
  storage: new MemoryDocumentStorage(),
});


// Set up WebSocket handlers
serve({
  websocket: getWebsocketHandlers({
    server,
    onUpgrade: async () => {
      // Extract user context from the request
      // In production, you'd verify authentication here
      return {
        context: { userId: "nick", room: "test" },
      };
    },
  }),
  fetch: () => new Response("Not found", { status: 404 }),
});
```

## Client Setup

[Section titled “Client Setup”](#client-setup)

```typescript
import { Provider } from "teleportal/providers";
import { createEncryptionKey } from "teleportal/encryption-key";


const provider = await Provider.create({
  url: `ws://localhost:3000`,
  document: "test",
  encryptionKey: createEncryptionKey(),
});


await provider.synced;


provider.doc.getText("test").insert(0, "Hello, world!");


console.log(provider.doc.getText("test").toString());


provider.doc.on("update", () => {
  console.log(provider.doc.getText("test").toString());
});
```

## How It Works

[Section titled “How It Works”](#how-it-works)

1. **Server**: Creates a `Server` instance with in-memory `MemoryDocumentStorage`
2. **WebSocket Handlers**: Uses `getWebsocketHandlers` to handle WebSocket connections
3. **Context Extraction**: `onUpgrade` callback extracts user context from the request
4. **Client**: `Provider.create()` automatically connects via WebSocket
5. **Synchronization**: `provider.synced` waits for the document to be fully synchronized

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [HTTP Transport](/docs/guides/http-transport/) - Learn about HTTP/SSE transport
* [Authentication](/docs/guides/authentication/) - Add authentication to your server
* [Persistent Storage](/docs/guides/persistent-storage/) - Use persistent storage instead of in-memory

# Integration Guide

> Decision tree and considerations for integrating Teleportal into your application

This guide helps you make decisions about how to integrate Teleportal into your application.

## Storage

[Section titled “Storage”](#storage)

Teleportal supports any storage backend through the `DocumentStorage` interface. **Unstorage** (recommended) works with Redis, PostgreSQL, S3, and many other backends. For development, use **in-memory storage**. For special requirements, implement a custom `DocumentStorage` interface.

```typescript
// Unstorage (production)
import { createStorage } from "unstorage";
import { UnstorageDocumentStorage } from "teleportal/storage";
import redisDriver from "unstorage/drivers/redis";


const storage = createStorage({
  driver: redisDriver({ base: "teleportal:" }),
});


const server = new Server({
  storage: async (ctx) => {
    return new UnstorageDocumentStorage(storage, {
      keyPrefix: "doc",
      encrypted: ctx.encrypted,
    });
  },
});


// In-memory (development)
import { MemoryDocumentStorage } from "teleportal/storage";


const server = new Server({
  storage: async (ctx) => {
    return new MemoryDocumentStorage(ctx.encrypted);
  },
});
```

See [Custom Storage](/docs/advanced/custom-storage/) for custom implementations.

## Transport

[Section titled “Transport”](#transport)

**WebSocket** (default) provides bidirectional communication with low latency. Use **HTTP with Server-Sent Events** for corporate networks that block WebSockets. The client can automatically use a **fallback connection** that tries WebSocket first, then falls back to HTTP.

```typescript
// WebSocket
import { getWebsocketHandlers } from "teleportal/websocket-server";


const handlers = getWebsocketHandlers({
  server,
  onUpgrade: async (request) => {
    return { context: { userId: "user-123" } };
  },
});


// HTTP/SSE
import { getHTTPHandlers } from "teleportal/http";


const handlers = getHTTPHandlers({
  server,
  getContext: async (request) => {
    return { userId: "user-123" };
  },
});
```

## Encryption

[Section titled “Encryption”](#encryption)

**Content-level end-to-end encryption is the default.** The client encrypts document content into sidecars before it leaves the device; the server only ever sees the plaintext CRDT structure (needed for merge/sync) and the encrypted sidecars — never your content or keys. Every `Provider` requires an `encryptionKey` (a `CryptoKey`); to deliberately run a plaintext document, pass `encryptionKey: false`.

```typescript
import { Provider } from "teleportal/providers";
import {
  createEncryptionKey,
  exportEncryptionKey,
  keyToUrlFragment,
} from "teleportal/encryption-key";


// Encrypted by default
const provider = await Provider.create({
  url: "wss://example.com",
  document: "my-document",
  encryptionKey: createEncryptionKey(),
});


// Share the key with collaborators via the URL fragment (never sent to the server)
location.hash = keyToUrlFragment(await exportEncryptionKey(provider.encryptionKey));


// Opt a single document out into plaintext
const plaintextProvider = await Provider.create({
  url: "wss://example.com",
  document: "public-doc",
  encryptionKey: false,
});
```

The server enforces that all clients of one document agree on encryption mode (mixing plaintext and encrypted clients on the same document throws). Note that this is distinct from [Encryption at Rest](/docs/guides/encryption-at-rest/), which encrypts data in the storage backend server-side.

## Runtime

[Section titled “Runtime”](#runtime)

Teleportal works on any JavaScript runtime: **Bun** (recommended, fastest), Node.js, Deno, Cloudflare Workers, and edge runtimes (Vercel, Netlify).

## Authentication

[Section titled “Authentication”](#authentication)

**JWT tokens** (built-in) include IAM-like permissions. For existing auth systems, implement custom authentication in `onUpgrade`.

```typescript
// JWT (built-in)
import { createTokenManager } from "teleportal/token";


const tokenManager = createTokenManager({
  secret: "your-secret-key",
  expiresIn: 3600,
});


const token = await tokenManager.createToken("user-123", "org-456", [
  { pattern: "user-123/*", permissions: ["read", "write"] },
]);


// Custom auth
const handlers = getWebsocketHandlers({
  server,
  onUpgrade: async (request) => {
    const user = await verifySession(request);
    if (!user) throw new Response("Unauthorized", { status: 401 });
    return { context: { userId: user.id } };
  },
});
```

## Features

[Section titled “Features”](#features)

**Document synchronization** is always included. **File synchronization** and **milestone synchronization** are optional. You can also implement custom RPC handlers.

```typescript
// File sync (optional)
import { getFileRpcHandlers } from "teleportal/protocols/file";


const server = new Server({
  rpcHandlers: {
    ...getFileRpcHandlers(fileStorage),
  },
});


// Milestone sync (optional)
import { getMilestoneRpcHandlers } from "teleportal/protocols/milestone";


const server = new Server({
  rpcHandlers: {
    ...getMilestoneRpcHandlers(milestoneStorage),
  },
});
```

## Deployment

[Section titled “Deployment”](#deployment)

For **single-node** deployments, run one server instance with in-memory or local storage. For **multi-node** deployments, use shared storage with PubSub (Redis, NATS) for message coordination, or use an HTTP load balancer with sticky sessions.

```typescript
// Multi-node with PubSub
import { RedisPubSub } from "teleportal/transports/redis";


const server = new Server({
  storage: async (ctx) => {
    // Shared storage
  },
  pubSub: new RedisPubSub({
    path: "redis://localhost:6379",
  }),
  nodeId: process.env.NODE_ID,
});
```

See [Scaling](/docs/advanced/scaling/) for custom deployment strategies.

## Monitoring & Logging

[Section titled “Monitoring & Logging”](#monitoring--logging)

Built-in **Prometheus metrics** and **health checks** are available. Integrate custom monitoring using server events. Teleportal uses `@logtape/logtape` for structured logging—configure adapters for your logging system.

```typescript
// Metrics & health
import { getMetricsHandler, getHealthHandler } from "teleportal/http";


app.get("/metrics", getMetricsHandler(server));
app.get("/health", getHealthHandler(server));


// Custom monitoring
server.on("client-connect", (data) => {
  // Send to your monitoring system
});
```

## Decision Tree

[Section titled “Decision Tree”](#decision-tree)

```
flowchart TD
    Start([Start]) --> Storage{Storage}
    Storage -->|Most cases| Unstorage[Unstorage]
    Storage -->|Dev / test| InMemory[In-Memory]

    Start --> Transport{Transport}
    Transport -->|Default| WS[WebSocket]
    Transport -->|Firewalls| HTTP[HTTP / SSE]

    Start --> Runtime{Runtime}
    Runtime -->|Recommended| Bun[Bun]
    Runtime -->|Also supported| Node[Node.js]
    Runtime -->|Also supported| Deno[Deno]
    Runtime -->|Also supported| Edge[Edge Runtimes]

    Start --> Auth{Auth}
    Auth -->|Built-in| JWT[JWT]
    Auth -->|Existing system| Custom[Custom onUpgrade]

    Start --> Encryption{Encryption}
    Encryption -->|Default| E2EE[Content E2EE]
    Encryption -->|Per-doc opt-out| Plaintext["Plaintext (encryptionKey: false)"]

    Start --> Features{Features}
    Features -->|Always| DocSync[Document Sync]
    Features -->|Optional| FileSync[File Sync]
    Features -->|Optional| Milestones[Milestones]

    Start --> Deployment{Deployment}
    Deployment -->|One instance| Single[Single-node]
    Deployment -->|Scaling| Multi[Multi-node + PubSub]

    Start --> Monitoring{Monitoring}
    Monitoring -->|Built-in| Prometheus[Prometheus]
    Monitoring -->|Custom| CustomMon[Server Events]
```

## Next Steps

[Section titled “Next Steps”](#next-steps)

* [Core Concepts](/docs/core-concepts/) - Understand the architecture
* [Guides](/docs/guides/) - Step-by-step implementation guides
* [Advanced Topics](/docs/advanced/) - Custom implementations and optimizations

# What is Teleportal?

> Why Teleportal exists and how it compares to other Y.js server solutions

Teleportal is a real-time collaborative editing framework built on Y.js. It allows you to sync documents in a conflict-free manner, with support for any storage, transport, and runtime.

## What Y.js Provides

[Section titled “What Y.js Provides”](#what-yjs-provides)

Y.js is a powerful CRDT (Conflict-free Replicated Data Type) library that enables real-time collaborative editing. At its core, Y.js provides:

**Conflict-free edits** through its CRDT data structure. This means that changes from multiple clients can be merged together automatically without manual conflict resolution. No matter how edits are applied or in what order, all clients will converge to the same final state.

**Offline editing support**. You can make changes while disconnected, and when you come back online, your changes will be automatically merged with any changes that occurred while you were offline. The CRDT guarantees that the final state will be consistent across all clients.

**Real-time synchronization** of document changes, including awareness information like cursor positions and user presence. This enables rich collaborative experiences where users can see each other’s cursors and selections in real-time.

**Flexible data modeling**. Y.js supports anything that can be represented as JSON—from simple text documents to complex structured data like nested objects, arrays, and maps. This makes it suitable for everything from collaborative text editors to real-time data synchronization for complex applications.

The key insight is that with Y.js, multiple clients can edit the same document simultaneously, and regardless of network conditions, timing, or the order in which changes arrive, they will all converge to the same result.

## What Teleportal Provides

[Section titled “What Teleportal Provides”](#what-teleportal-provides)

Teleportal is an abstraction and utility library built on top of Y.js that focuses on helping you create the **server implementation** for your Y.js-based application.

While Y.js handles the client-side CRDT operations and conflict resolution, you still need a server that:

* **Persists document state to storage** so data survives server restarts and can be loaded on demand
* **Synchronizes changes between clients** by receiving updates from one client and broadcasting them to others
* **Enforces permissions and access control** to ensure only authorized users can read or modify documents
* **Provides observability** through metrics, logging, and monitoring to understand system behavior and debug issues
* **Handles connection management** for WebSocket and HTTP transports, managing client sessions and reconnections
* **Supports scaling** across multiple server instances with pub/sub and distributed storage

Teleportal provides all of these server-side capabilities out of the box, with a flexible, extensible architecture that lets you customize storage backends, transport mechanisms, and middleware to fit your specific needs. It’s designed to be storage-agnostic, transport-agnostic, and runtime-agnostic, giving you the building blocks to create a production-ready sync server without having to implement all the infrastructure yourself.

## How Teleportal Compares

[Section titled “How Teleportal Compares”](#how-teleportal-compares)

There are several ways to add Y.js sync to your application. Here is how Teleportal differs from the most common alternatives:

**y-websocket** is the reference WebSocket server from the Y.js project. It gets you running quickly, but it stores documents only in memory (or LevelDB), uses a single transport (WebSocket), and has no built-in auth, encryption, or scaling story. If you outgrow it, you end up writing most of the infrastructure Teleportal already provides.

**Hocuspocus** is a Node.js-based Y.js server with a hook/plugin system. It gives you lifecycle hooks for auth and storage, but it couples you to a specific server runtime and WebSocket library. Teleportal takes a different approach: instead of hooks on a fixed server, it gives you composable building blocks (storage interfaces, transport adapters, middleware) that you assemble however your architecture demands.

**Liveblocks** is a hosted platform that includes Y.js support as part of a broader collaboration suite. It is a managed service – you don’t run the server. Teleportal is for teams that want to own their infrastructure: self-hosted, no vendor lock-in, full control over storage, networking, and encryption.

### Key Differentiators

[Section titled “Key Differentiators”](#key-differentiators)

* **Framework, not a server.** Teleportal gives you interfaces and adapters that you compose into your own server. You are not locked into a particular server shape or deployment model.
* **Storage and transport agnostic.** Bring any database (SQLite, Postgres, S3, or your own) and any transport (WebSocket, HTTP/SSE, or your own). Swap them without changing application code.
* **End-to-end encryption by default.** Content E2EE is on by default – the server never sees plaintext document content. This is not an add-on; it is the baseline.
* **No in-memory storage requirement.** Documents are loaded from storage on demand and can be evicted from memory when idle. This means your server’s memory footprint stays constant regardless of how many documents exist.
* **Runtime agnostic.** Teleportal uses standard JavaScript APIs and runs on Bun, Node.js, Deno, or Cloudflare Workers without platform-specific adapters.