perplexity/the-matrix
2
1
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

[Infra] WS Gateway server — central message hub for Matrix clients, agents, and bots #70

New Issue
Closed
opened 2026-03-19 20:19:39 -04:00 by perplexity · 1 comment
perplexity commented 2026-03-19 20:19:39 -04:00
Owner
Copy Link

Context

The Matrix needs a central WebSocket server that all participants connect to: the 3D browser client, Timmy's cognitive bridge (#69), the Perplexity interview bot, and any future agents. This is the server-side complement to the client-side behavior system (#68).

Issue #48 scoped this for the interview use case. This issue expands it to be the permanent gateway for all Matrix communication.

What to build

server/gateway.py — WebSocket gateway

A lightweight Python (FastAPI or standalone websockets) server:

  1. Endpoint: ws://0.0.0.0:8765/ws/matrix
  2. Auth: ?token=<agent_token> query param
  3. Registration: clients send agent_register on connect with their agentId and role
  4. Message routing:
    • Broadcast: messages without target_id go to all connected clients
    • Directed: messages with target_id route only to that agent's connection
    • Browser: Matrix 3D client gets all visual messages (agent_state, bark, scene_*, agent_move, agent_behavior)
  5. Connection tracking: maintain a registry of who's connected, expose via REST /api/gateway/status
  6. Heartbeat: 30s ping/pong, prune dead connections

Message flow

Timmy's Dashboard ──ws──→ Gateway ──ws──→ Matrix 3D Client (iPad)
                              ↕
                    Perplexity Bot ──ws──→
                              ↕
                    Future agents ──ws──→

Supported message types (pass-through)

The gateway does NOT interpret messages — it routes them. All Matrix protocol messages pass through:

  • agent_state, agent_move, agent_behavior
  • bark, agent_message
  • scene_add, scene_update, scene_remove, scene_clear, scene_batch
  • world_register, world_load, world_unregister
  • zone_add, zone_remove
  • task_created, task_update, task_action
  • payment_flow, system_status
  • chat_message, visitor_message

Connection lifecycle

Client connects → sends agent_register → gateway broadcasts agent_joined to all
Client disconnects → gateway broadcasts agent_left to all

Acceptance Criteria

  • Gateway starts on configurable port (default 8765)
  • Matrix 3D client connects and receives messages from Timmy's bridge
  • Timmy's cognitive bridge (#69) connects and sends agent_state, bark, agent_move
  • Directed messages (target_id) route only to the target
  • agent_joined / agent_left broadcast on connect/disconnect
  • /api/gateway/status returns connected agents and their roles
  • Multiple simultaneous browser clients supported
  • 30s heartbeat with dead connection pruning
  • Graceful shutdown — notify all clients

Where it lives

New server/ directory in perplexity/the-matrix. Lightweight enough to run alongside the Vite dev server or deploy independently.

Supersedes

  • #48 (WS Gateway for interview) — this is the permanent version
  • The WS server portion of #8 (gateway adapter)

Dependencies

  • None for the gateway itself
  • #69 (cognitive bridge) is the primary client
  • #67 and #68 are the primary consumers on the Matrix 3D side
## Context The Matrix needs a central WebSocket server that all participants connect to: the 3D browser client, Timmy's cognitive bridge (#69), the Perplexity interview bot, and any future agents. This is the server-side complement to the client-side behavior system (#68). Issue #48 scoped this for the interview use case. This issue expands it to be the permanent gateway for all Matrix communication. ## What to build ### `server/gateway.py` — WebSocket gateway A lightweight Python (FastAPI or standalone `websockets`) server: 1. **Endpoint**: `ws://0.0.0.0:8765/ws/matrix` 2. **Auth**: `?token=<agent_token>` query param 3. **Registration**: clients send `agent_register` on connect with their `agentId` and `role` 4. **Message routing**: - **Broadcast**: messages without `target_id` go to all connected clients - **Directed**: messages with `target_id` route only to that agent's connection - **Browser**: Matrix 3D client gets all visual messages (`agent_state`, `bark`, `scene_*`, `agent_move`, `agent_behavior`) 5. **Connection tracking**: maintain a registry of who's connected, expose via REST `/api/gateway/status` 6. **Heartbeat**: 30s ping/pong, prune dead connections ### Message flow ``` Timmy's Dashboard ──ws──→ Gateway ──ws──→ Matrix 3D Client (iPad) ↕ Perplexity Bot ──ws──→ ↕ Future agents ──ws──→ ``` ### Supported message types (pass-through) The gateway does NOT interpret messages — it routes them. All Matrix protocol messages pass through: - `agent_state`, `agent_move`, `agent_behavior` - `bark`, `agent_message` - `scene_add`, `scene_update`, `scene_remove`, `scene_clear`, `scene_batch` - `world_register`, `world_load`, `world_unregister` - `zone_add`, `zone_remove` - `task_created`, `task_update`, `task_action` - `payment_flow`, `system_status` - `chat_message`, `visitor_message` ### Connection lifecycle ``` Client connects → sends agent_register → gateway broadcasts agent_joined to all Client disconnects → gateway broadcasts agent_left to all ``` ## Acceptance Criteria - [ ] Gateway starts on configurable port (default 8765) - [ ] Matrix 3D client connects and receives messages from Timmy's bridge - [ ] Timmy's cognitive bridge (#69) connects and sends `agent_state`, `bark`, `agent_move` - [ ] Directed messages (`target_id`) route only to the target - [ ] `agent_joined` / `agent_left` broadcast on connect/disconnect - [ ] `/api/gateway/status` returns connected agents and their roles - [ ] Multiple simultaneous browser clients supported - [ ] 30s heartbeat with dead connection pruning - [ ] Graceful shutdown — notify all clients ## Where it lives New `server/` directory in `perplexity/the-matrix`. Lightweight enough to run alongside the Vite dev server or deploy independently. ## Supersedes - #48 (WS Gateway for interview) — this is the permanent version - The WS server portion of #8 (gateway adapter) ## Dependencies - None for the gateway itself - #69 (cognitive bridge) is the primary client - #67 and #68 are the primary consumers on the Matrix 3D side
perplexity commented 2026-03-19 20:45:06 -04:00
Author
Owner
Copy Link

Closed by PR #72 — server/gateway.py is the central WS message hub. Configurable port (default 8765), optional token auth, broadcast+directed routing, agent_register/joined/left lifecycle, 30s heartbeat pruning, optional REST /api/gateway/status. Pure pass-through router — does not interpret messages. 10/10 integration tests pass. Supersedes #48.

Closed by PR #72 — `server/gateway.py` is the central WS message hub. Configurable port (default 8765), optional token auth, broadcast+directed routing, agent_register/joined/left lifecycle, 30s heartbeat pruning, optional REST /api/gateway/status. Pure pass-through router — does not interpret messages. 10/10 integration tests pass. Supersedes #48.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
deep-research
Kimi deep research request — assigned to Alex for manual prompt execution
 enhancement
New feature or improvement
 researched
Deep research complete — findings posted as comment
 timmy
Related to Timmy agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
google manus replit perplexity
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: perplexity/the-matrix#70
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?