perplexity/the-matrix
2
1
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

docs: INTEGRATION.md v2 — Replit fork coordination + economy protocol

Browse Source 
- Added Fork Coordination section documenting replit/the-matrix changes
- Acknowledged Replit resolved issues #1 (Vite), #2 (agent-defs), #6 (UUID)
- Updated architecture diagram to reflect Vite + ES module structure
- Documented Replit's module exports for each file
- Aligned field names to camelCase (matching Replit's codebase)
- Updated Phase 2 ownership: Replit primary, Perplexity review
- Added coordination strategy: code flows replit→canonical, docs stay canonical
- Resolved WS path decision (#10): VITE_WS_URL env var
This commit is contained in:
Perplexity Computer
2026-03-18 23:54:13 +00:00
parent b0fb85dac3
commit 22daaade36
1 changed files with 377 additions and 283 deletions
Download Patch File Download Diff File Expand all files Collapse all files

660
INTEGRATION.md
View File

@@ -1,355 +1,449 @@
# Integration Plan: The Matrix × Timmy Time
**Date:** 2026-03-18
**Date:** 2026-03-18 (v2 — updated against live codebase + Replit fork)
**Author:** Perplexity Computer
**Status:** Draft — ready for dev team review
**Status:** Active — cross-referenced with issues across all three repos
---
## Overview
The Matrix is a standalone 3D world that visualizes and commands the Timmy Time agent swarm. It currently runs on a mock WebSocket. This document defines how it integrates with the two other modes of the Timmy Time system:
The Matrix is a standalone 3D world that visualizes and commands the Timmy Time agent swarm. It currently runs on a `MockWebSocket`. This document defines how it integrates with the two other systems:
1. **Timmy Dashboard** — the existing FastAPI/HTMX mission control UI (`/home`, `/briefing`, `/tasks`, `/swarm`, `/spark`, `/market`, `/tools`, etc.)
2. **Agent Token Economy** — the Lightning Network-based economic layer where agents earn/spend sats for task execution
1. **Timmy Dashboard** — FastAPI/HTMX mission control (`rockachopa/Timmy-time-dashboard`)
2. **Token-Gated Economy** — Lightning-based agent economy (`replit/token-gated-economy`)
The Matrix does not replace either system. It is a third interface — a spatial, persistent, iPad-native way to interact with the same backend.
The Matrix does not replace either system. It is a third interface — spatial, persistent, iPad-native.
### Related Issues
| Repo | Key Issues |
|---|---|
| `perplexity/the-matrix` | #1–#17 (full backlog) |
| `replit/token-gated-economy` | #1 (EPIC), #2 (WebSocket), #3 (SSE), #7 (3D env), #9 (chat UI) |
| `rockachopa/Timmy-time-dashboard` | #325 (cognitive state → Matrix), #326 (Timmy hands), #324 (three-phase loop) |
---
## Architecture
## Fork Coordination: `perplexity/the-matrix` ↔ `replit/the-matrix`
```
┌─────────────────────────────────────────────────────────┐
│ TIMMY TIME BACKEND │
│ │
│ FastAPI ─── SQLite ─── Ollama ─── Lightning (LND/CLN) │
│ │ │ │ │ │
│ ▼ ▼ ▼ ▼ │
│ /api/tasks /api/agents /api/chat /api/economy │
│ │ │ │ │ │
│ └───────────┴──────────┴────────────┘ │
│ │ │
│ WebSocket Gateway │
│ ws://tower:8080/ws │
│ │ │
└──────────────────────┼──────────────────────────────────┘
│
┌────────────┼────────────┐
│ │ │
▼ ▼ ▼
┌──────────┐ ┌──────────┐ ┌──────────┐
│ Timmy │ │ The │ │ Telegram │
│Dashboard │ │ Matrix │ │ Bot │
│(HTMX/WS)│ │(Three.js)│ │ (bridge) │
└──────────┘ └──────────┘ └──────────┘
```
Replit forked the canonical repo and is actively developing on branch `feat/vite-build-agent-defs`. This creates a two-track development model:
All three interfaces share the same backend. The Matrix connects via the WebSocket gateway, same as the dashboard's live feeds.
### What Replit Has Done (as of 2026-03-18)
| Change | Addresses Issue | Details |
|---|---|---|
| Vite build system | #1 | `package.json` + `vite.config.js`, Three.js as npm dep (`0.171.0`) |
| Consolidated agent defs | #2 | New `js/agent-defs.js` — single source of truth for agent id/color/role/position |
| `crypto.randomUUID()` | #6 | Used in `websocket.js` subscribe handshake |
| ES module refactor | — | All files use `import/export`, Three.js via npm not CDN |
| CSS inlined | — | Styles moved into `index.html <style>`, `style.css` removed |
| `PROTOCOL.md` removed | — | Working from clean slate on their fork |
| `INTEGRATION.md` removed | — | Working from clean slate on their fork |
| Simplified interaction | — | `OrbitControls` via `three/addons`, much cleaner |
| WS URL via env var | #7 partial | `import.meta.env.VITE_WS_URL` — configurable at build time |
### Coordination Strategy
**Replit = upstream for code.** Their Vite refactor is a clean break. The canonical `perplexity/the-matrix` keeps docs (`PROTOCOL.md`, `INTEGRATION.md`) and the issue tracker. Replit's code will be the production codebase.
**Workflow:**
1. Replit develops features on their fork (`replit/the-matrix`)
2. Replit opens PRs to canonical (`perplexity/the-matrix`) when features are stable
3. Perplexity (this account) reviews, tests, and merges upstream
4. Issues stay on the canonical repo — both teams reference them
5. `INTEGRATION.md` and `PROTOCOL.md` live in canonical only — Replit reads them here
**Sync cadence:** Replit syncs their fork from canonical before starting new work. Cross-fork PRs merge at feature boundaries, not every commit.
### Issues Resolved by Replit's Branch
When the `feat/vite-build-agent-defs` branch is merged upstream, the following issues can be closed:
- **#1** — Build system: Vite replaces esm.sh CDN ✓
- **#2** — Agent definition consolidation ✓
- **#6** — `crypto.randomUUID()` ✓
- **#11/#12** — partially addressed (commit mentions them, needs verification)
---
## Mode 1: Timmy Dashboard Integration
## Architecture (Current State)
### What Already Exists (Dashboard)
```
┌─── Timmy Tower (FastAPI + SQLite + Ollama) ──────────────────────┐
│ │
│ src/dashboard/ → HTMX pages, WebSocket live feeds │
│ src/timmy/ → Agent core, agentic loop, memory │
│ src/timmy/adapters/ → gitea_adapter, time_adapter (NEW) │
│ src/infrastructure/ → EventBus (async, SQLite-backed, │
│ wildcard subscriptions, replay) │
│ src/dashboard/routes/chat_api_v1.py → SSE chat, v1 API (NEW) │
│ │
└───────────────────────────────────────────────────────────────────┘
- `/tasks` — three-column queue (Pending, Running, Completed) with Approve/Modify/Veto
- `/swarm` — agent cards with status, live WebSocket log
- `/briefing` — morning briefing with task summaries
- `/home` — agent overview with TASK buttons
- SQLite `tasks` table with full lifecycle (pending_approval → approved → running → completed/vetoed/failed)
- WebSocket live feeds for task updates and agent state
┌─── Token Economy (Express + PostgreSQL + LNbits) ─────────────────┐
│ │
│ artifacts/api-server/src/routes/ │
│ jobs.ts → eval invoice → work invoice → execute → refund │
│ sessions.ts → deposit → macaroon auth → balance management │
│ ui.ts → Timmy chat interface │
│ lib/ │
│ pricing.ts → cost-based pricing, BTC oracle, model rates │
│ lnbits.ts → Lightning invoices (stub mode available) │
│ agent.ts → Anthropic eval + work execution │
│ event-bus.ts → typed EventEmitter (job:*, session:*) │
│ stream-registry.ts → SSE stream management (PR #20) │
│ │
└────────────────────────────────────────────────────────────────────┘
### What The Matrix Needs
The Matrix already speaks the right protocol (defined in `PROTOCOL.md`). The integration points:
#### 1. WebSocket Gateway Adapter
**File:** `backend/ws_gateway.py` (new)
The backend needs a WebSocket endpoint that translates between the existing FastAPI internals and the Matrix protocol format:
| Matrix Message (Client → Server) | Backend Action |
|---|---|
| `{"type": "chat_message", "agent_id": "timmy", "content": "..."}` | Forward to agent chat endpoint, return streaming response |
| `{"type": "task_action", "task_id": "...", "action": "approve"}` | Call `PATCH /api/tasks/{id}/approve` |
| `{"type": "task_action", "task_id": "...", "action": "veto"}` | Call `PATCH /api/tasks/{id}/veto` |
| `{"type": "subscribe", "channels": ["agents", "tasks"]}` | Register client for push events |
| Matrix Message (Server → Client) | Backend Trigger |
|---|---|
| `{"type": "agent_state", "agent_id": "timmy", "state": "working", ...}` | Agent state change in swarm coordinator |
| `{"type": "task_created", "task_id": "...", ...}` | New row in tasks table |
| `{"type": "task_update", "task_id": "...", "status": "completed"}` | Task status change |
| `{"type": "agent_message", "agent_id": "timmy", "content": "..."}` | Agent chat response (streamed) |
| `{"type": "memory_event", "agent_id": "...", ...}` | New memory entry logged |
| `{"type": "system_status", ...}` | Periodic heartbeat (every 5s) |
**Implementation:** Wrap the existing WebSocket infrastructure (same one powering `/swarm` live log) with a JSON message router that speaks the Matrix protocol.
#### 2. Replace MockWebSocket
**File:** `js/websocket.js` → `js/websocket-live.js`
```javascript
// Current: MockWebSocket simulates everything locally
// Target: Real WebSocket to backend
export class LiveWebSocket {
constructor(url) {
this.url = url || `ws://${location.hostname}:8080/ws/matrix`;
this.ws = null;
this.handlers = {};
this.reconnectDelay = 1000;
this._connect();
}
_connect() {
this.ws = new WebSocket(this.url);
this.ws.onmessage = (e) => {
const msg = JSON.parse(e.data);
this._dispatch(msg);
};
this.ws.onclose = () => {
setTimeout(() => this._connect(), this.reconnectDelay);
this.reconnectDelay = Math.min(this.reconnectDelay * 2, 30000);
};
this.ws.onopen = () => {
this.reconnectDelay = 1000;
this.send({ type: 'subscribe', channels: ['agents', 'tasks', 'system'] });
};
}
send(msg) {
if (this.ws?.readyState === WebSocket.OPEN) {
this.ws.send(JSON.stringify(msg));
}
}
// ... event handling same interface as MockWebSocket
}
┌─── The Matrix (Three.js + Vite + WebSocket) ─────────────────────┐
│ │
│ js/agent-defs.js → AGENT_DEFS single source of truth (NEW) │
│ js/websocket.js → WS client, VITE_WS_URL env var │
│ js/agents.js → Agent class, 3D avatars, connection lines │
│ js/ui.js → HUD, agent list, chat panel │
│ js/effects.js → Matrix rain particles, starfield │
│ js/interaction.js → OrbitControls (three/addons) │
│ js/main.js → Init + render loop │
│ js/world.js → Scene, camera, renderer, grid │
│ vite.config.js → Build config (esnext target) │
│ package.json → three@0.171.0, vite@^5.4.0 │
│ │
│ Docs (canonical repo only): │
│ PROTOCOL.md → WebSocket message spec │
│ INTEGRATION.md → This document │
│ │
└────────────────────────────────────────────────────────────────────┘
```
**Key:** The `LiveWebSocket` must expose the exact same interface as `MockWebSocket` — same `.on()`, `.send()`, `.getAgent()`, `.getAgentTasks()` methods. This keeps `ui.js`, `main.js`, and all other files untouched.
### Module Structure (Replit's Vite Refactor)
#### 3. Connection Config
**File:** `js/config.js` (new)
```javascript
export const CONFIG = {
// Auto-detect: if served from the backend, use relative WS
// If standalone, prompt or use saved endpoint
WS_URL: localStorage.getItem('matrix_ws_url')
|| (location.port === '8080' ? `ws://${location.host}/ws/matrix` : null),
// Fallback to mock mode for demo/development
USE_MOCK: false,
};
```
js/
agent-defs.js → exports: AGENT_DEFS[], colorToCss()
agents.js → exports: initAgents(), updateAgents(), setAgentState(),
getAgentCount(), getAgentDefs()
effects.js → exports: initEffects(), updateEffects()
interaction.js → exports: initInteraction(), updateControls()
main.js → entry point, imports all modules, runs animate loop
ui.js → exports: initUI(), updateUI(), appendChatMessage()
websocket.js → exports: initWebSocket(), getConnectionState(), getJobCount()
world.js → exports: initWorld(), onWindowResize()
```
**`main.js` change:** Import config, instantiate `LiveWebSocket` or `MockWebSocket` based on `CONFIG.USE_MOCK`.
Key design: `agent-defs.js` is the single source of truth. To add an agent, append one entry to `AGENT_DEFS`. No other file needs editing. WebSocket handler maps `msg.agentId` → agent defs for state and chat rendering.
#### 4. Agent Registry Sync
---
Current: 4 agents hardcoded in `AGENT_DEFS` (`websocket.js`)
## The Hard Problem: Two Payment Models
Target: Agents discovered from backend at connect time. The handshake response should include:
> Tracked in: `the-matrix#9`
```json
The Matrix protocol currently assumes simple state transitions:
```
agent_state → idle | active
job_started → increment counter
job_completed → decrement counter
```
The Token Economy has a multi-step payment flow:
```
POST /api/jobs → eval invoice (10 sats)
→ pay eval invoice → Timmy evaluates
→ work invoice (variable sats, cost-based)
→ pay work invoice → Timmy executes
→ honest accounting → refund if overpaid
```
### Resolution: Extend WebSocket message types
The Matrix message handler (`websocket.js handleMessage()`) currently handles: `agent_state`, `job_started`, `job_completed`, `chat`, `agent_count`. New message types needed:
| Economy State | New WS message type | Matrix behavior |
|---|---|---|
| `created` | `invoice_request` (type=eval) | Show eval cost in chat panel |
| `evaluating` | `agent_state` (state=active) | Agent glows active |
| `awaiting_work_payment` | `invoice_request` (type=work) | Show work cost + pay button |
| `work_paid` | `invoice_settled` | Log in chat, start work animation |
| `completed` | `job_completed` (extended) | Show result + actual cost + refund |
| `rejected` | `job_completed` (with error) | Log rejection |
New message schemas for `PROTOCOL.md`:
```jsonc
// Server → Client: invoice for operator to pay
{
"type": "connection",
"status": "connected",
"agents": {
"timmy": { "name": "Timmy", "role": "Main Orchestrator", "color": "#00ff41" },
"forge": { "name": "Forge", "role": "Builder Agent", "color": "#ff8c00" },
...
}
"type": "invoice_request",
"jobId": "...",
"invoiceType": "eval" | "work",
"paymentRequest": "lnbc...", // BOLT11
"amountSats": 500,
"description": "Work fee for: analyze sentiment..."
}
// Server → Client: payment confirmed
{
"type": "invoice_settled",
"jobId": "...",
"invoiceType": "eval" | "work"
}
// Server → Client: job completed with accounting
{
"type": "job_completed",
"jobId": "...",
"agentId": "alpha",
"result": "...",
"actualCostSats": 380,
"estimatedCostSats": 500,
"refundSats": 120
}
```
This lets the backend define which agents exist without changing frontend code.
Note: field names use camelCase to match Replit's refactored code (`agentId`, not `agent_id`).
---
## Mode 2: Agent Token Economy Integration
## WebSocket Path Alignment
### What the Token Economy Is
> Tracked in: `the-matrix#10`
From the Timmy Time spec:
- Agents earn sats for completing tasks
- Agents spend sats to use tools, call APIs (L402 gating)
- The human operator sets budgets and approves high-cost operations
- Lightning Network is the settlement layer
- Per-agent sat limits and daily global caps
Three systems, three WS paths:
- Matrix (Replit refactor): `import.meta.env.VITE_WS_URL` — fully configurable
- Economy PR #20: SSE on HTTP (no WS path yet); issue #2/#15 track WS
- Dashboard: `ws://tower:8080/ws`
### Integration Points
**Resolved:** Replit's refactor uses `VITE_WS_URL` env var. No hardcoded path. Set it at build/dev time:
#### 1. Economy Data in Agent State
```bash
VITE_WS_URL=ws://tower:8080/ws/matrix npm run dev
```
Extend the `agent_state` message to include economic fields:
If empty, the Matrix runs disconnected (no mock — just shows OFFLINE).
```json
---
## Phase 1: WebSocket Gateway
> Tracked in: `the-matrix#8`, `dashboard#325`
**Owner:** Dashboard team (Kimi or Claude)
**Effort:** 2–3 days
The dashboard's `infrastructure/events/bus.py` already provides:
- Typed `Event` dataclass with source, type, data, timestamp
- Async pub/sub with wildcard matching (`agent.task.*`)
- SQLite persistence + replay
- Existing subscribers: gitea_adapter, time_adapter
What's needed: A WebSocket endpoint that bridges the EventBus to Matrix clients.
```python
# src/infrastructure/ws_gateway/matrix_handler.py
@router.websocket("/ws/matrix")
async def matrix_ws(websocket: WebSocket):
await websocket.accept()
# Send agent registry on connect
await websocket.send_json({
"type": "connection",
"status": "connected",
"agents": get_agent_registry()
})
# Subscribe to relevant bus events
@bus.subscribe("agent.*")
async def on_agent_event(event: Event):
await websocket.send_json(translate_to_matrix(event))
@bus.subscribe("task.*")
async def on_task_event(event: Event):
await websocket.send_json(translate_to_matrix(event))
# Handle inbound Matrix messages
async for msg in websocket.iter_json():
await handle_matrix_message(msg)
```
The `translate_to_matrix()` function maps EventBus events → Matrix protocol messages (using the camelCase field names from Replit's codebase). The `handle_matrix_message()` function routes Matrix commands → internal APIs.
### Cognitive State Signal
Dashboard issue #325 asks for Timmy's cognitive state to drive his 3D avatar behavior. This feeds into the gateway:
```jsonc
// New event on the bus: timmy.cognitive_state
{
"type": "agent_state",
"agent_id": "forge",
"state": "working",
"glow_intensity": 0.8,
"economy": {
"balance_sats": 15420,
"earned_today": 8500,
"spent_today": 3200,
"daily_limit": 50000,
"pending_invoices": 2
}
"agentId": "timmy",
"state": "deep_focus",
"focusTopic": "analyzing PR #315",
"engagement": 0.9,
"coherence": 0.85
}
```
#### 2. Status Tab — Economy Section
Add economy rows to the Status tab when data is present:
| Label | Value |
|---|---|
| BALANCE | ⚡ 15,420 sats |
| EARNED TODAY | ⚡ 8,500 |
| SPENT TODAY | ⚡ 3,200 |
| DAILY LIMIT | ⚡ 50,000 |
| PENDING | 2 invoices |
Style the lightning bolt in `--matrix-warning` (orange) for visual pop.
#### 3. Task Cost Display
Extend task items in the Tasks tab to show cost:
```json
{
"task_id": "...",
"title": "Run sentiment analysis",
"status": "pending_approval",
"priority": "normal",
"estimated_cost_sats": 500,
"agent_id": "seer"
}
```
The Approve/Veto buttons become more meaningful when you see: "This task will cost ⚡ 500 sats."
#### 4. Core System Panel — Treasury
The Core (system status) panel should show aggregate economy:
| Label | Value |
|---|---|
| TREASURY | ⚡ 142,800 sats |
| DAILY BURN | ⚡ 12,400 |
| DAILY EARN | ⚡ 18,200 |
| NET TODAY | +⚡ 5,800 |
| INVOICES PAID | 47 |
| INVOICES PENDING | 3 |
#### 5. Visual Economy Indicators in 3D World
**Agent glow color shift:** Agents burning through their budget fast could shift from green → orange → red glow. This is already supported — `glow_intensity` controls brightness, and the agent's base color is set per-agent. Add a `budget_stress` field (0.0–1.0) that blends the glow toward red.
**Task object color:** Floating task cubes near agents could be colored by cost:
- Free / low cost → green
- Medium cost → orange
- High cost → red
- Pending approval → pulsing white
**Connection lines:** When agent A is paying agent B (inter-agent economy), the connection line between them could pulse brighter, with small particles flowing along it to represent sat flow.
#### 6. New Message Types
| Message | Direction | Purpose |
|---|---|---|
| `economy_update` | Server → Client | Agent balance changed |
| `invoice_created` | Server → Client | Agent generated a Lightning invoice |
| `invoice_paid` | Server → Client | Invoice settled |
| `budget_alert` | Server → Client | Agent approaching daily limit |
| `set_budget` | Client → Server | Operator adjusts agent's daily limit |
---
## Implementation Phases
## Phase 2: Frontend Integration
### Phase 1: Live WebSocket (Backend Team)
> Tracked in: `the-matrix#7`
**Owner:** Replit (primary), Perplexity (review)
**Effort:** 1–2 days
- [ ] Create `ws_gateway.py` with Matrix protocol router
- [ ] Wire to existing task and agent state events
- [ ] Agent chat forwarding (Ollama responses streamed as `agent_message`)
- [ ] Handshake with dynamic agent registry
**Blocked on:** Phase 1 (need a real WS endpoint to connect to)
### Phase 2: Frontend Switch (Matrix Repo)
**Effort:** 1 day
- [ ] Create `js/websocket-live.js` with same interface as mock
- [ ] Create `js/config.js` with endpoint config + mock fallback
- [ ] Update `js/main.js` to use config
- [ ] Test with backend WebSocket
Replit's refactor already handles most of this:
- `websocket.js` connects to `VITE_WS_URL` with auto-reconnect
- `handleMessage()` dispatches by `msg.type`
- `setAgentState()` drives 3D avatar glow/animation
### Phase 3: Economy Display (Both)
What remains:
- Add `invoice_request` / `invoice_settled` / extended `job_completed` handlers to `handleMessage()`
- Expose economy data to `ui.js` for payment UI
- Add connection handshake → populate agent registry from server (not just `AGENT_DEFS`)
---
## Phase 3: Economy Display
> Tracked in: `the-matrix#15`, `#17`
**Owner:** Perplexity (this repo) or Replit
**Effort:** 2–3 days
- [ ] Backend: Add economy fields to agent_state and task messages
- [ ] Frontend: Extend Status tab with economy rows
- [ ] Frontend: Add cost display to task items
- [ ] Frontend: Extend Core panel with treasury view
- [ ] Style lightning amounts with ⚡ prefix and orange accent
**Blocked on:** Phase 1 (need real data)
### Phase 4: 3D Economy Visuals (Matrix Repo)
**Effort:** 2–3 days
- [ ] Budget stress → glow color blending in `agents.js`
- [ ] Task object coloring by cost
- [ ] Sat flow particles on connection lines
- [ ] Budget alert toast/notification overlay
### Status Panel — Economy Section
### Phase 5: Full Economy Controls (Both)
Extend `ui.js updateUI()` to show economy data when available:
```
┌─────────────────────────────────────────┐
│ BALANCE ⚡ 15,420 sats │
│ EARNED TODAY ⚡ 8,500 │
│ SPENT TODAY ⚡ 3,200 │
│ DAILY LIMIT ⚡ 50,000 │
│ PENDING 2 invoices │
└─────────────────────────────────────────┘
```
### Chat Panel — Payment Flow
Payment events render in the existing chat panel via `appendChatMessage()`:
```
[SYS] JOB a4f2... eval invoice: ⚡10 sats
[SYS] JOB a4f2... eval paid — evaluating...
[ALPHA] Task looks feasible. Work cost: ⚡500 sats
[SYS] JOB a4f2... work invoice: ⚡500 sats [PAY]
[SYS] JOB a4f2... work paid — executing...
[ALPHA] Analysis complete. Actual cost: ⚡380 sats, refund: ⚡120 sats
```
### Core Panel — Treasury
New DOM element in `index.html`, rendered by `ui.js`:
```
┌─────────────────────────────────────────┐
│ TREASURY ⚡ 142,800 sats │
│ DAILY BURN ⚡ 12,400 │
│ NET TODAY +⚡ 5,800 │
│ SESSION BALANCE ⚡ 2,340 │
└─────────────────────────────────────────┘
```
---
## Phase 4: 3D Economy Visuals
> Tracked in: `the-matrix#13`, `#15`
**Owner:** Perplexity or Replit
**Effort:** 2–3 days
- [ ] Set budget from panel UI
- [ ] Invoice approval flow in Tasks tab
- [ ] Transaction history in Memory tab (reuse memory UI for econ log)
- [ ] Emergency kill switch — freeze all agent spending from Core panel
**Blocked on:** Phase 3
### Budget Stress Glow
In `agents.js` — the Agent class `update()` method already blends `emissiveIntensity` based on state. Extend to include `budgetStress` (0.0–1.0):
- 0.0: normal agent color
- 0.5: shifts toward warning orange
- 1.0: emergency red
### Sat Flow Particles
On `invoice_settled` events, animate particles along connection lines from payer → payee. Small glowing dots traveling the path. Use `effects.js` pattern (BufferGeometry + Points).
### Dynamic Agent Hot-Add (#12)
When `agent_joined` fires via WS, create a new Agent instance from the message data and call `scene.add()`. The `AGENT_DEFS` array becomes the default set; live agents can extend it.
---
## Phase 5: Session Mode
> Maps to: `economy/sessions.ts`
**Owner:** Both repos
**Effort:** 2–3 days
**Blocked on:** Phase 2
The economy supports session mode: deposit sats upfront, get a macaroon, chat freely until balance runs out. This is the iPad-native experience.
### Flow in The Matrix
1. First connection: "Deposit ⚡100–10,000 sats to start a session" prompt
2. Show Lightning invoice QR (render in chat panel or overlay)
3. On payment: session activates, macaroon stored in localStorage
4. Chat freely — each message deducts from session balance
5. Balance display in status panel, warning at minimum threshold
6. Session expiry countdown
---
## File Changes Summary
### Matrix repo (`replit/the-matrix` → merged to `perplexity/the-matrix`)
| File | Change | Phase | Issue |
|---|---|---|---|
| `js/websocket.js` | Add economy message handlers | 2 | #7, #9 |
| `js/ui.js` | Economy rows, payment chat, treasury panel | 3 | #15, #17 |
| `js/agents.js` | Budget stress glow, dynamic hot-add | 4 | #12, #13, #15 |
| `js/effects.js` | Sat flow particles | 4 | #13 |
| `js/agent-defs.js` | May become runtime-extensible for hot-add | 4 | #12 |
| `index.html` | Treasury DOM, PWA manifest link | 3 | #5, #17 |
| `PROTOCOL.md` | Add economy message types, cognitive state | 2 | #9 |
### Dashboard repo (`rockachopa/Timmy-time-dashboard`)
| File | Change | Phase |
|---|---|---|
| `js/websocket.js` | Keep as-is (mock mode) | — |
| `js/websocket-live.js` | New — real WebSocket client | 2 |
| `js/config.js` | New — connection config | 2 |
| `js/main.js` | Import config, conditional WS | 2 |
| `js/ui.js` | Economy rows in Status, cost in Tasks, treasury in Core | 3 |
| `js/agents.js` | Budget stress glow, task cost coloring, sat flow particles | 4 |
| `style.css` | Economy styling, lightning bolt accent | 3 |
| `PROTOCOL.md` | Add economy message types | 3 |
| `backend/ws_gateway.py` | New — WebSocket gateway (Timmy Time repo) | 1 |
| `src/infrastructure/ws_gateway/` | New — Matrix protocol bridge | 1 |
| `src/timmy/cognitive_state.py` | New — observable cognitive state | 1 |
### Economy repo (`replit/token-gated-economy`)
| File | Change | Phase |
|---|---|---|
| WebSocket endpoint | New — payment events over WS | 1 |
| `lib/event-bus.ts` | Emit to Matrix-format WS channel | 1 |
---
## Open Questions
## Open Decisions
1. **Serving model:** Should The Matrix be served by the FastAPI backend (as a static route), or stay standalone and connect remotely? Standalone is cleaner for iPad PWA deployment but needs CORS config on the backend WS.
2. **Authentication:** The Matrix WS should require auth before accepting commands. Options:
- Token in WS URL query param: `ws://tower:8080/ws/matrix?token=...`
- L402 auth (pay-to-connect, on-brand with the sovereignty model)
- Simple shared secret for now, upgrade to L402 later
3. **Multi-operator:** If multiple iPads connect, should they see each other's cursors/selections? Not needed for v1 but worth designing for.
4. **Agent hot-add:** When a new agent joins the swarm at runtime, the backend should push an `agent_joined` event so The Matrix can spawn the 3D avatar dynamically without reload.
| # | Question | Options | Status |
|---|---|---|---|
| 1 | WS path | `VITE_WS_URL` env var — resolved | ✓ Resolved by Replit |
| 2 | Auth | Token param for now, L402 later | Proposed in #11 |
| 3 | Which backend? | Dashboard (Tower) or Economy (Replit) | Depends on deployment |
| 4 | Multi-operator | Defer to v2 | — |
| 5 | Agent hot-add | Backend pushes `agent_joined` event | Tracked in #12 |
| 6 | Session vs per-task | Support both, session default on iPad | Phase 5 |
| 7 | Fork merge cadence | Feature-boundary PRs, not every commit | Proposed above |
---
## Dependencies
- Timmy Time backend must expose a WebSocket endpoint speaking the protocol in `PROTOCOL.md`
- Lightning integration (LND or CLN) must expose balance/invoice APIs to the backend
- The Matrix has zero backend dependencies — it's pure static files + WebSocket
---
*This document lives in the `perplexity/the-matrix` repo. Backend implementation tasks should be tracked in the main Timmy Time repo.*
*This document lives in `perplexity/the-matrix` (canonical) and is the source of truth for how the three systems connect. Replit's fork reads this doc; code PRs flow from `replit/the-matrix` → `perplexity/the-matrix`. Backend tasks are cross-referenced to their respective repos via issue numbers.*