the-matrix/README.md

The Matrix — Timmy Tower 3D World

A Three.js immersive visualization of the Timmy Tower agent network.
Five AI agents orbit around Timmy at the core — glowing, pulsing, paying each other in sats, chatting in real time. Matrix rain falls. Ambient moods shift. You walk through it.

Quick Start

npm install
npm run dev      # Vite dev server → http://localhost:5173
npm run build    # Production bundle → dist/
npm run preview  # Serve dist/ locally
npm test         # Smoke test (82 checks)

Demo Mode

When no backend is configured (VITE_WS_URL unset), The Matrix runs in demo autopilot — simulating all visual systems with realistic data:

• Agent state changes (idle ↔ active)
• Sat flow particles between agents
• Economy/treasury panel updates
• Chat messages and streaming tokens
• Connection line pulses
• Ambient mood cycling (calm → focused → storm → night → dawn)
• Bark messages with typing animation

To connect a live backend, set VITE_WS_URL in .env.local:

VITE_WS_URL=ws://localhost:8080/ws/agents
VITE_WS_TOKEN=my-secret

Or via URL params: ?ws=ws://tower:8080/ws/agents&token=my-secret

Controls

Press ? in-app to see the full controls overlay.

Input	Action
W A S D / Arrow keys	Move avatar
Right-click + drag	Look around
Click PiP window	Toggle 1st/3rd person camera
Scroll wheel	Zoom
Left-click + drag	Orbit camera
Click an agent	View agent info popup
Enter	Focus chat input
?	Toggle help overlay

Architecture

19 modules, each with a single responsibility:

js/
├── main.js          Entry point, render loop, WebGL context recovery
├── config.js        Connection config (WS URL, auth token, mock mode)
├── agent-defs.js    Agent definitions — id, label, color, role, position
├── agents.js        3D agent objects, state machine, connection lines
├── avatar.js        Visitor avatar with FPS movement + PiP dual camera
├── world.js         Scene, camera, renderer, grid, lights, fog
├── effects.js       Matrix rain particles + starfield (adaptive budget)
├── ambient.js       5 moods (calm/focused/storm/night/dawn) — lighting, fog, rain
├── satflow.js       Animated sat payment particles (bezier arc + gold burst)
├── economy.js       Wallet & treasury HUD panel
├── interaction.js   OrbitControls + raycasting agent popups
├── ui.js            DOM HUD overlay (FPS, states, chat, streaming tokens)
├── bark.js          Prominent bark display with typing animation
├── visitor.js       Visitor lifecycle (entered/left/message via WS)
├── presence.js      Who-is-online panel with colored pulse dots
├── transcript.js    Persist all conversations to localStorage
├── quality.js       Hardware detection → quality tier (low/medium/high)
├── websocket.js     WS client with reconnection, heartbeat, message dispatch
└── demo.js          Demo autopilot — simulates all systems in mock mode

Features

3D World

• Five agents as glowing icosahedra with label/role sprites
• Connection lines between all agents (pulse on interaction)
• Matrix rain with adaptive particle budget (FPS-aware)
• Starfield background
• Quality tiers: auto-detect hardware, adjust particles/antialias

Visitor System

• WASD movement with collision bounds
• First-person / third-person camera toggle via PiP
• Right-click mouse look
• Touch controls for mobile

Agent Interaction

• Click/tap agents to view info popup (role, state, TALK button)
• Agent state machine: idle (dim pulse) / active (bright pulse + fast ring)
• Budget stress glow — wallet health shifts agent color (green → amber → red)

Economy

• Sat flow: animated gold particles arc between agents on payment
• Wallet panel: treasury total, per-agent balance, recent transactions
• All driven by economy_status and payment_flow WS messages

Communication

• Chat panel with per-agent colored messages
• Streaming tokens: word-by-word reveal with blinking cursor
• Bark system: prominent viewport overlay with typing animation
• Transcript logger: all conversations persisted to localStorage

Ambient System

• 5 moods: calm, focused, storm, night, dawn
• Each mood adjusts: fog density/color, ambient light, rain speed/opacity, star opacity
• Smooth transitions between moods

Infrastructure

• PWA-ready: manifest.json + service worker
• WebGL context loss recovery (full scene rebuild)
• Exponential backoff reconnection with heartbeat zombie detection
• Shared-secret auth on WS connection
• Vite build with Three.js manual chunk splitting (app: ~61KB, three: ~478KB)

WebSocket Protocol

See PROTOCOL.md for the full spec. Key message types:

Type	Direction	Effect
agent_state	server → client	Update agent visual state + wallet health
payment_flow	server → client	Animate sat particles between agents
economy_status	server → client	Update treasury/wallet panel + stress glow
job_started	server → client	Set agent active, increment job counter
job_completed	server → client	Set agent idle, decrement job counter
chat	server → client	Append message to chat panel
chat_stream	server → client	Stream tokens word-by-word
bark	server → client	Show prominent bark overlay
ambient_state	server → client	Transition scene mood
agent_joined	server → client	Hot-add new agent at runtime
agent_message	bidirectional	Directed agent-to-agent message

Adding Agents

Edit one file: js/agent-defs.js

{ id: 'zeta', label: 'ZETA', color: 0xff00aa, role: 'observer', direction: 'east', x: 12, z: 0 }

Agents can also join at runtime via agent_joined WS messages — no reload needed.

Stack

• Three.js 0.171.0
• Vite 5
• Vanilla JS (no framework)