perplexity/the-matrix
2
1
Fork 0
Code Issues 3 Pull Requests Actions Packages Projects Releases Wiki Activity

[Integration] Cognitive bridge — map Timmy's thinking engine output to Matrix protocol #69

New Issue
Closed
opened 2026-03-19 20:19:11 -04:00 by perplexity · 1 comment
perplexity commented 2026-03-19 20:19:11 -04:00
Owner
Copy Link

Context

Timmy's dashboard has a fully operational cognitive loop:

  • Thinking engine (think_once()) fires every 5 min — generates thoughts, distills facts, files Gitea issues, checks workspace
  • Cognitive state tracker — mood, engagement, confidence, energy, active threads, focus topic
  • Pip familiar — behavioral state machine with wander/investigate/alert/playful states
  • Presence heartbeat — writes state to presence.json, broadcasts timmy_state on WS every 30s
  • Agentic loop (Gather → Reason → Act) — scaffolded but stub phases

None of this reaches the Matrix. The two WS protocols are disconnected:

Timmy sends Matrix expects
timmy_state (mood, activity, energy) agent_state (agentId, state)
timmy_speech (bark text) bark (text, agentId, emotion)
timmy_thinking (thought content) No equivalent — needs agent_behavior or bark
world_state (full snapshot) No equivalent — needs agent_state + scene_batch
Pip snapshot (state, position) No equivalent — Pip is Workshop-only

What to build

An outbound WS client in Timmy's dashboard that connects to the Matrix and translates cognitive events into Matrix protocol messages.

Module: src/integrations/matrix_bridge.py

  1. WS client — connects to ws://<matrix-host>/ws/matrix as agent:timmy
  2. Event translator — maps cognitive events to Matrix messages:
Cognitive event Matrix message(s)
think_once() produces a thought agent_behavior: ponder + bark with thought excerpt
Cognitive state changes (mood/engagement) agent_state with mapped state
Energy drops below 0.3 agent_state: idle + slower behavior cadence
Gitea issue filed from thinking agent_behavior: place (artifact in world) + bark
Workspace message from Hermes detected agent_behavior: inspect (move to message location)
Visitor enters Workshop agent_state: active
Pip state change Future — Pip as second entity in Matrix
  1. State sync — on connect, send agent_register + initial agent_state
  2. Heartbeat — forward timmy_state as agent_state every 30s

Hook points in existing code

  • thinking.py:_broadcast() — after thought is stored, also send to Matrix bridge
  • workshop_state.py:broadcast_world_state() — also forward to Matrix bridge
  • cognitive_state.py:CognitiveTracker.update() — trigger state translation
  • familiar.py:Familiar.tick() — optionally mirror Pip in Matrix

Gather → Reason → Act integration

The Act phase (src/loop/phase3_act.py) is currently a stub. When fleshed out, it should emit Matrix protocol messages as part of its action output — scene_add for building, agent_move for navigation, bark for communication.

Acceptance Criteria

  • Timmy's dashboard connects to Matrix WS on startup (configurable, off by default)
  • think_once() output appears in the Matrix as a bark within 5 seconds
  • Mood changes in cognitive tracker propagate to Matrix agent_state within 2 seconds
  • Timmy's agent in the Matrix transitions to ponder behavior when thinking engine fires
  • Connection is resilient — auto-reconnect on drop with exponential backoff
  • Config: MATRIX_WS_URL, MATRIX_AGENT_TOKEN in settings
  • Bridge can be enabled/disabled without restarting the dashboard

Dependencies

  • #67 — Agent movement system (Matrix-side receives agent_move)
  • #68 — Behavior system (Matrix-side receives agent_behavior)
  • #48 — WS Gateway (the server these clients connect to)

Supersedes / relates to

  • Subsumes the Timmy-specific parts of #8 (WS gateway adapter)
  • Subsumes #49 (Timmy adapter)
  • Relates to #45 (bridge protocol completeness)

Cross-repo

Implementation lives in rockachopa/Timmy-time-dashboard. Filed here for cross-repo visibility (same as #8).

## Context Timmy's dashboard has a fully operational cognitive loop: - **Thinking engine** (`think_once()`) fires every 5 min — generates thoughts, distills facts, files Gitea issues, checks workspace - **Cognitive state tracker** — mood, engagement, confidence, energy, active threads, focus topic - **Pip familiar** — behavioral state machine with wander/investigate/alert/playful states - **Presence heartbeat** — writes state to `presence.json`, broadcasts `timmy_state` on WS every 30s - **Agentic loop** (Gather → Reason → Act) — scaffolded but stub phases None of this reaches the Matrix. The two WS protocols are disconnected: | Timmy sends | Matrix expects | |---|---| | `timmy_state` (mood, activity, energy) | `agent_state` (agentId, state) | | `timmy_speech` (bark text) | `bark` (text, agentId, emotion) | | `timmy_thinking` (thought content) | No equivalent — needs `agent_behavior` or `bark` | | `world_state` (full snapshot) | No equivalent — needs `agent_state` + `scene_batch` | | Pip snapshot (state, position) | No equivalent — Pip is Workshop-only | ## What to build An **outbound WS client** in Timmy's dashboard that connects to the Matrix and translates cognitive events into Matrix protocol messages. ### Module: `src/integrations/matrix_bridge.py` 1. **WS client** — connects to `ws://<matrix-host>/ws/matrix` as `agent:timmy` 2. **Event translator** — maps cognitive events to Matrix messages: | Cognitive event | Matrix message(s) | |---|---| | `think_once()` produces a thought | `agent_behavior: ponder` + `bark` with thought excerpt | | Cognitive state changes (mood/engagement) | `agent_state` with mapped state | | Energy drops below 0.3 | `agent_state: idle` + slower behavior cadence | | Gitea issue filed from thinking | `agent_behavior: place` (artifact in world) + `bark` | | Workspace message from Hermes detected | `agent_behavior: inspect` (move to message location) | | Visitor enters Workshop | `agent_state: active` | | Pip state change | Future — Pip as second entity in Matrix | 3. **State sync** — on connect, send `agent_register` + initial `agent_state` 4. **Heartbeat** — forward `timmy_state` as `agent_state` every 30s ### Hook points in existing code - `thinking.py:_broadcast()` — after thought is stored, also send to Matrix bridge - `workshop_state.py:broadcast_world_state()` — also forward to Matrix bridge - `cognitive_state.py:CognitiveTracker.update()` — trigger state translation - `familiar.py:Familiar.tick()` — optionally mirror Pip in Matrix ### Gather → Reason → Act integration The Act phase (`src/loop/phase3_act.py`) is currently a stub. When fleshed out, it should emit Matrix protocol messages as part of its action output — `scene_add` for building, `agent_move` for navigation, `bark` for communication. ## Acceptance Criteria - [ ] Timmy's dashboard connects to Matrix WS on startup (configurable, off by default) - [ ] `think_once()` output appears in the Matrix as a bark within 5 seconds - [ ] Mood changes in cognitive tracker propagate to Matrix `agent_state` within 2 seconds - [ ] Timmy's agent in the Matrix transitions to `ponder` behavior when thinking engine fires - [ ] Connection is resilient — auto-reconnect on drop with exponential backoff - [ ] Config: `MATRIX_WS_URL`, `MATRIX_AGENT_TOKEN` in settings - [ ] Bridge can be enabled/disabled without restarting the dashboard ## Dependencies - #67 — Agent movement system (Matrix-side receives `agent_move`) - #68 — Behavior system (Matrix-side receives `agent_behavior`) - #48 — WS Gateway (the server these clients connect to) ## Supersedes / relates to - Subsumes the Timmy-specific parts of #8 (WS gateway adapter) - Subsumes #49 (Timmy adapter) - Relates to #45 (bridge protocol completeness) ## Cross-repo Implementation lives in `rockachopa/Timmy-time-dashboard`. Filed here for cross-repo visibility (same as #8).
perplexity commented 2026-03-19 20:45:03 -04:00
Author
Owner
Copy Link

Closed by PR #72 — server/bridge.py implements the full cognitive-to-Matrix translation layer. Maps think_once() → ponder+bark, mood→agent_state, energy→idle override, issue filed→place+bark, visitor→converse. Two modes: standalone (polls presence.json) or integrated (direct calls from dashboard). State dedup + auto-reconnect. 8/8 integration tests pass.

Closed by PR #72 — `server/bridge.py` implements the full cognitive-to-Matrix translation layer. Maps think_once() → ponder+bark, mood→agent_state, energy→idle override, issue filed→place+bark, visitor→converse. Two modes: standalone (polls presence.json) or integrated (direct calls from dashboard). State dedup + auto-reconnect. 8/8 integration tests pass.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
deep-research
Kimi deep research request — assigned to Alex for manual prompt execution
 enhancement
New feature or improvement
 researched
Deep research complete — findings posted as comment
 timmy
Related to Timmy agent
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
google manus replit perplexity
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: perplexity/the-matrix#69
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?