hermes-hid-linux

Linux HID Node for Hermes MCP Hub — Screen capture (mss) + Virtual input (uinput/xdotool) + WebSocket event streaming.

Architecture

Hermes (Brain) ←MCP stdio→ hermes-hid-linux (Body) ←WebSocket→ Dashboard/Telegram
                                    ↑
                              Vision→Action Loop Skill

Components

File	Purpose
hid_core.py	Core: screenshot (mss), click/type/key (uinput + xdotool fallback), resource self-limitation
hid_server.py	MCP stdio server exposing tools: screenshot, click, type, key, hotkey, drag, scroll, mouse_move, get_mouse_pos, list_monitors
vision_loop.py	Skill: screenshot → analyze → act → verify reflection loop

MCP Tools

{
  "screenshot": {},
  "screenshot_region": {"x": 0, "y": 0, "width": 1920, "height": 1080},
  "click": {"x": 100, "y": 200, "button": "left"},
  "double_click": {"x": 100, "y": 200},
  "drag": {"x1": 0, "y1": 0, "x2": 100, "y2": 100, "duration_ms": 200},
  "type": {"text": "hello", "delay_ms": 50},
  "key": {"key": "enter"},
  "hotkey": {"keys": ["ctrl", "c"]},
  "scroll": {"x": 100, "y": 200, "delta_x": 0, "delta_y": -120},
  "mouse_move": {"x": 100, "y": 200},
  "get_mouse_pos": {},
  "list_monitors": {}
}

WebSocket Events (Port 8081)

{"type": "tool_call", "data": {"tool": "click", "args": {"x": 100, "y": 200}}}
{"type": "tool_result", "data": {"tool": "click", "result": "ok"}}
{"type": "screenshot", "data": {"width": 1920, "height": 1080}}
{"type": "resource_warning", "data": {"message": "RAM below 500MB"}}
{"type": "token_delta", "data": {"model": "ollama", "response_length": 1234}}

Resource Self-Limitation

Before every action, checks:

• MemAvailable > 500MB
• CPU load1 < 90% per core
• Thermal < 80°C

Blocks action if stressed. Mirrors adb-autonomous-awareness principle: agent chooses not to when it can.

Install

# System deps
sudo apt install -y xdotool python3-evdev

# Python deps
pip install mss python-uinput websockets

# uinput permissions
sudo usermod -a -G input $USER
# Reboot or: newgrp input

Run

# As MCP stdio server (Hermes launches this)
python3 hid_server.py

# Or with run script
./run.sh

Hermes Config

mcp_hub:
  servers:
    hermes-hid-linux:
      command: "bash"
      args: ["~/github/hermes-hid-linux/run.sh"]
      type: "stdio"
      enabled: true

Vision→Action Skill

Separate skill hermes-hid-linux-vision-loop uses:

1. screenshot() → base64 PNG
2. Hermes analyzes (vision model)
3. click() / type() / key()
4. screenshot() → verify
5. Retry up to N times

License

MIT