Discord-bot
discord-bot (deployment: HAI Discord Bot) is HAI's Discord surface — talks to users in voice channels (real-time VAD → ASR → LLM → TTS) and text channels / DMs (multimodal chat with image, audio, and video attachments). Runs HAI, Markus's robot dog and synthetic twin.
Stack: Python (3.13 in the deployed image). Private repo. Built on the voicepipe orchestrator (consumed as a path dep).
Features
- Voice channels —
/join; VAD, no push-to-talk. DAVE / E2EE via py-cord PR #3159 + the op-15media_sink_wantspatch (_voice_patches.py). - Text + DM chat — reply to @HAI or use
/ask; per-channel conversation memory survives restarts (bot_history.json). - Multimodal attachments — images, audio (mp3/wav/ogg/flac/m4a auto-transcoded), video, text files passed to the model as OpenAI-style parts.
- Reply-aware context — @HAI in a reply injects the replied-to message as its own user turn.
- Voice messages —
/ttssynthesizes via Qwen3-TTS, posts a native Discord voice message (Ogg/Opus + waveform), WAV fallback. - Voice cloning —
/cloneuploads a reference sample; survives restarts via theVOICE_CLONE_DIRPVC. - Tool-calling — emoji reactions, sticker / URL vision lookup, sticker send, gated image/video generation, gated web search via SearXNG, SSRF-guarded URL fetch, YouTube search + audio via yt-dlp.
- Role-scoped capability gates —
image_gen,web_search,audio_genread env-seed + runtime overlay (editable via/perms grant|revoke). Roles beat names so renames don't drop access. - Activity surface —
/showpushes images, videos, audio, URLs, markdown, or text to a single browser surface (aiohttp WebSocket hub); same backend powers an embeddable Discord Activity.
Hardening
Per-user rate limiting; text/author-name sanitization; sized attachment cap; reasoning-content DM-only mode; default context pruning; graceful shutdown; atomic state writes; SSRF allowlist on vision-tool fetches; HMAC-signed SearXNG image proxy.
Bulletproof voice pipeline — three-layer defence against the libopus silk_resampler hang class: Kubernetes livenessProbe on /healthz, AudioReader.callback AttributeError guard + Opus payload pre-validation, TTS calls wrapped in wait_for(15s). Every voice-path error carries stage=... attribution=... so the right layer gets fixed.