Long-running production side project · 2015–present
Telegram agent architecture: from commands to asynchronous workers
This bot has lived in the same group chats since 2015. It started as a single command handler; today it's a TypeScript monorepo with Lambda ingress, asynchronous workers, provider-routed model calls, tools, scoped memory and metrics. This post walks through the boundaries that appeared as load, latency and failures stopped being theoretical — and why most of them exist to protect the webhook, not the model.
The model call is the easy part
In an active group chat the hard questions sit upstream of any LLM: should the bot answer at all, how fast can the webhook acknowledge, how much context is actually worth loading, and when a provider fails at 2am — which stage do you blame? The architecture is shaped by those questions, not by the model.
- Group-chat messages use a default-ignore policy; the bot must earn each response.
- Telegram ingress acknowledges updates without waiting for model or media execution.
- History and memory are scoped by chat and loaded only for admitted agent turns.
- Provider, tool and delivery failures are recorded as separate operational events.
Webhook and worker boundaries
The Telegram-facing Lambda accepts an update, performs inexpensive routing and invokes a worker. Agent replies, activity aggregation, broadcasts, search and PNG rendering run in separate execution paths. A slow provider or failed render therefore does not extend the webhook request or block unrelated features.
What it runs on today
Reply gating and context assembly
Group-chat traffic uses a default-ignore policy. Direct mentions and deterministic address checks run before the model-based reply classifier. History, memory and tool definitions are loaded only after the message is admitted to the agent path.
- 01 Address checks
- 02 Reply gate
- 03 History + memory
- 04 Model routing
- 05 Tool execution
- 06 Telegram delivery
The typed tool registry covers web and image search, media generation, weather, code execution, history lookup, memory updates and dynamic commands. Execution order, timeout and rate-limit behavior are defined by the runtime rather than left to the model.
How it got here
Single-process command bot
The first implementation handled utility commands such as currency, weather and search in one Telegram process. This was sufficient while execution was fast and state was local.
Command registry and shared integrations
As the command surface expanded, handlers moved behind a registry with common validation, response and integration helpers. This reduced coupling between Telegram routing and feature code.
Asynchronous workload isolation
The webhook became a thin ingress Lambda. Model calls, statistics, broadcasts and media work moved to separate workers so Telegram acknowledgement time no longer depended on downstream latency.
Durable events and observable output
Chat events moved into DynamoDB, while WebSockets, search and a PNG renderer exposed operational data through Telegram and the companion application.
Controlled agent execution
The agent path added reply gating, provider routing, tool execution, memory and multimodal context without replacing deterministic ingress, timeout handling or failure telemetry.
Decisions that keep it debuggable
Isolate Telegram ingress
The Telegram-facing Lambda validates and routes an update, then invokes the relevant worker. It does not block on statistics, model calls, image generation or WebSocket fanout.
Apply a default-ignore reply policy
Direct mentions and deterministic address checks run first. Ambiguous messages reach a structured reply classifier; rejected messages never allocate model context or tools.
Schedule tools deterministically
A typed registry defines tool contracts. Rate-limited tools run sequentially, content generation waits for data-gathering tools, and every call has an explicit timeout.
Make provider failover explicit
Every model call records success, timeout or error state. The chat path can route to a configured secondary provider when the primary model fails.
Assemble context after admission
Recent history, media attachments and chat-scoped memory are loaded only after the reply policy accepts the message, reducing latency and unnecessary token use.
Record model and tool telemetry
Model and tool calls emit status, latency, model, provider and fallback source. Production failures can therefore be attributed to a specific execution stage.
What's next
The missing piece is repeatable evaluation. Metrics tell me where execution failed, but not whether an answer got better or worse after a prompt or model change. A replay corpus built from redacted production conversations would turn "feels smarter" into something I can actually measure before deploying.
- Build a replay harness from redacted production conversations.
- Track response quality, reply-gate precision, tool success and latency by feature.
- Add review and audit flows for dynamic tools and memory mutations.
- Extract the ingress, provider and tool-runtime boundaries into reusable packages.