Runtime

Architecture of the hosted Aomi runtime. Session manager lifecycle, message processing, tool execution coordination, SSE streaming, and event handling.

The Aomi runtime is the hosted execution environment that manages sessions, processes messages, and coordinates tool execution.

Session Manager

The session manager handles the full session lifecycle:

• Create: new conversation sessions with unique IDs
• Load: three-tier loading: memory cache → database → create new
• Persist: messages saved as the conversation progresses
• Cleanup: background task removes stale sessions

Session Lifecycle

User Message → Session Manager → Message Processing → Tool Execution → Response Streaming
                                                                    ↓
                                                          State Persistence

Reconstructor

The reconstructor handles session reconciliation. It restores session state from persisted data when a user returns to an existing conversation.

History Backend

Pluggable persistence layer for session history:

• In-memory for development
• Database-backed for production
• Custom persistence backends

Background Tasks

Task	Frequency	Purpose
Title generation	On session create	Auto-generate conversation titles
Session cleanup	Periodic	Remove expired/stale sessions

Streaming & Events

Chat replies come back as a JSON session snapshot, not a token stream. To follow work in real time, the runtime exposes a Server-Sent Events (SSE) channel at GET /api/updates. It pushes one JSON event per line, and each event carries a type and session_id:

data: {"type":"title_changed","session_id":"550e8400-...","new_title":"ETH price check"}

data: {"type":"tool_update","session_id":"550e8400-..."}

data: {"type":"tool_complete","session_id":"550e8400-..."}

The event types are title_changed, tool_update, tool_complete, and system_notice. Clients usually pair this channel with polling GET /api/state to pick up new messages. See the API Reference for headers and a consumer example.

App Isolation

Each App runs in an isolated environment with its own tool registry, model configuration, and API key scope. No cross-App data sharing.

Performance

• Sub-100ms message processing for cache hits
• Concurrent session handling
• Automatic scaling based on load

Next Steps

• API Reference: HTTP API documentation
• Sessions: session management details
• Architecture: platform overview