AI Agent

Overview

AI Agent powers phone AI (text and voice), optional Live Chat assistance, and a public Runs API for backends. Create an AI Agent API token with permission ai:invoke so your server can only invoke AI—not email, chat minting, or billing.

Get started

1. Enable AI Agent in Dashboard → Billing. For phone speak/voice also enable Business Phone.
2. Open Dashboard → AI to configure the agent name and automatic-agent behavior.
3. Optional: enable Live Chat auto-replies under chat AI settings (requires AI Agent entitlement).
4. Dashboard → Developers → create an API token. Product: AI Agent. Permission: ai:invoke. Environment: live or test. Optional IP allowlist.
5. Copy the secret once. Store it only on your server. Fetch third-party data with that system's credentials, then call POST /api/v1/ai/runs with input + context.

API tokens

An API token lets an agent access AI Agent only—not Live Chat session minting, Business Email, or billing. Prefer ai:invoke alone for invoke jobs.

Authorization: Bearer umk_live_ai_<prefix>_<secret>
# or
X-API-Key: umk_live_ai_<prefix>_<secret>

API base: https://umneyconnect.com/api

Full token guide: /developers/api-tokens

Concepts

Third-party integration

Connect does not crawl Nest, CRM, or other SaaS. Your backend authenticates to those systems, fetches records, and injects facts into AI via context. The AI Agent token only authorizes Connect AI — never the third party.

1. Issue an AI Agent token (ai:invoke) and store it on your server only.
2. In your Nest/CRM backend, authenticate to the third-party system with that system's own credentials.
3. Fetch the records the user needs (order, booking, profile, etc.) and serialize a short factual summary.
4. Call POST /api/v1/ai/runs with input (user question) and context (those facts). Optional metadata for correlation IDs only.
5. Return outputText to your client. Never embed the Connect token in the mobile/browser app.

# Your server
CONNECT_API_BASE=https://umneyconnect.com/api
AI_API_KEY=umk_live_ai_…

# 1) Fetch from third party with THEIR credentials
# order = await nestCrm.getOrder(orderId)

# 2) Invoke Connect AI with injected context
curl -X POST "$CONNECT_API_BASE/v1/ai/runs" \
  -H "Authorization: Bearer $AI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "text",
    "input": "Where is my order?",
    "locale": "en",
    "context": "Order ORD-123 status=shipped carrier=DHL tracking=JD014…",
    "metadata": { "orderId": "ORD-123", "source": "nest-safety" }
  }'

# 3) Response (succeeded)
# {
#   "id": "…",
#   "status": "succeeded",
#   "outputText": "…",
#   "input": "Where is my order?",
#   "context": "Order ORD-123 …",
#   "metadata": { "orderId": "ORD-123", "source": "nest-safety" }
# }

Dashboard AI API (available now)

Uses the signed-in workspace session (JWT). Phone speak/voice require both Business Phone and AI Agent entitlements. Models run on Workers AI (text, Whisper STT, Aura TTS).

{
  "text": "Where is my order?",
  "locale": "en",
  "automaticAgent": true,
  "context": "Optional third-party facts your backend already fetched"
}

Response: { text, done: true }. Empty text defaults to a greeting prompt. Inference budget ~30s; misconfigured AI binding returns 503. sessionId may appear on the wire but is not used for persistence today.

Content-Type: text/event-stream

# One SSE event with the full result (not token-by-token streaming)
data: {"text":"…","done":true}

{
  "text": "optional if audioBase64 set",
  "audioBase64": "PCM s16le mono base64",
  "sampleRate": 24000,
  "locale": "en"
}
// Response: { text, audioBase64, done: true }
// Pipeline: STT (if audio) → generate → TTS
// Neither text nor audio → { text: "", audioBase64: "", done: true }

Voice WebSocket (dashboard JWT)

Upgrade to WebSocket at /api/ai/voice/ws?token=<dashboard JWT>. Requires AI configured on the platform. Pipeline asserts Business Phone + AI Agent (same as REST voice). Duplex turns: client sends audio or text; server returns text, audio, then done.

{ "type": "audio" | "text" | "interrupt" | "config", "data"?: "…" }

{ "type": "audio" | "text" | "done" | "error", "data"?: "…", "done"?: true }

Live Chat AI

Enabling Live Chat auto-reply and inbox suggestions requires an active AI Agent entitlement only (Business Phone is not asserted on these chat paths). Session minting and chat REST still use Live Chat tokens—see /developers/live-chat.

{
  "enabled": true,
  "mode": "when_no_agent_online",
  "businessProfile": "We sell …",
  "handoffKeywords": ["human", "agent", "person"],
  "maxRepliesPerConversation": 20
}

Public AI API (token — live)

Requires an AI Agent umk_* token with ai:invoke. Runs execute synchronously today (status succeeded or failed on the create response). Connect merges workspace FAQ/terms with optional caller context — it does not crawl third-party systems.

curl -X POST "https://umneyconnect.com/api/v1/ai/runs" \
  -H "Authorization: Bearer $AI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "mode": "text",
    "input": "Where is my order?",
    "locale": "en",
    "context": "Order ORD-123 status=shipped tracking=JD014",
    "metadata": { "orderId": "ORD-123" }
  }'
# Response includes id, status, outputText

Errors and limits

• 403 — missing Business Phone or AI Agent entitlement (phone speak/voice/WS), wrong product token, or missing ai:invoke
• 403 — AI Agent missing when enabling chat auto-reply or requesting inbox suggestions (Business Phone not required on those paths)
• 502 — inbox suggestion returned empty AI output
• 503 — AI binding not configured, or inference timeout after retry
• Empty voice body (REST) — returns { text: '', audioBase64: '', done: true }
• Inbox ai-suggestion: 20 requests/minute
• Public /api/v1/* rate limit: 600 requests/minute per key when those handlers exist
• Text inference budget ~30s; STT ~20s; TTS ~30s (platform timeouts)

Agent recipe

# 1. Enable AI Agent in Billing (Business Phone also required for phone speak/voice)
# 2. Developers → create token → product "AI Agent" → ai:invoke
CONNECT_API_BASE=https://umneyconnect.com/api
AI_API_KEY=umk_live_ai_…

# 3. Fetch third-party data with that system's credentials (Nest/CRM/…)
# 4. POST /v1/ai/runs with input + context
# 5. Or JWT dashboard: POST /api/tenants/{id}/ai/speak with context
# 6. Never embed the token in clients
# 7. Connect does not crawl third parties — you inject context

Related services

Live Chat session minting and webhooks: /developers/live-chat. Business Calling (SIP/softphone): /developers/business-calling. AI Agent does not replace either—it adds speak/voice and optional chat assistance on top of Business Phone and Live Chat.