> ## Documentation Index > Fetch the complete documentation index at: https://docs.orchagent.io/llms.txt > Use this file to discover all available pages before exploring further. # Always-On Services > Deploy agents as persistent, always-running services with automatic restarts and crash loop detection Deploy agents as **always-on services** that run continuously in the cloud. Services are ideal for Discord bots, webhook listeners, background workers, and any long-running process that needs to stay alive 24/7. ## Quick Start ```bash theme={null} # Deploy a service (secrets auto-resolved from agent's required_secrets) orch service deploy acme/discord-bot # Check status orch service list # Update env vars on a running service orch service env set KEY=VALUE # View logs orch service logs # Restart a service orch service restart # Delete a service orch service delete ``` *** ## Deploying a Service ```bash theme={null} orch service deploy \ [--name ] \ [--min-instances ] \ [--max-instances ] \ [--env KEY=VALUE] \ [--secret ] \ [--command ] \ [--arg ] \ [--workspace ] \ [--json] ``` | Flag | Description | Default | | --------------------- | -------------------------------------------------------------------------------- | ----------------- | | `--name ` | Service name | Agent name | | `--min-instances ` | Minimum running instances | `1` | | `--max-instances ` | Maximum running instances | `1` | | `--env KEY=VALUE` | Environment variable (repeatable) | — | | `--secret ` | Extra workspace secret (repeatable). Agent `required_secrets` are auto-included. | — | | `--command ` | Override entrypoint command | Auto-detected | | `--arg ` | Command argument (repeatable) | — | | `--workspace ` | Workspace slug | Current workspace | | `--json` | Output as JSON | — | Only `tool` and `agent` types (those with `runtime.command` or `loop` config) with a published code bundle are eligible for service deployment. `prompt` types (direct LLM) cannot be deployed as services. ### Example ```bash theme={null} # Agent declares required_secrets: ["DISCORD_BOT_TOKEN", "DATABASE_URL"] # Those are auto-resolved — just add extras if needed orch service deploy acme/discord-bot \ --name "prod-discord-bot" \ --env LOG_LEVEL=info ``` On success, the CLI returns the service ID, name, state, and a logs command: ``` ✓ Service deployed successfully ID: svc_abc123 Name: prod-discord-bot Agent: acme/discord-bot State: provisioning View logs: orch service logs svc_abc123 ``` ### How Deployment Works 1. You publish an agent with `runtime.command` or `loop` config (`orch publish`) 2. You run `orch service deploy org/agent` 3. The platform provisions infrastructure with your configured instance count 4. A service runner container downloads your agent bundle, installs dependencies (pip or npm), detects the entrypoint, and starts the process 5. Your service gets a public HTTPS URL (visible in `orch service info`) 6. A health endpoint at `/health` returns `200` when the service is ready 7. If the process crashes, the platform automatically restarts it 8. The agent's `required_secrets` (plus any `--secret` extras) are resolved from the workspace vault and injected as environment variables *** ## Managing Services ### List Services ```bash theme={null} # List all services orch service list # Filter by state orch service list --status running # JSON output (for scripting) orch service list --json # Return only specific fields (implies --json) orch service list --fields service_name,current_state,health_status # Paginate orch service list --limit 10 --offset 0 ``` Output shows: Name, Agent, State (color-coded), Health, Restarts (with failure count), and Deployed timestamp. Services that have been auto-paused due to crash loops display a `CRASH-LOOP` badge. ### Service Info ```bash theme={null} orch service info ``` Shows detailed information including: * Service ID, name, agent, and version * Current state and health status * Restart count and consecutive failure count * Instance configuration (min/max) * Public service URL * Deploy and last restart timestamps * Last error message (if any) * Alert webhook URL (if configured) * Recent events timeline ### View Logs The easiest way to view logs for an always-on service is through `orch logs`: ```bash theme={null} # View run history + live service logs together orch logs my-agent # View only the live service logs (skip run history) orch logs my-agent --live ``` When you filter by agent name, `orch logs` automatically detects if the agent has an always-on service and shows its live logs alongside the run history. For advanced options (time filtering, higher limits), use `orch service logs` directly: ```bash theme={null} # Fetch recent logs (default: 100 lines) orch service logs # Fetch more lines orch service logs --limit 500 # Fetch logs since a specific time orch service logs --since "2026-02-10T00:00:00Z" ``` | Flag | Description | Default | | --------------------- | ------------------------- | ----------------- | | `--limit ` | Number of log lines | `100` (max `500`) | | `--since ` | ISO 8601 timestamp filter | — | | `--workspace ` | Workspace slug | Current workspace | | `--json` | Output as JSON | — | Logs include timestamps and severity levels. Sensitive values (API keys, tokens, passwords) are automatically redacted. ### Restart a Service ```bash theme={null} orch service restart ``` Restarts the service by deploying a fresh instance. The platform automatically refreshes the signed bundle URL before restarting, which prevents issues with expired download links. ### Delete a Service ```bash theme={null} orch service delete ``` Permanently removes the service and its infrastructure. *** ## HTTP Ingress (Receiving Webhooks) Every always-on service gets a **public HTTPS URL** automatically. Use `orch service info` to find yours: ```bash theme={null} orch service info # → URL: https://orch-xxxx-my-agent.services.orchagent.io ``` ### How It Works The platform runs an HTTP gateway on port 8080 that: * Handles `GET /health` internally (platform health checks) * **Proxies all other HTTP requests to your app on port 3000** Your app should bind to **port 3000** (or set the `ORCHAGENT_HTTP_PORT` env var for a different port). The `PORT` env var is also set to this value for framework compatibility (Express, Uvicorn, etc.). ### Example — Telegram Webhook Receiver ```python theme={null} from fastapi import FastAPI, Request app = FastAPI() @app.post("/webhook/telegram") async def telegram_webhook(request: Request): data = await request.json() # Handle Telegram update... return {"ok": True} # Start with: uvicorn app:app --host 0.0.0.0 --port 3000 ``` After deploying, register the webhook URL with Telegram: ```bash theme={null} # Get your service URL orch service info # Register with Telegram curl "https://api.telegram.org/bot/setWebhook?url=/webhook/telegram" ``` ### Services That Don't Need HTTP Ingress Services that only make **outbound** connections (Discord bots using WebSocket, pollers, cron-triggered workers) don't need to bind to any port — the gateway handles health checks automatically. Your code must **not** bind to port 8080 — the platform health server uses it. Bind to port 3000 (or your configured `ORCHAGENT_HTTP_PORT`). *** ## Workspace Secrets Secrets are **automatically resolved** from the agent's `required_secrets` field. If your agent declares what it needs in `orchagent.json`, you don't need any `--secret` flags at deploy time: ```json theme={null} // orchagent.json { "name": "my-bot", "required_secrets": ["DISCORD_BOT_TOKEN", "DATABASE_URL"] } ``` ```bash theme={null} # Secrets are auto-resolved from the agent — no --secret flags needed orch service deploy acme/my-bot ``` At deploy time, the platform: 1. Reads `required_secrets` from the agent 2. Validates that all secrets exist in the workspace vault 3. Injects them as environment variables into the container If any declared secrets are missing from the vault, the deploy fails with an actionable error telling you which secrets to add. ### The `--secret` escape hatch Use `--secret` to inject **additional** secrets not declared in `required_secrets` (e.g., operational tokens, debug credentials): ```bash theme={null} orch service deploy acme/my-bot --secret MONITORING_TOKEN ``` The `--secret` flags are merged with `required_secrets` — you get both. On version updates and restarts, the platform re-merges to pick up any new `required_secrets` from the latest version. Add secrets to your workspace vault in the [dashboard](https://orchagent.io/dashboard) under Settings → Secrets, or via `orch secrets set SECRET_NAME `. *** ## Environment Variables Pass non-sensitive configuration using the `--env` flag: ```bash theme={null} orch service deploy acme/my-worker \ --env LOG_LEVEL=debug \ --env REGION=us-east-1 \ --env BATCH_SIZE=50 ``` Environment variables matching sensitive patterns (`secret`, `token`, `password`, `api_key`, `credential`, `private_key`) are **rejected** when passed via `--env`. Use `--secret` instead to reference workspace secrets securely. *** ## Updating Environment & Secrets Update environment variables and secrets on a running service without redeploying. Changes trigger an automatic restart to apply the new values. ### Environment Variables ```bash theme={null} # Set or update variables (merges with existing) orch service env set KEY=VALUE [KEY2=VALUE2 ...] # Remove variables orch service env unset KEY [KEY2 ...] # List current variables orch service env list ``` **Example:** ```bash theme={null} # Switch from staging to production config orch service env set my-bot DEPLOY_ENV=production LOG_LEVEL=warn # Remove a debug variable orch service env unset my-bot DEBUG_MODE # Check current values orch service env list my-bot ``` ### Secrets ```bash theme={null} # Attach workspace secrets (injected as env vars at runtime) orch service secret add SECRET_NAME [SECRET_NAME2 ...] # Detach secrets orch service secret remove SECRET_NAME [SECRET_NAME2 ...] ``` Secret names must match existing workspace secrets. The secret values are never exposed — only the name is stored on the service. ### API You can also update env/secrets via the PATCH endpoint: ```bash theme={null} curl -X PATCH https://api.orchagent.io/workspaces/{workspace_id}/services/{service_id} \ -H "Authorization: Bearer $ORCHAGENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "env": {"DEPLOY_ENV": "production", "LOG_LEVEL": "warn"}, "secret_names": ["DISCORD_BOT_TOKEN", "DATABASE_URL"] }' ``` | Field | Type | Description | | -------------- | ---------- | -------------------------------------------------------------- | | `env` | `object` | Full replacement of all env vars. Pass `{}` to clear all. | | `secret_names` | `string[]` | Full replacement of attached secrets. Pass `[]` to detach all. | The API uses **full replacement** semantics — whatever you send replaces the entire set. The CLI commands handle merge/unset logic for you. If the service is running, changes trigger an automatic restart (2-5 seconds of downtime). If the service is paused, changes are saved and apply on the next resume. *** ## Crash Loop Detection & Alerts The platform monitors your service for repeated restart failures and automatically intervenes to prevent runaway restarts. ### How It Works 1. Each time a service restarts and fails again, the `consecutive_restart_failures` counter increments 2. When the counter reaches the **halfway point** of your threshold, a proactive warning alert fires 3. When the counter reaches the `max_restart_failures` threshold, the service is **auto-paused** and a crash loop alert fires 4. Auto-paused services show a `CRASH-LOOP` badge in `orch service list` and `orch service info` ### Configuring Alerts Set an alert webhook URL and customize the failure threshold via the API: ```bash theme={null} curl -X PATCH https://api.orchagent.io/workspaces/{workspace_id}/services/{service_id} \ -H "Authorization: Bearer $ORCHAGENT_API_KEY" \ -H "Content-Type: application/json" \ -d '{ "alert_webhook_url": "https://hooks.slack.com/services/T00/B00/xxx", "max_restart_failures": 3 }' ``` | Field | Description | Default | Range | | ---------------------- | -------------------------------------- | ------- | ------------- | | `alert_webhook_url` | HTTPS URL to receive alert payloads | — | Must be HTTPS | | `max_restart_failures` | Consecutive failures before auto-pause | `5` | `1`–`50` | Alerts fire at two points: a proactive warning at the halfway mark (e.g., 3 out of 5 failures) and a critical alert when the threshold is reached and the service is auto-paused. ### Alert Payload ```json theme={null} { "event": "service.crash_loop", "service_id": "...", "workspace_id": "...", "service_name": "...", "restart_count": 10, "consecutive_restart_failures": 5, "threshold": 5, "auto_paused": true, "last_error": "connection timeout" } ``` ### Recovering from a Crash Loop 1. Check the logs: `orch service logs ` 2. Check the events timeline: `orch service info ` 3. Fix the underlying issue (code bug, missing secret, dependency failure) 4. Restart the service: `orch service restart ` Restarting a service resets the consecutive failure counter. *** ## Entrypoint Detection When you deploy a service without a `--command` flag, the runner automatically detects your entrypoint using the following priority: 1. **`--command` flag** — explicit command override 2. **`run_command` in orchagent.json** — define a `run_command` field in your manifest 3. **Auto-scan** — the runner checks for common filenames in order: **Python** (checked first): | File | Command | | ----------- | ------------------ | | `main.py` | `python main.py` | | `app.py` | `python app.py` | | `bot.py` | `python bot.py` | | `server.py` | `python server.py` | | `index.py` | `python index.py` | **Node.js** (checked second): | File | Command | | ----------- | ---------------- | | `index.js` | `node index.js` | | `main.js` | `node main.js` | | `app.js` | `node app.js` | | `bot.js` | `node bot.js` | | `server.js` | `node server.js` | If your entrypoint doesn't match any of these names, use the `--command` flag or add a `run_command` field to your `orchagent.json` manifest: ```json theme={null} { "name": "my-agent", "type": "tool", "runtime": { "command": "python src/start.py" } } ``` *** ## Permissions | Operation | Required Role | | ---------------------- | ------------- | | List services | Member | | Get service info | Member | | View logs | Member | | Deploy service | Owner | | Restart service | Owner | | Update config (alerts) | Owner | | Update env/secrets | Owner | | Delete service | Owner | *** ## Examples ### Discord Bot ```bash theme={null} # Publish the agent (orchagent.json has required_secrets: ["DISCORD_BOT_TOKEN"]) cd my-discord-bot orch publish # Deploy — DISCORD_BOT_TOKEN is auto-resolved from the vault orch service deploy acme/my-discord-bot \ --env BOT_PREFIX="!" ``` ### Webhook Listener (Telegram, Stripe, etc.) ```bash theme={null} # Deploy — secrets auto-resolved from vault orch service deploy acme/webhook-processor # Find your public URL orch service info # → URL: https://orch-xxxx-webhook-processor.services.orchagent.io # Register the URL with your webhook provider curl "https://api.telegram.org/bot/setWebhook?url=/webhook" ``` Your HTTP server should bind to port 3000. See [HTTP Ingress](#http-ingress-receiving-webhooks) for details. ### Background Worker ```bash theme={null} # Agent declares required_secrets: ["REDIS_URL"] orch service deploy acme/queue-worker \ --command "python worker.py" \ --env QUEUE_NAME=tasks \ --env CONCURRENCY=4 ``` ### Custom Entrypoint with Arguments ```bash theme={null} orch service deploy acme/my-agent \ --command "python" \ --arg "src/main.py" \ --arg "--verbose" \ --arg "--port=3000" ``` *** ## Auto-Update on Publish When you publish a new version of an agent (`orch publish`), any running services **in the same workspace** with `auto_update` enabled are automatically updated to the new version. Auto-update is **workspace-scoped**. If you publish from your personal workspace but your service runs in a team workspace, the service won't be updated. You must publish from the same workspace where the service is deployed. ```bash theme={null} # Switch to the workspace where your service runs orch workspace use my-team # Now publish — services in "my-team" will auto-update orch publish ``` If auto-update didn't work, check: 1. **Are you in the right workspace?** Run `orch workspace list` and `orch service list --workspace ` to confirm 2. **Is auto\_update enabled?** Services deployed with `--no-auto-update` won't be updated 3. **Does the new version have a code bundle?** Agents without `runtime.command` or a code bundle won't trigger service updates To manually update a service to the latest version: ```bash theme={null} orch service update ``` `orch service restart` does **not** update the version — it only restarts the existing code. Use `orch service update` to pull a new published version. *** ## Troubleshooting **Service stuck in provisioning?** * Check the events timeline: `orch service info ` * Verify your agent has a `runtime.command` or `loop` config and a published code bundle (`prompt` type agents cannot be deployed as services) * Check the logs for dependency installation failures: `orch service logs ` **Service keeps restarting?** * Check the logs for crash output: `orch service logs ` * Look at the last error: `orch service info ` * Ensure all `required_secrets` from the agent's orchagent.json exist in your workspace vault * Verify your entrypoint process doesn't exit immediately * Your code must **not** bind to port 8080 — the platform health server uses it. Bind to port 3000 instead. **Service URL not resolving?** * New services may take up to 2 minutes for DNS to propagate * Try a public DNS resolver to confirm: `nslookup 1.1.1.1` * If your local machine cached a "not found" response, flush your DNS cache (`sudo dscacheutil -flushcache` on macOS) or wait \~30 minutes **Webhooks not reaching your app?** * Confirm your app binds to port 3000 (not 8080): `--host 0.0.0.0 --port 3000` * Check that the webhook provider is using the full URL including the path (e.g. `https:///webhook/telegram`) * Test locally: `curl https:///your-endpoint` — you should see your app's response, not a 404 **CRASH-LOOP badge showing?** * The service has been auto-paused after exceeding the failure threshold * Fix the root cause, then restart: `orch service restart ` * Check the events timeline for the failure pattern **"No entrypoint found" error?** * Add a `main.py` or `index.js` to your project root * Or set a `run_command` in your `orchagent.json` * Or use `--command` when deploying **Sensitive env var rejected?** * Variables with names matching `secret`, `token`, `password`, `api_key`, `credential`, or `private_key` must use `--secret` instead of `--env` * Store the value as a workspace secret first, then reference it by name **Bundle URL expired after a long-running service?** * Use `orch service restart ` — this automatically refreshes the signed bundle URL **Service not updating after `orch publish`?** * Auto-update is **workspace-scoped**. If you published from a different workspace than where the service runs, it won't auto-update * Switch to the correct workspace first: `orch workspace use `, then `orch publish` * Or manually update: `orch service update ` * Note: `orch service restart` does NOT update the version — use `orch service update` instead **Deploy fails with 422 after deleting a service?** * The previous service's infrastructure may not have fully cleaned up * Wait 1-2 minutes and retry * If it persists, try a different service name or contact support *** ## Limits Always-on services are available on Pro, Team, and Enterprise plans. Service compute time counts toward your workspace usage. Instance counts and concurrent service limits depend on your plan tier.