> ## Documentation Index
> Fetch the complete documentation index at: https://docs.sevalla.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> These are the docs for Sevalla, a platform for deploying applications, databases, static sites, object storage and load balancers. The dashboard is at https://app.sevalla.com.
> Bold text in steps is an exact dashboard label.
> To act on a Sevalla account, prefer the Sevalla MCP server (https://mcp.sevalla.com/mcp, see https://docs.sevalla.com/ai-agents/overview.md), then the Sevalla CLI (https://docs.sevalla.com/developer-tools/cli.md), then the REST API (https://api-docs.sevalla.com, OpenAPI spec at https://api.sevalla.com/v3/openapi.json).
> Every page index is at https://docs.sevalla.com/llms.txt.

# Deploy the LiteLLM proxy

> Deploy the LiteLLM proxy from a Dockerfile and config.yaml: PostgreSQL for virtual keys and spend, one key per app, and models on Ollama, Anthropic and OpenAI, private or public.

The LiteLLM proxy runs on Sevalla as an application that gives your other applications one OpenAI-compatible API for every model: models on your own [Ollama](https://docs.sevalla.com/guides/ai/ollama.md) server and hosted models from Anthropic and OpenAI. Each application gets its own virtual key with its own budget, and LiteLLM records the tokens and cost of every request in a Sevalla PostgreSQL database. The provider API keys stay in LiteLLM, never in your applications.

The official image has no models configured, and Sevalla doesn't mount files into containers, so this guide deploys a two-file repository: a `Dockerfile` that starts from the official image and adds your `config.yaml`. Model changes then go through Git and deploy on every push.

## At a glance

| | |
| - | - |
| Source | A Git repository with `config.yaml` and a `Dockerfile` |
| Build | The [Dockerfile](#create-the-project), `FROM ghcr.io/berriai/litellm-non_root:v1.106.6`. Ingot can't build this repository |
| Start command | The image's entry point with `--config /app/config.yaml` |
| Port | The `PORT` environment variable, `8080` by default |
| Processes | Web process, with HTTP traffic turned off |
| Services | PostgreSQL 18 over an internal connection; [Ollama](https://docs.sevalla.com/guides/ai/ollama.md) over an internal connection |
| Disk | None |
| Health check | `GET /health/liveliness` returns `200` (readiness probe) |
| Size | **S2** |

### Environment variables

| Variable | Required | Available during | Secret | Value |
| - | - | - | - | - |
| `DATABASE_URL` | Yes | Runtime | Yes | `DB_URL` from the internal connection, renamed |
| `LITELLM_MASTER_KEY` | Yes | Runtime | Yes | Generate with `printf 'sk-%s\n' "$(openssl rand -hex 32)"` |
| `LITELLM_SALT_KEY` | Yes | Runtime | Yes | Generate the same way, once. Never change it |
| `OLLAMA_API_BASE` | For Ollama models | Runtime | No | `http://` plus Ollama's internal hostname plus `:8080` |
| `ANTHROPIC_API_KEY` | For Anthropic models | Runtime | Yes | An API key from the Anthropic Console |
| `OPENAI_API_KEY` | For OpenAI models | Runtime | Yes | An API key from the OpenAI platform |
| `PORT` | No | Runtime | No | Set by Sevalla to the web process port |

## Before you begin

* A Sevalla company with a payment method, and a GitHub, GitLab or Bitbucket account for the repository. See [Git providers](https://docs.sevalla.com/git/overview.md).
* An [Ollama application](https://docs.sevalla.com/guides/ai/ollama.md) in the same location, with `lfm3:1.7b` and `embeddinggemma-2:270m` pulled. To use hosted models only, skip the Ollama entries in `config.yaml`.
* For hosted models, an [Anthropic API key](https://console.anthropic.com/settings/keys) and an [OpenAI API key](https://platform.openai.com/api-keys), with spending limits set on both provider accounts.

## How LiteLLM runs on Sevalla

| Part | Setup |
| - | - |
| Image | `ghcr.io/berriai/litellm-non_root:v1.106.6`, LiteLLM's official image that runs as an unprivileged user (UID `65534`), pinned to a release |
| Models | `config.yaml`, copied into the image by your `Dockerfile` |
| Port | The `PORT` variable, which Sevalla sets to `8080` |
| Database | Sevalla PostgreSQL, for virtual keys, budgets and spend logs. LiteLLM applies its schema migrations when it starts |
| Access | Private: HTTP traffic off, reached over internal connections. Or [public](#make-litellm-public), with a key required on every request |
| Size | **S2** (1 vCPU, 2 GB). On **S1**, LiteLLM's memory peaked at 936 MB of 1 GB |
| Instances | 1. LiteLLM's [production guide](https://docs.litellm.ai/docs/proxy/deploy) asks for Redis once you run more than one instance |

## Create the project

Create a folder for the repository:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
mkdir litellm-proxy && cd litellm-proxy
```

Add the models. Each `model_name` is the name your applications send in the `model` field:

```yaml config.yaml theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
model_list:
  - model_name: lfm3
    litellm_params:
      model: ollama_chat/lfm3:1.7b
      api_base: os.environ/OLLAMA_API_BASE
      think: false
  - model_name: embeddinggemma
    litellm_params:
      model: ollama/embeddinggemma-2:270m
      api_base: os.environ/OLLAMA_API_BASE
    model_info:
      mode: embedding

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL
```

| Setting | What it does |
| - | - |
| `ollama_chat/lfm3:1.7b` | Sends chats to Ollama's `/api/chat` endpoint, which LiteLLM recommends for Ollama |
| `os.environ/OLLAMA_API_BASE` | Reads the value from an environment variable at runtime, so no address or secret is in the repository |
| `think: false` | Turns off the model's reasoning, so `lfm3` answers directly. On a CPU, reasoning tokens cost seconds |
| `mode: embedding` | Marks the model for `/v1/embeddings` |
| `master_key`, `database_url` | The admin key and the PostgreSQL database, both from environment variables |

Add the `Dockerfile`:

```dockerfile Dockerfile theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
FROM ghcr.io/berriai/litellm-non_root:v1.106.6

COPY config.yaml /app/config.yaml

EXPOSE 8080
CMD ["--config", "/app/config.yaml"]
```

| Line or choice | Why |
| - | - |
| `litellm-non_root:v1.106.6` | The official image, in the variant that runs as UID `65534` instead of root. A release tag keeps every build on the same LiteLLM; change it to [update](#update-litellm) |
| `COPY config.yaml` | The only file the image needs from the repository. Root owns it, so the proxy can read it but not change it |
| `CMD ["--config", ...]` | Replaces the image's `--port 4000`. Without `--port`, LiteLLM listens on `PORT`, which Sevalla sets to `8080` |
| No init process | The image's entry point `exec`s the server, which handles `SIGTERM` itself |

Keep Git history and local secrets out of the build:

```text .dockerignore theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
.git
.env*
```

Commit the files and push them to your Git provider:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
git init
git add .
git commit -m "Add LiteLLM config"
git branch -M main
git remote add origin https://github.com/acme/litellm-proxy.git
git push -u origin main
```

## Deploy to Sevalla

These steps build from your `Dockerfile`. The repository has no application code, so [Ingot](https://docs.sevalla.com/applications/builds/ingot.md) stops with `Ingot couldn't tell what kind of application this is`: switch the build strategy before the first deployment.

**1. Create the database**

In the sidebar, click **Databases**, then click **Create database**. Click **PostgreSQL** and choose the latest version. Enter the **Display name** `litellm-db`, choose the **Data center** where Ollama runs, choose the **Size**, and click **Create database**. This guide was tested on **DB1**; spend logs grow with every request, so watch the storage. See [Create a database](https://docs.sevalla.com/databases/create.md).

**2. Create the application**

Click **Applications**, then click **Create** > **Application**. Under **Source**, click **Private repository**, then choose the **Git provider**, the **Repository** and the **Branch**. Enter the **Name** `litellm`, choose the same **Data center**, choose the **Size** **S2**, then click **Create application**. Don't deploy yet.

**3. Choose the Dockerfile build strategy**

On the **Settings** tab, in the **Build strategy** section, click **Update build strategy**. Click **Dockerfile**, keep the **Dockerfile path** `Dockerfile` and the **Docker context** `.`, then click **Update build strategy**.

**4. Connect the database**

Click the **Networking** tab. Under **Connected services**, click **Add internal connection** and choose `litellm-db` as the **Service**. Turn on **Add environment variables to the application**, clear **Available during build**, change the key `DB_URL` to `DATABASE_URL`, and click **Add internal connection**.

**5. Connect Ollama**

Click **Add internal connection** again, choose `ollama` as the **Service**, and click **Add internal connection**.

**6. Add the environment variables**

Click the **Environment variables** tab, then click **Add environment variables**. Add the three variables from the table after these steps. Under **Availability**, clear **Available during buildtime**, then click **Add environment variables**.

**7. Add a readiness probe**

Click the **Processes** tab, click **⋯** on the web process, then click **Update process**. Under **Health checks**, turn on **Readiness probe**, enter the **Path** `/health/liveliness` and the **Initial delay** `10`, then click **Update process**. New instances then take requests only once LiteLLM answers: 27 requests during a deployment all succeeded.

**8. Turn HTTP traffic off**

Click the **Networking** tab. In **Expose HTTP traffic**, click **Disable**, then click **Disable HTTP**. To reach LiteLLM from the internet instead, see [Make LiteLLM public](#make-litellm-public).

**9. Deploy**

Click the **Overview** tab, then click **Deploy now** under **Recent deployments**. Check the **Branch** and click **Deploy**.

| Variable | Value |
| - | - |
| `LITELLM_MASTER_KEY` | The admin key: it creates and deletes virtual keys and calls every model. Generate one with `printf 'sk-%s\n' "$(openssl rand -hex 32)"` |
| `LITELLM_SALT_KEY` | Encrypts the provider credentials LiteLLM stores in the database. Generate it the same way, once: if it changes, stored credentials can't be read |
| `OLLAMA_API_BASE` | `http://` plus Ollama's internal hostname plus `:8080`, for example `http://ollama-k3j4h-web.ollama-k3j4h.svc.cluster.local:8080`. Copy the hostname from **Internal hostnames** on the `ollama` application's **Networking** tab |

Both keys are runtime only: a build-time variable becomes a build argument of the image build.

## Verify the deployment

1. On the **Deployments** tab, click the deployment. Its **Build log** shows `📦 Building and pushing docker image using Dockerfile...`, and its **Dockerfile** tab shows your `Dockerfile`.

2. When the deployment is **Deployed**, open the application's [runtime logs](https://docs.sevalla.com/applications/logs.md). The first start applies the database migrations (`Applying migration` lines, then `No pending migrations to apply.` on later starts) and lists the models:

   ```text theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
   LiteLLM: Proxy initialized with Config, Set models:
       lfm3
       embeddinggemma
   INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
   ```

3. In the application's [web terminal](https://docs.sevalla.com/applications/web-terminal.md), check that the proxy answers:

   ```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
   lite --base-url "http://localhost:$PORT" --api-key "$LITELLM_MASTER_KEY" http request GET /health/liveliness
   ```

   ```text theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
   "I'm alive!"
   ```

4. With HTTP traffic off, the application's domain returns `404`:

   ```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
   curl -s -o /dev/null -w '%{http_code}\n' https://litellm-q9w3e.sevalla.app/health/liveliness
   ```

   ```text theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
   404
   ```

## Create a key for each application

Give every application its own virtual key: its spend is tracked separately, it can have a budget, and you can delete it without touching the others. Create keys with LiteLLM's `lite` command in the application's [web terminal](https://docs.sevalla.com/applications/web-terminal.md), which calls the proxy inside the same container with the master key:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
lite --base-url "http://localhost:$PORT" --api-key "$LITELLM_MASTER_KEY" \
  http request POST /key/generate -j '{"key_alias": "open-webui", "max_budget": 10}'
```

The answer is the new key's settings, with `"key_alias": "open-webui"`, `"max_budget": 10.0` and the key itself in `"key"`, which starts with `sk-`. LiteLLM stores only a hash of the key, so copy it now. Once the key's spend passes its `max_budget`, in US dollars, LiteLLM rejects its requests.

| Field | Effect |
| - | - |
| `key_alias` | The name the key appears under, for example the application's name |
| `max_budget` | The most the key can spend, in US dollars. Leave it out for no limit |
| `models` | A list such as `["lfm3"]` limits the key to those models. A request for another model gets a `403` with `"type":"key_model_access_denied"`. Without it, the key can call every model in `config.yaml`, including ones you add later |

Set the budget with `max_budget` as shown: the `--spend` option of `lite keys generate` sets what the key has already spent, not a limit. To delete a key, run the same `lite` command with `keys delete --key-aliases open-webui`.

## Call LiteLLM from an application

On the calling application's **Networking** tab, add an internal connection to `litellm`. Then add two variables on its **Environment variables** tab and deploy it:

| Variable | Value |
| - | - |
| `OPENAI_BASE_URL` | `http://` plus LiteLLM's internal hostname plus `:8080/v1`, for example `http://litellm-q9w3e-web.litellm-q9w3e.svc.cluster.local:8080/v1` |
| `OPENAI_API_KEY` | The application's virtual key |

The official OpenAI SDKs read both variables, so code that uses them needs no change. From the application's web terminal:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
curl "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "lfm3", "messages": [{"role": "user", "content": "Say hello in five words."}]}'
```

```json theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
{
  "id": "chatcmpl-0fb76c43-02e0-41bc-a359-52fd198ffbe9",
  "created": 1793333258,
  "model": "lfm3",
  "object": "chat.completion",
  "choices": [{"finish_reason": "stop", "index": 0, "message": {"content": "Hello, how can I help?", "role": "assistant"}}],
  "usage": {"completion_tokens": 8, "prompt_tokens": 17, "total_tokens": 25}
}
```

Streaming works the same way: with `"stream": true`, LiteLLM sends Server-Sent Events as the model writes them. To connect a chat interface, see [Deploy Open WebUI](https://docs.sevalla.com/guides/ai/open-webui.md).

## Add Anthropic and OpenAI models

Hosted models go in the same `config.yaml`, with their API keys in environment variables of the `litellm` application only.

**1. Add the models**

Add the models to `model_list` in `config.yaml`:

```yaml config.yaml theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
  - model_name: claude-haiku-5-5
    litellm_params:
      model: anthropic/claude-haiku-5-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: gpt-6-luna
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY
```

These are low-cost current models, a sensible default for chat. To use another model, change the part after `anthropic/` or `openai/` to a name from [Anthropic's](https://docs.anthropic.com/en/docs/about-claude/models/overview) or [OpenAI's](https://platform.openai.com/docs/models) model list, and `model_name` to the name your applications send.

**2. Add the provider keys**

On the `litellm` application's **Environment variables** tab, add `ANTHROPIC_API_KEY` and `OPENAI_API_KEY`. Clear **Available during buildtime**, then click **Add environment variables**.

**3. Deploy**

Commit and push `config.yaml`. With **Auto-deploy** on, the push starts the deployment; otherwise, click **Deploy now**.

The runtime logs now list four models. Keys created without `models` can call them too. LiteLLM prices each request from its built-in price list and adds it to the key's spend. To see it, run in the web terminal:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
lite --base-url "http://localhost:$PORT" --api-key "$LITELLM_MASTER_KEY" \
  keys list --key-alias open-webui --format table
```

After one chat each with `claude-haiku-5-5` and `gpt-6-luna` from [Open WebUI](https://docs.sevalla.com/guides/ai/open-webui.md), with the titles and suggestions Open WebUI generated for them, the **Spend** column read `0.001983075`, in US dollars. Requests to Ollama models cost `0`.

To reach models from many providers with one key, add OpenRouter models to the same `config.yaml`: see [OpenRouter in LiteLLM](https://docs.sevalla.com/guides/ai/openrouter.md#litellm).

## Make LiteLLM public

Keep LiteLLM private when only your own applications on Sevalla call it. Make it public when applications or people outside Sevalla need it, for example a laptop running an AI coding tool:

1. On the application's **Networking** tab, in **Expose HTTP traffic**, click **Enable**, then click **Expose HTTP**.
2. Call it on its domain, for example `https://litellm-q9w3e.sevalla.app/v1`, with a virtual key.

Every model and admin endpoint then needs a key; only the health checks and the Admin UI's sign-in page answer without one:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
curl https://litellm-q9w3e.sevalla.app/v1/models
```

```json theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
{"error":{"message":"Authentication Error, No api key passed in.","type":"auth_error","param":"None","code":"401"}}
```

The Admin UI is at `/ui`: sign in with the username `admin` and the master key as the password. Anyone with the master key controls the proxy and every provider key in it, so keep it in the `litellm` application's variables only, hand out virtual keys with budgets, and limit who can reach the domain with [IP restrictions](https://docs.sevalla.com/applications/networking.md#ip-restrictions) where you can. Streams reach the client unbuffered through Sevalla's proxy: in a test over the public domain, the first of 82 events arrived after 0.89 s and the last after 6.63 s.

## Update LiteLLM

**1. Back up the database**

Create a backup of `litellm-db` on its **Backups** tab. LiteLLM changes the schema when a new version starts, and a backup is the way back. See [Backups](https://docs.sevalla.com/databases/backups.md).

**2. Change the version**

In the `Dockerfile`, change the tag in `FROM` to the new release, for example from [LiteLLM's releases](https://github.com/BerriAI/litellm/releases). Use a release without `-rc` or `-dev` in its name.

**3. Deploy**

Commit and push. The runtime logs show the migrations the new version applies, then the models.

## Deploy changes

Edit `config.yaml` to add, rename or remove models, then push. With **Auto-deploy** on, every push to the branch starts a deployment; otherwise, click **Deploy now**. See [Deployments](https://docs.sevalla.com/applications/deployments.md).

## Troubleshoot

| Symptom | Cause | Fix |
| - | - | - |
| `Authentication Error, No api key passed in.` | The request has no `Authorization: Bearer` header | Send a virtual key |
| `403` with `key_model_access_denied` | The key was created with a `models` list that doesn't include the model | Create a key without `models`, or with the model in the list |
| The instance restarts with exit code `137` | LiteLLM ran out of memory | Use **S2** or larger. See [Troubleshoot runtime errors](https://docs.sevalla.com/applications/troubleshooting/runtime-errors.md) |
| Ollama models fail while other models work | LiteLLM has no internal connection to `ollama`, or `OLLAMA_API_BASE` has the wrong host or port | Add the connection and check the variable, then deploy |
| Requests fail for a few seconds during a deployment | LiteLLM has no readiness probe, or Ollama is restarting: with its disk, it stops before its new instance starts | Add the [readiness probe](#deploy-to-sevalla). For Ollama, deploy when few requests run |

## Deploy with the API

The same deployment as the dashboard steps, as API calls an agent can run in order. It needs:

* An [API key](https://docs.sevalla.com/developer-tools/api-keys.md) that can create applications and databases, in `SEVALLA_API_KEY`.
* The repository on a Git provider connected to the company. The API has no call that lists connected accounts: [Create application](https://api-docs.sevalla.com/v3/applications/create-application) checks access itself and answers `400` with `Cannot access repository <url>: Repository not found or you don't have access.` when Sevalla can't read the repository.
* The ID of the [Ollama application](https://docs.sevalla.com/guides/ai/ollama.md#deploy-with-the-api), in `OLLAMA_APP_ID`.
* `curl`, `jq` and `openssl`.

Each step names the API operation, the CLI command (`@sevalla/cli` 2.1.0) and the tool of the [Sevalla MCP server](https://docs.sevalla.com/ai-agents/overview.md).

**1. Set the variables**

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
export SEVALLA_API=https://api.sevalla.com/v3
export SEVALLA_API_KEY=<your API key>
REPO_URL=https://github.com/acme/litellm-proxy
BRANCH=main
OLLAMA_APP_ID=<the Ollama application's ID>
```

**2. Find the location**

[List clusters](https://api-docs.sevalla.com/v3/resources/list-clusters) (`listClustersV3`):

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
CLUSTER_ID=$(curl -s "$SEVALLA_API/resources/clusters" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.location == "eu-frankfurt-1") | .id')
```

Replace `eu-frankfurt-1` with the region ID of your location from [Locations](https://docs.sevalla.com/platform/locations.md#locations-on-oracle-cloud-infrastructure), and pick by region ID, not by display name. CLI: `sevalla resources clusters`. MCP: `get_platform_options`.

**3. Look up the sizes and the PostgreSQL version**

[List process resource types](https://api-docs.sevalla.com/v3/resources/list-process-resource-types) (`listProcessResourceTypesV3`), [list database resource types](https://api-docs.sevalla.com/v3/resources/list-database-resource-types) (`listDatabaseResourceTypesV3`) and [list database versions](https://api-docs.sevalla.com/v3/resources/list-database-versions) (`listDatabaseVersionsV3`):

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
SIZE_ID=$(curl -s "$SEVALLA_API/resources/process-resource-types" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.name == "s2") | .id')
DB_SIZE_ID=$(curl -s "$SEVALLA_API/resources/database-resource-types" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.name == "db1") | .id')
PG_VERSION=$(curl -s "$SEVALLA_API/resources/database-versions?cluster_id=$CLUSTER_ID&type=postgresql" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[0].versions | last')
```

CLI: `sevalla resources process-types`, `sevalla resources db-types` and `sevalla resources db-versions --cluster $CLUSTER_ID --type postgresql`. MCP: `get_platform_options`.

**4. Create the database**

[Create database](https://api-docs.sevalla.com/v3/databases/create-database) (`createDatabaseV3`), then [get database](https://api-docs.sevalla.com/v3/databases/get-database) (`getDatabaseV3`) until its `status` is `ready`. Keep `DB_PASSWORD`: the API never returns it:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
DB_PASSWORD=$(openssl rand -hex 16)
DB_ID=$(curl -s -X POST "$SEVALLA_API/databases" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"display_name\": \"litellm-db\", \"type\": \"postgresql\", \"version\": \"$PG_VERSION\", \"cluster_id\": \"$CLUSTER_ID\", \"resource_type_id\": \"$DB_SIZE_ID\", \"db_name\": \"litellm\", \"db_password\": \"$DB_PASSWORD\"}" \
  | jq -r .id)
for i in $(seq 1 120); do
  DB_STATUS=$(curl -s "$SEVALLA_API/databases/$DB_ID" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" | jq -r '.status // empty')
  [ "$DB_STATUS" = "ready" ] && break
  sleep 15
done
echo "$DB_STATUS"
```

CLI: `sevalla databases create --name litellm-db --type postgresql --db-version $PG_VERSION --cluster $CLUSTER_ID --resource-type $DB_SIZE_ID --db-name litellm --db-password $DB_PASSWORD`. MCP: `create_database`, which generates the password itself and never returns it; with `connect_to_application_id` set to the application and `env_var_name` `DATABASE_URL`, it connects the database and stores a `postgresql://` URL, which LiteLLM accepts. Create the application first in that case.

**5. Create the application**

[Create application](https://api-docs.sevalla.com/v3/applications/create-application) (`createApplicationV3`). It doesn't deploy, and it creates the web process at **S1**:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
APP_ID=$(curl -s -X POST "$SEVALLA_API/applications" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"display_name\": \"litellm\", \"cluster_id\": \"$CLUSTER_ID\", \"source\": \"privateGit\", \"git_type\": \"github\", \"repo_url\": \"$REPO_URL\", \"default_branch\": \"$BRANCH\"}" \
  | jq -r .id)
```

CLI: `sevalla apps create --name litellm --cluster $CLUSTER_ID --source privateGit --git-type github --repository $REPO_URL --branch $BRANCH`. MCP: `create_application` with `"deploy": false`.

**6. Choose the Dockerfile build strategy**

[Update application](https://api-docs.sevalla.com/v3/applications/update-application) (`updateApplicationV3`):

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
curl -s -X PATCH "$SEVALLA_API/applications/$APP_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"build_type": "dockerfile", "dockerfile_path": "Dockerfile", "docker_context": "."}' \
  | jq -c '{build_type, dockerfile_path, docker_context}'
```

```json theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
{"build_type":"dockerfile","dockerfile_path":"Dockerfile","docker_context":"."}
```

CLI: `sevalla apps update $APP_ID --data '{"build_type": "dockerfile", "dockerfile_path": "Dockerfile", "docker_context": "."}'`. MCP: `sevalla_api_update` with `updateApplicationV3`.

**7. Set the size, the readiness probe and HTTP traffic**

[List processes](https://api-docs.sevalla.com/v3/applications/list-processes) (`listProcessesV3`), then [update process](https://api-docs.sevalla.com/v3/applications/update-process) (`updateProcessV3`). Send `timeoutSeconds`: a probe set through the API without it waits 120 seconds, not the dashboard's 1:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
WEB_ID=$(curl -s "$SEVALLA_API/applications/$APP_ID/processes" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.data[] | select(.type == "web") | .id')
curl -s -X PATCH "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"resource_type_id\": \"$SIZE_ID\", \"is_ingress_enabled\": false, \"readiness_probe\": {\"httpGet\": {\"path\": \"/health/liveliness\", \"port\": 8080}, \"initialDelaySeconds\": 10, \"periodSeconds\": 10, \"timeoutSeconds\": 1, \"successThreshold\": 1, \"failureThreshold\": 3}}" \
  | jq -c '{resource_type_name, is_ingress_enabled}'
```

```json theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
{"resource_type_name":"s2","is_ingress_enabled":false}
```

CLI: `sevalla apps processes update $WEB_ID --app-id $APP_ID --resource-type $SIZE_ID` sets the size; the CLI can't set probes or turn HTTP traffic off. MCP: `sevalla_api_update` with `updateProcessV3` and the same body.

**8. Connect the database and Ollama**

[Create internal connection](https://api-docs.sevalla.com/v3/applications/create-internal-connection) (`createApplicationInternalConnectionV3`), once for each:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
curl -s -X POST "$SEVALLA_API/applications/$APP_ID/internal-connections" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"target_id\": \"$DB_ID\", \"target_type\": \"database\"}"
curl -s -X POST "$SEVALLA_API/applications/$APP_ID/internal-connections" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"target_id\": \"$OLLAMA_APP_ID\", \"target_type\": \"app\"}"
```

CLI: `sevalla apps internal-connections create $APP_ID --target-id $DB_ID --target-type database`. MCP: `sevalla_api_create` with `createApplicationInternalConnectionV3`.

**9. Add the environment variables**

The API connection adds no variables, so build `DATABASE_URL` in the dashboard's format, `postgres://<user>:<password>@<host>:<port>/<database>`, from [get database](https://api-docs.sevalla.com/v3/databases/get-database), and `OLLAMA_API_BASE` from Ollama's web process. Then [create environment variable](https://api-docs.sevalla.com/v3/applications/create-environment-variable) (`createEnvironmentVariableV3`) for each, runtime only:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
DATABASE_URL=$(curl -s "$SEVALLA_API/databases/$DB_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r --arg pw "$DB_PASSWORD" '"postgres://\(.db_user):\($pw)@\(.internal_hostname):\(.internal_port)/\(.db_name)"')
OLLAMA_HOSTNAME=$(curl -s "$SEVALLA_API/applications/$OLLAMA_APP_ID/processes" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.data[] | select(.type == "web") | .internal_hostname')
LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"

add_var() {
  jq -n --arg key "$1" --arg value "$2" '{key: $key, value: $value, is_runtime: true, is_buildtime: false}' \
    | curl -s -X POST "$SEVALLA_API/applications/$APP_ID/env-vars" \
      -H "Authorization: Bearer $SEVALLA_API_KEY" \
      -H "Content-Type: application/json" \
      -d @- \
    | jq -r .key
}
add_var DATABASE_URL "$DATABASE_URL"
add_var LITELLM_MASTER_KEY "$LITELLM_MASTER_KEY"
add_var LITELLM_SALT_KEY "sk-$(openssl rand -hex 32)"
add_var OLLAMA_API_BASE "http://$OLLAMA_HOSTNAME:8080"
```

Keep `LITELLM_MASTER_KEY` for the next steps. [List environment variables](https://api-docs.sevalla.com/v3/applications/list-environment-variables) (`listEnvironmentVariablesV3`) returns the values later too; the MCP server withholds them. When `config.yaml` lists hosted models, also run `add_var ANTHROPIC_API_KEY "$ANTHROPIC_API_KEY"` and `add_var OPENAI_API_KEY "$OPENAI_API_KEY"`. CLI: `sevalla apps env-vars create $APP_ID --key LITELLM_MASTER_KEY --value "$LITELLM_MASTER_KEY"`, which makes the variable available during build and runtime. MCP: `set_environment_variables` with `runtime: true` and `buildtime: false`.

**10. Deploy**

[Trigger deployment](https://api-docs.sevalla.com/v3/applications/trigger-deployment) (`triggerDeploymentV3`), then [get deployment](https://api-docs.sevalla.com/v3/applications/get-deployment) (`getDeploymentV3`) until `status` is `success` or `failed`, for at most 30 minutes:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
DEPLOYMENT_ID=$(curl -s -X POST "$SEVALLA_API/applications/$APP_ID/deployments" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  | jq -r .id)
for i in $(seq 1 120); do
  STATUS=$(curl -s "$SEVALLA_API/applications/$APP_ID/deployments/$DEPLOYMENT_ID" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" | jq -r '.status // empty')
  case $STATUS in success|failed|cancelled|skipped) break ;; esac
  sleep 15
done
echo "$STATUS"
```

CLI: `sevalla apps deployments trigger $APP_ID`, then `sevalla apps deployments get $DEPLOYMENT_ID --app-id $APP_ID`. MCP: `trigger_deployment`, then `get_deployment`.

**11. Check the build and the start**

[Get application deployment logs](https://api-docs.sevalla.com/v3/applications/get-application-deployment-logs) (`getApplicationDeploymentLogsV3`) and [get application runtime logs](https://api-docs.sevalla.com/v3/applications/get-application-runtime-logs) (`getApplicationRuntimeLogsV3`):

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
curl -s "$SEVALLA_API/applications/$APP_ID/deployments/$DEPLOYMENT_ID/logs?limit=5000" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[].message' | grep -m1 'Building and pushing'
curl -s "$SEVALLA_API/applications/$APP_ID/runtime-logs?search=Proxy%20initialized" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[].message'
```

The first prints `📦 Building and pushing docker image using Dockerfile...`, the second `LiteLLM: Proxy initialized with Config, Set models:`. CLI: `sevalla apps deployments logs $DEPLOYMENT_ID --app-id $APP_ID` and `sevalla apps logs runtime $APP_ID --search "Proxy initialized"`. MCP: `get_deployment_logs` and `search_runtime_logs`.

**12. Create a key and list the models**

[Run a command in a process](https://api-docs.sevalla.com/v3/applications/run-a-command-in-a-process) (`execProcessCommandV3`) runs `lite` without a shell, so pass the master key as a value, not as `$LITELLM_MASTER_KEY`:

```bash theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
VIRTUAL_KEY=$(jq -n --arg mk "$LITELLM_MASTER_KEY" \
    '{command: ["lite", "--base-url", "http://localhost:8080", "--api-key", $mk, "http", "request", "POST", "/key/generate", "-j", "{\"key_alias\": \"open-webui\", \"max_budget\": 10}"], timeout: 60}' \
  | curl -s -X POST "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/exec" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" \
    -H "Content-Type: application/json" \
    -d @- \
  | jq -r .stdout | jq -r .key)

jq -n --arg vk "$VIRTUAL_KEY" \
    '{command: ["lite", "--base-url", "http://localhost:8080", "--api-key", $vk, "http", "request", "GET", "/v1/models"], timeout: 60}' \
  | curl -s -X POST "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/exec" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" \
    -H "Content-Type: application/json" \
    -d @- \
  | jq -r .stdout | jq -r '.data[].id'
```

With the hosted models in `config.yaml`, the second command prints:

```text theme={"theme":{"light":"github-dark-default","dark":"github-dark-default"}}
lfm3
embeddinggemma
claude-haiku-5-5
gpt-6-luna
```

`VIRTUAL_KEY` is Open WebUI's key. CLI: `sevalla apps processes exec $WEB_ID --app-id $APP_ID --timeout 60 -- lite --base-url http://localhost:8080 --api-key <master key> http request GET /health/liveliness`. MCP: `sevalla_api_update` with `execProcessCommandV3`.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.