Deploy the LiteLLM proxy

The LiteLLM proxy runs on Sevalla as an application that gives your other applications one OpenAI-compatible API for every model: models on your own Ollama server and hosted models from Anthropic and OpenAI. Each application gets its own virtual key with its own budget, and LiteLLM records the tokens and cost of every request in a Sevalla PostgreSQL database. The provider API keys stay in LiteLLM, never in your applications. The official image has no models configured, and Sevalla doesn’t mount files into containers, so this guide deploys a two-file repository: a Dockerfile that starts from the official image and adds your config.yaml. Model changes then go through Git and deploy on every push.

At a glance


Source	A Git repository with `config.yaml` and a `Dockerfile`
Build	The Dockerfile, `FROM ghcr.io/berriai/litellm-non_root:v1.106.6`. Ingot can’t build this repository
Start command	The image’s entry point with `--config /app/config.yaml`
Port	The `PORT` environment variable, `8080` by default
Processes	Web process, with HTTP traffic turned off
Services	PostgreSQL 18 over an internal connection; Ollama over an internal connection
Disk	None
Health check	`GET /health/liveliness` returns `200` (readiness probe)
Size	S2

Environment variables

Variable	Required	Available during	Secret	Value
`DATABASE_URL`	Yes	Runtime	Yes	`DB_URL` from the internal connection, renamed
`LITELLM_MASTER_KEY`	Yes	Runtime	Yes	Generate with `printf 'sk-%s\n' "$(openssl rand -hex 32)"`
`LITELLM_SALT_KEY`	Yes	Runtime	Yes	Generate the same way, once. Never change it
`OLLAMA_API_BASE`	For Ollama models	Runtime	No	`http://` plus Ollama’s internal hostname plus `:8080`
`ANTHROPIC_API_KEY`	For Anthropic models	Runtime	Yes	An API key from the Anthropic Console
`OPENAI_API_KEY`	For OpenAI models	Runtime	Yes	An API key from the OpenAI platform
`PORT`	No	Runtime	No	Set by Sevalla to the web process port

Before you begin

A Sevalla company with a payment method, and a GitHub, GitLab or Bitbucket account for the repository. See Git providers.
An Ollama application in the same location, with lfm3:1.7b and embeddinggemma-2:270m pulled. To use hosted models only, skip the Ollama entries in config.yaml.
For hosted models, an Anthropic API key and an OpenAI API key, with spending limits set on both provider accounts.

How LiteLLM runs on Sevalla

Part	Setup
Image	`ghcr.io/berriai/litellm-non_root:v1.106.6`, LiteLLM’s official image that runs as an unprivileged user (UID `65534`), pinned to a release
Models	`config.yaml`, copied into the image by your `Dockerfile`
Port	The `PORT` variable, which Sevalla sets to `8080`
Database	Sevalla PostgreSQL, for virtual keys, budgets and spend logs. LiteLLM applies its schema migrations when it starts
Access	Private: HTTP traffic off, reached over internal connections. Or public, with a key required on every request
Size	S2 (1 vCPU, 2 GB). On S1, LiteLLM’s memory peaked at 936 MB of 1 GB
Instances	1. LiteLLM’s production guide asks for Redis once you run more than one instance

Create the project

Create a folder for the repository:

mkdir litellm-proxy && cd litellm-proxy

Add the models. Each model_name is the name your applications send in the model field:

config.yaml

model_list:
  - model_name: lfm3
    litellm_params:
      model: ollama_chat/lfm3:1.7b
      api_base: os.environ/OLLAMA_API_BASE
      think: false
  - model_name: embeddinggemma
    litellm_params:
      model: ollama/embeddinggemma-2:270m
      api_base: os.environ/OLLAMA_API_BASE
    model_info:
      mode: embedding

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

Setting	What it does
`ollama_chat/lfm3:1.7b`	Sends chats to Ollama’s `/api/chat` endpoint, which LiteLLM recommends for Ollama
`os.environ/OLLAMA_API_BASE`	Reads the value from an environment variable at runtime, so no address or secret is in the repository
`think: false`	Turns off the model’s reasoning, so `lfm3` answers directly. On a CPU, reasoning tokens cost seconds
`mode: embedding`	Marks the model for `/v1/embeddings`
`master_key`, `database_url`	The admin key and the PostgreSQL database, both from environment variables

Add the Dockerfile:

Dockerfile

FROM ghcr.io/berriai/litellm-non_root:v1.106.6

COPY config.yaml /app/config.yaml

EXPOSE 8080
CMD ["--config", "/app/config.yaml"]

Line or choice	Why
`litellm-non_root:v1.106.6`	The official image, in the variant that runs as UID `65534` instead of root. A release tag keeps every build on the same LiteLLM; change it to update
`COPY config.yaml`	The only file the image needs from the repository. Root owns it, so the proxy can read it but not change it
`CMD ["--config", ...]`	Replaces the image’s `--port 4000`. Without `--port`, LiteLLM listens on `PORT`, which Sevalla sets to `8080`
No init process	The image’s entry point `exec`s the server, which handles `SIGTERM` itself

Keep Git history and local secrets out of the build:

.dockerignore

.git
.env*

Commit the files and push them to your Git provider:

git init
git add .
git commit -m "Add LiteLLM config"
git branch -M main
git remote add origin https://github.com/acme/litellm-proxy.git
git push -u origin main

Deploy to Sevalla

These steps build from your Dockerfile. The repository has no application code, so Ingot stops with Ingot couldn't tell what kind of application this is: switch the build strategy before the first deployment.

Create the database

In the sidebar, click Databases, then click Create database. Click PostgreSQL and choose the latest version. Enter the Display name litellm-db, choose the Data center where Ollama runs, choose the Size, and click Create database. This guide was tested on DB1; spend logs grow with every request, so watch the storage. See Create a database.

Create the application

Click Applications, then click Create > Application. Under Source, click Private repository, then choose the Git provider, the Repository and the Branch. Enter the Name litellm, choose the same Data center, choose the Size S2, then click Create application. Don’t deploy yet.

Choose the Dockerfile build strategy

On the Settings tab, in the Build strategy section, click Update build strategy. Click Dockerfile, keep the Dockerfile path Dockerfile and the Docker context ., then click Update build strategy.

Connect the database

Click the Networking tab. Under Connected services, click Add internal connection and choose litellm-db as the Service. Turn on Add environment variables to the application, clear Available during build, change the key DB_URL to DATABASE_URL, and click Add internal connection.

Connect Ollama

Click Add internal connection again, choose ollama as the Service, and click Add internal connection.

Add the environment variables

Click the Environment variables tab, then click Add environment variables. Add the three variables from the table after these steps. Under Availability, clear Available during buildtime, then click Add environment variables.

Add a readiness probe

Click the Processes tab, click ⋯ on the web process, then click Update process. Under Health checks, turn on Readiness probe, enter the Path /health/liveliness and the Initial delay 10, then click Update process. New instances then take requests only once LiteLLM answers: 27 requests during a deployment all succeeded.

Turn HTTP traffic off

Click the Networking tab. In Expose HTTP traffic, click Disable, then click Disable HTTP. To reach LiteLLM from the internet instead, see Make LiteLLM public.

Deploy

Click the Overview tab, then click Deploy now under Recent deployments. Check the Branch and click Deploy.

Variable	Value
`LITELLM_MASTER_KEY`	The admin key: it creates and deletes virtual keys and calls every model. Generate one with `printf 'sk-%s\n' "$(openssl rand -hex 32)"`
`LITELLM_SALT_KEY`	Encrypts the provider credentials LiteLLM stores in the database. Generate it the same way, once: if it changes, stored credentials can’t be read
`OLLAMA_API_BASE`	`http://` plus Ollama’s internal hostname plus `:8080`, for example `http://ollama-k3j4h-web.ollama-k3j4h.svc.cluster.local:8080`. Copy the hostname from Internal hostnames on the `ollama` application’s Networking tab

Both keys are runtime only: a build-time variable becomes a build argument of the image build.

Verify the deployment

On the Deployments tab, click the deployment. Its Build log shows 📦 Building and pushing docker image using Dockerfile..., and its Dockerfile tab shows your Dockerfile.
When the deployment is Deployed, open the application’s runtime logs. The first start applies the database migrations (Applying migration lines, then No pending migrations to apply. on later starts) and lists the models:
LiteLLM: Proxy initialized with Config, Set models: lfm3 embeddinggemma INFO: Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)

In the application’s web terminal, check that the proxy answers:

lite --base-url "http://localhost:$PORT" --api-key "$LITELLM_MASTER_KEY" http request GET /health/liveliness

"I'm alive!"

With HTTP traffic off, the application’s domain returns 404:

curl -s -o /dev/null -w '%{http_code}\n' https://litellm-q9w3e.sevalla.app/health/liveliness

Create a key for each application

Give every application its own virtual key: its spend is tracked separately, it can have a budget, and you can delete it without touching the others. Create keys with LiteLLM’s lite command in the application’s web terminal, which calls the proxy inside the same container with the master key:

lite --base-url "http://localhost:$PORT" --api-key "$LITELLM_MASTER_KEY" \
  http request POST /key/generate -j '{"key_alias": "open-webui", "max_budget": 10}'

The answer is the new key’s settings, with "key_alias": "open-webui", "max_budget": 10.0 and the key itself in "key", which starts with sk-. LiteLLM stores only a hash of the key, so copy it now. Once the key’s spend passes its max_budget, in US dollars, LiteLLM rejects its requests.

Field	Effect
`key_alias`	The name the key appears under, for example the application’s name
`max_budget`	The most the key can spend, in US dollars. Leave it out for no limit
`models`	A list such as `["lfm3"]` limits the key to those models. A request for another model gets a `403` with `"type":"key_model_access_denied"`. Without it, the key can call every model in `config.yaml`, including ones you add later

Set the budget with max_budget as shown: the --spend option of lite keys generate sets what the key has already spent, not a limit. To delete a key, run the same lite command with keys delete --key-aliases open-webui.

Call LiteLLM from an application

On the calling application’s Networking tab, add an internal connection to litellm. Then add two variables on its Environment variables tab and deploy it:

Variable	Value
`OPENAI_BASE_URL`	`http://` plus LiteLLM’s internal hostname plus `:8080/v1`, for example `http://litellm-q9w3e-web.litellm-q9w3e.svc.cluster.local:8080/v1`
`OPENAI_API_KEY`	The application’s virtual key

The official OpenAI SDKs read both variables, so code that uses them needs no change. From the application’s web terminal:

curl "$OPENAI_BASE_URL/chat/completions" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model": "lfm3", "messages": [{"role": "user", "content": "Say hello in five words."}]}'

{
  "id": "chatcmpl-0fb76c43-02e0-41bc-a359-52fd198ffbe9",
  "created": 1793333258,
  "model": "lfm3",
  "object": "chat.completion",
  "choices": [{"finish_reason": "stop", "index": 0, "message": {"content": "Hello, how can I help?", "role": "assistant"}}],
  "usage": {"completion_tokens": 8, "prompt_tokens": 17, "total_tokens": 25}
}

Streaming works the same way: with "stream": true, LiteLLM sends Server-Sent Events as the model writes them. To connect a chat interface, see Deploy Open WebUI.

Add Anthropic and OpenAI models

Hosted models go in the same config.yaml, with their API keys in environment variables of the litellm application only.

Add the models

Add the models to model_list in config.yaml:

config.yaml

  - model_name: claude-haiku-5-5
    litellm_params:
      model: anthropic/claude-haiku-5-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: gpt-6-luna
    litellm_params:
      model: openai/gpt-6-luna
      api_key: os.environ/OPENAI_API_KEY

These are low-cost current models, a sensible default for chat. To use another model, change the part after anthropic/ or openai/ to a name from Anthropic’s or OpenAI’s model list, and model_name to the name your applications send.

Add the provider keys

On the litellm application’s Environment variables tab, add ANTHROPIC_API_KEY and OPENAI_API_KEY. Clear Available during buildtime, then click Add environment variables.

Deploy

Commit and push config.yaml. With Auto-deploy on, the push starts the deployment; otherwise, click Deploy now.

The runtime logs now list four models. Keys created without models can call them too. LiteLLM prices each request from its built-in price list and adds it to the key’s spend. To see it, run in the web terminal:

lite --base-url "http://localhost:$PORT" --api-key "$LITELLM_MASTER_KEY" \
  keys list --key-alias open-webui --format table

After one chat each with claude-haiku-5-5 and gpt-6-luna from Open WebUI, with the titles and suggestions Open WebUI generated for them, the Spend column read 0.001983075, in US dollars. Requests to Ollama models cost 0. To reach models from many providers with one key, add OpenRouter models to the same config.yaml: see OpenRouter in LiteLLM.

Make LiteLLM public

Keep LiteLLM private when only your own applications on Sevalla call it. Make it public when applications or people outside Sevalla need it, for example a laptop running an AI coding tool:

On the application’s Networking tab, in Expose HTTP traffic, click Enable, then click Expose HTTP.
Call it on its domain, for example https://litellm-q9w3e.sevalla.app/v1, with a virtual key.

Every model and admin endpoint then needs a key; only the health checks and the Admin UI’s sign-in page answer without one:

curl https://litellm-q9w3e.sevalla.app/v1/models

{"error":{"message":"Authentication Error, No api key passed in.","type":"auth_error","param":"None","code":"401"}}

The Admin UI is at /ui: sign in with the username admin and the master key as the password. Anyone with the master key controls the proxy and every provider key in it, so keep it in the litellm application’s variables only, hand out virtual keys with budgets, and limit who can reach the domain with IP restrictions where you can. Streams reach the client unbuffered through Sevalla’s proxy: in a test over the public domain, the first of 82 events arrived after 0.89 s and the last after 6.63 s.

Update LiteLLM

Back up the database

Create a backup of litellm-db on its Backups tab. LiteLLM changes the schema when a new version starts, and a backup is the way back. See Backups.

Change the version

In the Dockerfile, change the tag in FROM to the new release, for example from LiteLLM’s releases. Use a release without -rc or -dev in its name.

Deploy

Commit and push. The runtime logs show the migrations the new version applies, then the models.

Deploy changes

Edit config.yaml to add, rename or remove models, then push. With Auto-deploy on, every push to the branch starts a deployment; otherwise, click Deploy now. See Deployments.

Troubleshoot

Symptom	Cause	Fix
`Authentication Error, No api key passed in.`	The request has no `Authorization: Bearer` header	Send a virtual key
`403` with `key_model_access_denied`	The key was created with a `models` list that doesn’t include the model	Create a key without `models`, or with the model in the list
The instance restarts with exit code `137`	LiteLLM ran out of memory	Use S2 or larger. See Troubleshoot runtime errors
Ollama models fail while other models work	LiteLLM has no internal connection to `ollama`, or `OLLAMA_API_BASE` has the wrong host or port	Add the connection and check the variable, then deploy
Requests fail for a few seconds during a deployment	LiteLLM has no readiness probe, or Ollama is restarting: with its disk, it stops before its new instance starts	Add the readiness probe. For Ollama, deploy when few requests run

Deploy with the API

The same deployment as the dashboard steps, as API calls an agent can run in order. It needs:

An API key that can create applications and databases, in SEVALLA_API_KEY.
The repository on a Git provider connected to the company. The API has no call that lists connected accounts: Create application checks access itself and answers 400 with Cannot access repository <url>: Repository not found or you don't have access. when Sevalla can’t read the repository.
The ID of the Ollama application, in OLLAMA_APP_ID.
curl, jq and openssl.

Each step names the API operation, the CLI command (@sevalla/cli 2.1.0) and the tool of the Sevalla MCP server.

Set the variables

export SEVALLA_API=https://api.sevalla.com/v3
export SEVALLA_API_KEY=<your API key>
REPO_URL=https://github.com/acme/litellm-proxy
BRANCH=main
OLLAMA_APP_ID=<the Ollama application's ID>

Find the location

List clusters (listClustersV3):

CLUSTER_ID=$(curl -s "$SEVALLA_API/resources/clusters" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.location == "eu-frankfurt-1") | .id')

Replace eu-frankfurt-1 with the region ID of your location from Locations, and pick by region ID, not by display name. CLI: sevalla resources clusters. MCP: get_platform_options.

Look up the sizes and the PostgreSQL version

List process resource types (listProcessResourceTypesV3), list database resource types (listDatabaseResourceTypesV3) and list database versions (listDatabaseVersionsV3):

SIZE_ID=$(curl -s "$SEVALLA_API/resources/process-resource-types" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.name == "s2") | .id')
DB_SIZE_ID=$(curl -s "$SEVALLA_API/resources/database-resource-types" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.name == "db1") | .id')
PG_VERSION=$(curl -s "$SEVALLA_API/resources/database-versions?cluster_id=$CLUSTER_ID&type=postgresql" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[0].versions | last')

CLI: sevalla resources process-types, sevalla resources db-types and sevalla resources db-versions --cluster $CLUSTER_ID --type postgresql. MCP: get_platform_options.

Create the database

Create database (createDatabaseV3), then get database (getDatabaseV3) until its status is ready. Keep DB_PASSWORD: the API never returns it:

DB_PASSWORD=$(openssl rand -hex 16)
DB_ID=$(curl -s -X POST "$SEVALLA_API/databases" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"display_name\": \"litellm-db\", \"type\": \"postgresql\", \"version\": \"$PG_VERSION\", \"cluster_id\": \"$CLUSTER_ID\", \"resource_type_id\": \"$DB_SIZE_ID\", \"db_name\": \"litellm\", \"db_password\": \"$DB_PASSWORD\"}" \
  | jq -r .id)
for i in $(seq 1 120); do
  DB_STATUS=$(curl -s "$SEVALLA_API/databases/$DB_ID" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" | jq -r '.status // empty')
  [ "$DB_STATUS" = "ready" ] && break
  sleep 15
done
echo "$DB_STATUS"

CLI:

sevalla databases create --name litellm-db --type postgresql --db-version $PG_VERSION --cluster $CLUSTER_ID --resource-type $DB_SIZE_ID --db-name litellm --db-password $DB_PASSWORD

. MCP: create_database, which generates the password itself and never returns it; with connect_to_application_id set to the application and env_var_name DATABASE_URL, it connects the database and stores a postgresql:// URL, which LiteLLM accepts. Create the application first in that case.

Create the application

Create application (createApplicationV3). It doesn’t deploy, and it creates the web process at S1:

APP_ID=$(curl -s -X POST "$SEVALLA_API/applications" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"display_name\": \"litellm\", \"cluster_id\": \"$CLUSTER_ID\", \"source\": \"privateGit\", \"git_type\": \"github\", \"repo_url\": \"$REPO_URL\", \"default_branch\": \"$BRANCH\"}" \
  | jq -r .id)

CLI:

sevalla apps create --name litellm --cluster $CLUSTER_ID --source privateGit --git-type github --repository $REPO_URL --branch $BRANCH

. MCP: create_application with "deploy": false.

Choose the Dockerfile build strategy

Update application (updateApplicationV3):

curl -s -X PATCH "$SEVALLA_API/applications/$APP_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"build_type": "dockerfile", "dockerfile_path": "Dockerfile", "docker_context": "."}' \
  | jq -c '{build_type, dockerfile_path, docker_context}'

{"build_type":"dockerfile","dockerfile_path":"Dockerfile","docker_context":"."}

CLI:

sevalla apps update $APP_ID --data '{"build_type": "dockerfile", "dockerfile_path": "Dockerfile", "docker_context": "."}'

. MCP: sevalla_api_update with updateApplicationV3.

Set the size, the readiness probe and HTTP traffic

List processes (listProcessesV3), then update process (updateProcessV3). Send timeoutSeconds: a probe set through the API without it waits 120 seconds, not the dashboard’s 1:

WEB_ID=$(curl -s "$SEVALLA_API/applications/$APP_ID/processes" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.data[] | select(.type == "web") | .id')
curl -s -X PATCH "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"resource_type_id\": \"$SIZE_ID\", \"is_ingress_enabled\": false, \"readiness_probe\": {\"httpGet\": {\"path\": \"/health/liveliness\", \"port\": 8080}, \"initialDelaySeconds\": 10, \"periodSeconds\": 10, \"timeoutSeconds\": 1, \"successThreshold\": 1, \"failureThreshold\": 3}}" \
  | jq -c '{resource_type_name, is_ingress_enabled}'

{"resource_type_name":"s2","is_ingress_enabled":false}

CLI: sevalla apps processes update $WEB_ID --app-id $APP_ID --resource-type $SIZE_ID sets the size; the CLI can’t set probes or turn HTTP traffic off. MCP: sevalla_api_update with updateProcessV3 and the same body.

Connect the database and Ollama

Create internal connection (createApplicationInternalConnectionV3), once for each:

curl -s -X POST "$SEVALLA_API/applications/$APP_ID/internal-connections" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"target_id\": \"$DB_ID\", \"target_type\": \"database\"}"
curl -s -X POST "$SEVALLA_API/applications/$APP_ID/internal-connections" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"target_id\": \"$OLLAMA_APP_ID\", \"target_type\": \"app\"}"

CLI: sevalla apps internal-connections create $APP_ID --target-id $DB_ID --target-type database. MCP: sevalla_api_create with createApplicationInternalConnectionV3.

Add the environment variables

The API connection adds no variables, so build DATABASE_URL in the dashboard’s format, postgres://<user>:<password>@<host>:<port>/<database>, from get database, and OLLAMA_API_BASE from Ollama’s web process. Then create environment variable (createEnvironmentVariableV3) for each, runtime only:

DATABASE_URL=$(curl -s "$SEVALLA_API/databases/$DB_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r --arg pw "$DB_PASSWORD" '"postgres://\(.db_user):\($pw)@\(.internal_hostname):\(.internal_port)/\(.db_name)"')
OLLAMA_HOSTNAME=$(curl -s "$SEVALLA_API/applications/$OLLAMA_APP_ID/processes" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.data[] | select(.type == "web") | .internal_hostname')
LITELLM_MASTER_KEY="sk-$(openssl rand -hex 32)"

add_var() {
  jq -n --arg key "$1" --arg value "$2" '{key: $key, value: $value, is_runtime: true, is_buildtime: false}' \
    | curl -s -X POST "$SEVALLA_API/applications/$APP_ID/env-vars" \
      -H "Authorization: Bearer $SEVALLA_API_KEY" \
      -H "Content-Type: application/json" \
      -d @- \
    | jq -r .key
}
add_var DATABASE_URL "$DATABASE_URL"
add_var LITELLM_MASTER_KEY "$LITELLM_MASTER_KEY"
add_var LITELLM_SALT_KEY "sk-$(openssl rand -hex 32)"
add_var OLLAMA_API_BASE "http://$OLLAMA_HOSTNAME:8080"

Keep LITELLM_MASTER_KEY for the next steps. List environment variables (listEnvironmentVariablesV3) returns the values later too; the MCP server withholds them. When config.yaml lists hosted models, also run add_var ANTHROPIC_API_KEY "$ANTHROPIC_API_KEY" and add_var OPENAI_API_KEY "$OPENAI_API_KEY". CLI: sevalla apps env-vars create $APP_ID --key LITELLM_MASTER_KEY --value "$LITELLM_MASTER_KEY", which makes the variable available during build and runtime. MCP: set_environment_variables with runtime: true and buildtime: false.

Deploy

Trigger deployment (triggerDeploymentV3), then get deployment (getDeploymentV3) until status is success or failed, for at most 30 minutes:

DEPLOYMENT_ID=$(curl -s -X POST "$SEVALLA_API/applications/$APP_ID/deployments" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  | jq -r .id)
for i in $(seq 1 120); do
  STATUS=$(curl -s "$SEVALLA_API/applications/$APP_ID/deployments/$DEPLOYMENT_ID" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" | jq -r '.status // empty')
  case $STATUS in success|failed|cancelled|skipped) break ;; esac
  sleep 15
done
echo "$STATUS"

CLI: sevalla apps deployments trigger $APP_ID, then sevalla apps deployments get $DEPLOYMENT_ID --app-id $APP_ID. MCP: trigger_deployment, then get_deployment.

Check the build and the start

Get application deployment logs (getApplicationDeploymentLogsV3) and get application runtime logs (getApplicationRuntimeLogsV3):

curl -s "$SEVALLA_API/applications/$APP_ID/deployments/$DEPLOYMENT_ID/logs?limit=5000" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[].message' | grep -m1 'Building and pushing'
curl -s "$SEVALLA_API/applications/$APP_ID/runtime-logs?search=Proxy%20initialized" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[].message'

The first prints 📦 Building and pushing docker image using Dockerfile..., the second LiteLLM: Proxy initialized with Config, Set models:. CLI: sevalla apps deployments logs $DEPLOYMENT_ID --app-id $APP_ID and sevalla apps logs runtime $APP_ID --search "Proxy initialized". MCP: get_deployment_logs and search_runtime_logs.

Create a key and list the models

Run a command in a process (execProcessCommandV3) runs lite without a shell, so pass the master key as a value, not as $LITELLM_MASTER_KEY:

VIRTUAL_KEY=$(jq -n --arg mk "$LITELLM_MASTER_KEY" \
    '{command: ["lite", "--base-url", "http://localhost:8080", "--api-key", $mk, "http", "request", "POST", "/key/generate", "-j", "{\"key_alias\": \"open-webui\", \"max_budget\": 10}"], timeout: 60}' \
  | curl -s -X POST "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/exec" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" \
    -H "Content-Type: application/json" \
    -d @- \
  | jq -r .stdout | jq -r .key)

jq -n --arg vk "$VIRTUAL_KEY" \
    '{command: ["lite", "--base-url", "http://localhost:8080", "--api-key", $vk, "http", "request", "GET", "/v1/models"], timeout: 60}' \
  | curl -s -X POST "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/exec" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" \
    -H "Content-Type: application/json" \
    -d @- \
  | jq -r .stdout | jq -r '.data[].id'

With the hosted models in config.yaml, the second command prints:

lfm3
embeddinggemma
claude-haiku-5-5
gpt-6-luna

VIRTUAL_KEY is Open WebUI’s key. CLI:

sevalla apps processes exec $WEB_ID --app-id $APP_ID --timeout 60 -- lite --base-url http://localhost:8080 --api-key <master key> http request GET /health/liveliness

. MCP: sevalla_api_update with execProcessCommandV3.

Get started

Migrate

AI

JavaScript

Python

PHP

Ruby

Go

Rust

Java and Kotlin

.NET

Elixir

Static site generators

Deploy the LiteLLM proxy

At a glance

Environment variables

Before you begin

How LiteLLM runs on Sevalla

Create the project

Deploy to Sevalla

Verify the deployment

Create a key for each application

Call LiteLLM from an application

Add Anthropic and OpenAI models

Make LiteLLM public

Update LiteLLM

Deploy changes

Troubleshoot

Deploy with the API

​At a glance

​Environment variables

​Before you begin

​How LiteLLM runs on Sevalla

​Create the project

​Deploy to Sevalla

​Verify the deployment

​Create a key for each application

​Call LiteLLM from an application

​Add Anthropic and OpenAI models

​Make LiteLLM public

​Update LiteLLM

​Deploy changes

​Troubleshoot

​Deploy with the API

At a glance

Environment variables

Before you begin

How LiteLLM runs on Sevalla

Create the project

Deploy to Sevalla

Verify the deployment

Create a key for each application

Call LiteLLM from an application

Add Anthropic and OpenAI models

Make LiteLLM public

Update LiteLLM

Deploy changes

Troubleshoot

Deploy with the API