Run Ollama

Ollama runs on Sevalla as a private application: the official ollama/ollama image serves the Ollama API from the web process, the models it downloads live on a disk, and your other applications call it over an internal connection, never through the internet. Sevalla’s process sizes have CPUs and memory but no GPUs, so Ollama runs models on the CPU: small models of 1 to 3 billion parameters answer at 16 to 33 tokens per second.

At a glance


Source	The `ollama/ollama:0.40.13` Docker image
Build	None: Sevalla pulls the image
Start command	The image’s own, `/bin/ollama serve`
Port	`8080`, set with `OLLAMA_HOST`. The image’s default is `11434`
Processes	Web process, with HTTP traffic turned off
Services	None. Other applications reach Ollama over an internal connection
Disk	10 GB at `/root/.ollama`, on the web process
Health check	None
Size	S2 for a 1.7B chat model and an embedding model; S3 for a 3B model

Environment variables

Variable	Required	Available during	Secret	Value
`OLLAMA_HOST`	Yes	Runtime	No	`0.0.0.0:8080`
`LLAMA_ARG_THREADS`	No	Runtime	No	The size’s vCPUs: `1` on S2, `2` on S3. Without it, answers are 3 times slower
`OLLAMA_KEEP_ALIVE`	No	Runtime	No	`-1`, to keep models loaded
`OLLAMA_MAX_LOADED_MODELS`	No	Runtime	No	`2`

Before you begin

A Sevalla company with a payment method.
An application in the same location that will call Ollama, or a plan for one, such as LiteLLM or Open WebUI.

How Ollama runs on Sevalla

Part	Setup
Image	`ollama/ollama:0.40.13` from Docker Hub, pinned to a release so every deployment runs the same version
Port	`8080`, set with `OLLAMA_HOST`. The image’s own default is `11434`
Models	A disk mounted at `/root/.ollama`, where Ollama keeps its models and its key. Models survive deployments and restarts
Access	HTTP traffic turned off. Ollama has no authentication, so only services with an internal connection reach it
Hardware	CPU only. Memory decides which models fit; vCPUs decide how fast prompts are read
Instances	1. A process with a disk runs 1 instance and stops for a few moments during each deployment. See Persistent storage

Choose a model and a size

A model needs memory for its weights plus its context, and the process needs room for every model that stays loaded at once. Measured on Sevalla with Ollama 0.40.13, a 4,096-token context, thinking turned off and 160-token answers:

Model	Download	Memory when loaded	S2 (1 vCPU, 2 GB)	S3 (2 vCPU, 4 GB)	C3 (4 vCPU, 4 GB)
`lfm3:1.7b` (chat, tools)	1.1 GB	1.2 GB	20 tokens/s	30 tokens/s	33 tokens/s
`granite4.2:3b` (chat, tools)	2.2 GB	2.7 GB	Doesn’t fit	16 tokens/s	18 tokens/s
`embeddinggemma-2:270m` (embeddings)	303 MB	528 MB	16 sentences in 1.1 s	0.65 s	0.35 s

Writing an answer is limited by memory speed, so doubling the vCPUs from S3 to C3 made answers about 10% faster.
Reading the prompt scales with the vCPUs: lfm3:1.7b read a short prompt at 37, 72 and 134 tokens per second on S2, S3 and C3, and a 2,050-token prompt at 50 tokens per second on S2. Long prompts, such as chat history, documents or the tool definitions a chat app adds, delay the first word by that much: 41 s for the 2,050 tokens.
Memory is a hard limit. With lfm3:1.7b and granite4.2:3b loaded on S3, loading the embedding model too used more than 4 GB, and Sevalla stopped the instance with exit code 137 (out of memory) and restarted it.

This guide uses S2 for lfm3:1.7b and embeddinggemma-2:270m, which use 1.7 GB together. Use S3 for a 3-billion-parameter model. See Instance sizes for every size.

Deploy Ollama

Create the application

In the sidebar, click Applications, then click Create > Application. Under Source, click Docker image and enter ollama/ollama:0.40.13 in Docker image. Keep the Registry credential at None (public image). Enter the Name ollama, choose the Data center where your other applications run, choose the Size S2, then click Create application. Don’t deploy yet.

Add a disk for the models

Click the Disks tab, then click Create disk. In Process, pick the web process. Enter the Path /root/.ollama, pick the Size 10 GB, and click Create disk.

Add the environment variables

Click the Environment variables tab, then click Add environment variables. Add the four variables from the table after these steps, then click Add environment variables.

Turn HTTP traffic off

Click the Networking tab. In Expose HTTP traffic, click Disable, then click Disable HTTP.

Deploy

Click the Overview tab, click Deploy now, then click Deploy.

Variable	Value	Why
`OLLAMA_HOST`	`0.0.0.0:8080`	Ollama listens on all interfaces on port `8080`, the web process port. The `ollama` command in the web terminal reads the same variable to find the server
`LLAMA_ARG_THREADS`	`1`	The number of vCPUs of the size: `1` on S2, `2` on S3, `4` on C3. Without it, the model runner sees the server’s 16 CPUs, starts 8 threads, and the size’s CPU limit throttles them: on S3, `lfm3:1.7b` wrote 11 tokens per second instead of 30
`OLLAMA_KEEP_ALIVE`	`-1`	Models stay loaded until the next restart. The default unloads a model after 5 minutes without requests, and the next request waits while it loads again: 2 to 43 s for `lfm3:1.7b` in the test
`OLLAMA_MAX_LOADED_MODELS`	`2`	At most 2 models in memory. When a request needs a third, Ollama first unloads the one used least recently, so pick a size that holds your two largest models

When the deployment is Deployed, the domains on the Networking tab show HTTP is disabled: Ollama answers only on its internal hostname. The application’s domain returns 404:

curl -s -o /dev/null -w '%{http_code}\n' https://ollama-k3j4h.sevalla.app/

The application’s runtime logs show Listening on [::]:8080 (version 0.40.13).

The Disable HTTP traffic dialog, which says the web process stops receiving HTTP traffic and its domains stop answering, with the Disable HTTP button

Pull the models

Pull models once from the web terminal. They land on the disk, so they’re still there after every deployment and restart:

A pull in a custom start command runs on every start: the instance answers only after Ollama has checked each model against ollama.com, so every deployment waits on it.
A job has no disk, so it can’t keep the models it pulls.

Open the web terminal

On the application’s Web terminal tab, wait for the status Live.

Pull the models

Run:

ollama pull lfm3:1.7b
ollama pull embeddinggemma-2:270m

Each pull ends with success. The 1.1 GB chat model took about 25 seconds.

Check the models

Run ollama list:

NAME                     ID              SIZE      MODIFIED
embeddinggemma-2:270m    c21dd30c1e38    303 MB    Less than a second ago
lfm3:1.7b                b36a103fe2f8    1.1 GB    15 seconds ago

Browse models at ollama.com/library and check each model’s download size against the sizes before you pull it. Some models need a recent Ollama: lfm3 needs 0.40.13 or later. To delete a model and free the disk, run ollama rm <model>.

Connect another application

An application reaches Ollama through an internal connection, on the web process’s internal hostname and port 8080.

Add the internal connection

Open the application that calls Ollama, and click its Networking tab. Under Connected services, click Add internal connection, choose ollama as the Service, and click Add internal connection.

Add the address as a variable

On the ollama application’s Networking tab, copy the web process’s hostname from Internal hostnames, for example ollama-k3j4h-web.ollama-k3j4h.svc.cluster.local. On the calling application’s Environment variables tab, add OLLAMA_BASE_URL with the value http://ollama-k3j4h-web.ollama-k3j4h.svc.cluster.local:8080.

Deploy

Deploy the calling application, so it gets the variable.

The Private networking part of the Networking tab with the Internal hostnames of each process, the Connected services and the Private ports

To check the connection, run curl -s "$OLLAMA_BASE_URL/api/version" in the calling application’s web terminal. It returns {"version":"0.40.13"}. Ollama serves an OpenAI-compatible API under /v1 and its own API under /api. From the calling application’s web terminal (with curl in its image), call the OpenAI-compatible chat endpoint:

curl "$OLLAMA_BASE_URL/v1/chat/completions" \
  -H "Content-Type: application/json" \
  -d '{"model": "lfm3:1.7b", "reasoning_effort": "none", "messages": [{"role": "user", "content": "Say hello in five words."}]}'

{"id":"chatcmpl-915","object":"chat.completion","created":1793333885,"model":"lfm3:1.7b","system_fingerprint":"fp_ollama","choices":[{"index":0,"message":{"role":"assistant","content":"Hello, how are you?"},"finish_reason":"stop"}],"usage":{"prompt_tokens":17,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens":7,"total_tokens":24}}

The native chat endpoint answers with the timings Ollama measured:

curl "$OLLAMA_BASE_URL/api/chat" \
  -d '{"model": "lfm3:1.7b", "think": false, "stream": false, "messages": [{"role": "user", "content": "Say hello in five words."}]}'

{"model":"lfm3:1.7b","created_at":"2026-10-30T04:18:07.293303232Z","message":{"role":"assistant","content":"Hello, how can I help?"},"done":true,"done_reason":"stop","total_duration":950940126,"load_duration":602779,"prompt_eval_count":17,"prompt_eval_cached_count":0,"prompt_eval_duration":590382000,"eval_count":8,"eval_duration":358624000}

eval_count divided by eval_duration (in nanoseconds) is the speed: tokens per second. For embeddings, send {"model": "embeddinggemma-2:270m", "input": "..."} to /v1/embeddings, which returns 768 numbers per input. In code, an OpenAI SDK works with the base URL $OLLAMA_BASE_URL/v1 and any API key: Ollama ignores the Authorization header. To give each application its own key, a spending view and hosted models next to Ollama’s, put LiteLLM in front of Ollama.

Turn thinking off

lfm3:1.7b thinks before it answers unless you turn it off: a question that needed a one-sentence answer produced 453 tokens, most of them reasoning. Send "reasoning_effort": "none" to /v1/chat/completions or "think": false to /api/chat for a direct answer, as in the examples.

Settings that matter in a container

Besides the four variables you added, these Ollama settings change memory and speed:

Variable	Default	Effect
`OLLAMA_CONTEXT_LENGTH`	`4096` without a GPU	The tokens a model can hold: prompt, history and answer. A larger context needs more memory for every loaded model
`OLLAMA_NUM_PARALLEL`	`1`	Requests a model answers at the same time. Others wait in a queue. Each parallel request needs its own context memory
`OLLAMA_NO_CLOUD`	Not set	Set it to `1` to turn off Ollama’s cloud models and web search, so every request runs on your instance

Change a variable on the Environment variables tab, then deploy. After each deployment, the first request to each model loads it from the disk: 43 s for lfm3:1.7b, about 90 s for granite4.2:3b and 11 s for embeddinggemma-2:270m in the test. Later requests start at once while OLLAMA_KEEP_ALIVE is -1.

Update Ollama

Deploy the new version

On the application’s Deployments tab, click Deploy now. Enter the new Tag, for example the latest release from Ollama’s releases, and click Deploy.

Keep the new version

When the deployment works, make the tag permanent: on the Settings tab, in the Source section, click Update source, enter ollama/ollama:<tag> in Docker image, and click Update source.

The models stay on the disk. Ollama stops for a few moments during the deployment, so calling applications get connection errors until it’s back. See Deploy a new image version.

Troubleshoot

Symptom	Cause	Fix
The instance restarts with exit code `137` when a model loads (see Troubleshoot runtime errors)	The loaded models need more memory than the size has	Use a larger size, a smaller model, or a lower `OLLAMA_MAX_LOADED_MODELS`
Answers are 3 times slower than in the table	`LLAMA_ARG_THREADS` is missing, or doesn’t match the size	Set it to the size’s number of vCPUs, then deploy
The first answer after a deployment takes a minute	The model loads from the disk on its first request	Expected. Send one request after each deployment to load it
A chat app waits long before the first word	The prompt is long, and the CPU reads it at tens of tokens per second	Shorten the prompt (in Open WebUI, turn off built-in tools), use a size with more vCPUs, or send long prompts to a hosted model through LiteLLM
The calling application can’t connect	It has no internal connection to `ollama`, or uses port `11434`	Add the internal connection and use port `8080`

Deploy with the API

The same deployment as the dashboard steps, as API calls an agent can run in order. It needs:

An API key that can create and update applications, in SEVALLA_API_KEY.
curl and jq.
The location’s cluster ID, which the first step looks up by the location’s region (eu-frankfurt-1 here).

The image is public, so no Git provider or registry credential is involved. Each step names the API operation, the CLI command (@sevalla/cli 2.1.0) and the tool of the Sevalla MCP server.

Set the API base URL and key

export SEVALLA_API=https://api.sevalla.com/v3
export SEVALLA_API_KEY=<your API key>

Find the location

List clusters (listClustersV3):

CLUSTER_ID=$(curl -s "$SEVALLA_API/resources/clusters" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.location == "eu-frankfurt-1") | .id')

Replace eu-frankfurt-1 with the region ID of your location from Locations, and pick by region ID, not by display name. CLI: sevalla resources clusters. MCP: get_platform_options.

Look up the sizes

List process resource types (listProcessResourceTypesV3) and list disk resource types (listDiskResourceTypesV3):

SIZE_ID=$(curl -s "$SEVALLA_API/resources/process-resource-types" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.name == "s2") | .id')
DISK_SIZE_ID=$(curl -s "$SEVALLA_API/resources/disk-resource-types" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.[] | select(.size_gib == 10) | .id')

CLI: sevalla resources process-types and sevalla resources disk-types. MCP: get_platform_options, and sevalla_api_read with listDiskResourceTypesV3.

Create the application

Create application (createApplicationV3). It doesn’t deploy, and it creates the web process at S1, so the next steps set the size:

APP_ID=$(curl -s -X POST "$SEVALLA_API/applications" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"display_name\": \"ollama\", \"cluster_id\": \"$CLUSTER_ID\", \"source\": \"dockerImage\", \"docker_image\": \"ollama/ollama:0.40.13\"}" \
  | jq -r .id)

CLI: sevalla apps create --name ollama --cluster $CLUSTER_ID --source dockerImage --docker-image ollama/ollama:0.40.13. MCP: create_application with "deploy": false.

Find the web process

List processes (listProcessesV3):

WEB_ID=$(curl -s "$SEVALLA_API/applications/$APP_ID/processes" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.data[] | select(.type == "web") | .id')

CLI: sevalla apps processes list --app-id $APP_ID. MCP: get_application.

Set the size and turn HTTP traffic off

Update process (updateProcessV3). is_ingress_enabled: false is the dashboard’s Disable HTTP:

curl -s -X PATCH "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"resource_type_id\": \"$SIZE_ID\", \"is_ingress_enabled\": false}" \
  | jq '{resource_type_name, is_ingress_enabled, internal_hostname}'

{
  "resource_type_name": "s2",
  "is_ingress_enabled": false,
  "internal_hostname": "ollama-k3j4h-web.ollama-k3j4h.svc.cluster.local"
}

CLI: sevalla apps processes update $WEB_ID --app-id $APP_ID --resource-type $SIZE_ID sets the size; the CLI can’t turn HTTP traffic off. MCP: sevalla_api_update with updateProcessV3 and the same body.

Add the disk

Attach or update process disk (updateProcessDiskV3):

curl -s -X PUT "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/disk" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"resource_type_id\": \"$DISK_SIZE_ID\", \"mount_path\": \"/root/.ollama\"}"

CLI: sevalla apps processes disk update $WEB_ID --app-id $APP_ID --resource-type $DISK_SIZE_ID --mount-path /root/.ollama. MCP: sevalla_api_update with updateProcessDiskV3.

Add the environment variables

Create environment variable (createEnvironmentVariableV3), once per variable:

for VAR in OLLAMA_HOST=0.0.0.0:8080 LLAMA_ARG_THREADS=1 OLLAMA_KEEP_ALIVE=-1 OLLAMA_MAX_LOADED_MODELS=2; do
  curl -s -X POST "$SEVALLA_API/applications/$APP_ID/env-vars" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"key\": \"${VAR%%=*}\", \"value\": \"${VAR#*=}\", \"is_runtime\": true, \"is_buildtime\": false}" \
    | jq -r .key
done

CLI: sevalla apps env-vars create $APP_ID --key OLLAMA_HOST --value 0.0.0.0:8080, once per variable. MCP: set_environment_variables.

Deploy

Trigger deployment (triggerDeploymentV3):

DEPLOYMENT_ID=$(curl -s -X POST "$SEVALLA_API/applications/$APP_ID/deployments" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}' \
  | jq -r .id)

CLI: sevalla apps deployments trigger $APP_ID. MCP: trigger_deployment.

Wait for the deployment

Get deployment (getDeploymentV3) until status is success or failed, for at most 30 minutes:

for i in $(seq 1 120); do
  STATUS=$(curl -s "$SEVALLA_API/applications/$APP_ID/deployments/$DEPLOYMENT_ID" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" | jq -r '.status // empty')
  case $STATUS in success|failed|cancelled|skipped) break ;; esac
  sleep 15
done
echo "$STATUS"

If it fails, read the log with Get application deployment logs (getApplicationDeploymentLogsV3). CLI: sevalla apps deployments get $DEPLOYMENT_ID --app-id $APP_ID and sevalla apps deployments logs $DEPLOYMENT_ID --app-id $APP_ID. MCP: get_deployment and get_deployment_logs.

Pull the models

Run a command in a process (execProcessCommandV3) runs one command without a shell, for up to 60 seconds. Both models finished in that time:

for MODEL in lfm3:1.7b embeddinggemma-2:270m; do
  curl -s -X POST "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/exec" \
    -H "Authorization: Bearer $SEVALLA_API_KEY" \
    -H "Content-Type: application/json" \
    -d "{\"command\": [\"ollama\", \"pull\", \"$MODEL\"], \"timeout\": 60}" \
    | jq .exit_code
done

Each pull prints 0. Pull models that take longer than 60 seconds in the web terminal. CLI: sevalla apps processes exec $WEB_ID --app-id $APP_ID --timeout 60 -- ollama pull lfm3:1.7b. MCP: sevalla_api_update with execProcessCommandV3.

Check the deployment

List the models with the same operation:

curl -s -X POST "$SEVALLA_API/applications/$APP_ID/processes/$WEB_ID/exec" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"command": ["ollama", "list"]}' \
  | jq -r .stdout

NAME                     ID              SIZE      MODIFIED
embeddinggemma-2:270m    c21dd30c1e38    303 MB    10 seconds ago
lfm3:1.7b                b36a103fe2f8    1.1 GB    20 seconds ago

Get application runtime logs (getApplicationRuntimeLogsV3) with search=Listening%20on returns the line msg="Listening on [::]:8080 (version 0.40.13)". CLI: sevalla apps logs runtime $APP_ID --search "Listening on". MCP: search_runtime_logs.

Connect an application

Create internal connection (createApplicationInternalConnectionV3) on the application that calls Ollama, CALLER_APP_ID. The API adds no environment variables, so add OLLAMA_BASE_URL from the web process’s internal_hostname, then deploy the caller:

curl -s -X POST "$SEVALLA_API/applications/$CALLER_APP_ID/internal-connections" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"target_id\": \"$APP_ID\", \"target_type\": \"app\"}"

OLLAMA_HOSTNAME=$(curl -s "$SEVALLA_API/applications/$APP_ID/processes" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  | jq -r '.data[] | select(.type == "web") | .internal_hostname')

curl -s -X POST "$SEVALLA_API/applications/$CALLER_APP_ID/env-vars" \
  -H "Authorization: Bearer $SEVALLA_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{\"key\": \"OLLAMA_BASE_URL\", \"value\": \"http://$OLLAMA_HOSTNAME:8080\", \"is_runtime\": true, \"is_buildtime\": false}"

If the caller already has the variable, the last call answers 400 with Environment variable with key "OLLAMA_BASE_URL" already exists: change it with update environment variable (updateEnvironmentVariableV3) instead. From a process of the caller that has curl, execProcessCommandV3 with ["curl", "-s", "http://<OLLAMA_HOSTNAME>:8080/api/version"] returns {"version":"0.40.13"}. CLI: sevalla apps internal-connections create $CALLER_APP_ID --target-id $APP_ID --target-type app. MCP: sevalla_api_create with createApplicationInternalConnectionV3.

Get started

Migrate

AI

JavaScript

Python

PHP

Ruby

Go

Rust

Java and Kotlin

.NET

Elixir

Static site generators

At a glance

Environment variables

Before you begin

How Ollama runs on Sevalla

Choose a model and a size

Deploy Ollama

Pull the models

Connect another application

Turn thinking off

Settings that matter in a container

Update Ollama

Troubleshoot

Deploy with the API

​At a glance

​Environment variables

​Before you begin

​How Ollama runs on Sevalla

​Choose a model and a size

​Deploy Ollama

​Pull the models

​Connect another application

​Turn thinking off

​Settings that matter in a container

​Update Ollama

​Troubleshoot

​Deploy with the API

At a glance

Environment variables

Before you begin

How Ollama runs on Sevalla

Choose a model and a size

Deploy Ollama

Pull the models

Connect another application

Turn thinking off

Settings that matter in a container

Update Ollama

Troubleshoot

Deploy with the API