Skip to main content
AnythingLLM runs on Sevalla as an application from its official Docker image, mintplexlabs/anythingllm: the web process serves the chat interface, the developer API and the document collector, and a disk keeps everything AnythingLLM stores, from its SQLite database and your uploaded documents to the vectors in LanceDB and the embedding model. AnythingLLM embeds documents on the CPU with its built-in embedder, so a document chat needs only a model provider: Anthropic in this guide, or LiteLLM or Ollama on Sevalla over an internal connection.

At a glance

Environment variables

An application from a Docker image has no build, so every variable is a runtime variable. Set the environment variables explains each one.

Before you begin

  • A Sevalla company with a payment method.
  • An API key from Anthropic, or a LiteLLM proxy or an Ollama application in the location where AnythingLLM will run.
  • OpenSSL on your computer, to generate the secrets (openssl version prints its version).

How AnythingLLM runs on Sevalla

Create the application

1

Choose the image

In the sidebar, click Applications, then click Create > Application. Under Source, click Docker image and enter mintplexlabs/anythingllm:1.19.0 in Docker image. Keep the Registry credential at None (public image).
2

Name, place and size it

Enter the Name anythingllm, choose the Data center, and choose the Size S1. To use LiteLLM or Ollama, choose the location where they run.
3

Create it without deploying

Click Create application. The application opens with the status Not deployed yet.
4

Add the disk

Click the Disks tab, then click Create disk. In Process, pick the web process. Enter the Path /app/server/storage, pick the Size 10 GB, and click Create disk.
The tag pins one AnythingLLM release: 1.19.0 is the current one. AnythingLLM migrates its database when a new version starts, so pin a version and update on purpose.

Set the environment variables

Generate the password and the three secrets on your computer, and store them in your password manager:
On the application’s Environment variables tab, click Add environment variables, paste these lines into the first key field, replace the password, the secrets and the Anthropic key with yours, and click Add environment variables:
.env

Add a readiness probe

A readiness probe tells Sevalla when the new version is ready to receive requests.
1

Open the web process

Click the Processes tab, click ⋯ on the web process, then click Update process.
2

Turn on the probe

Under Health checks, turn on Readiness probe, enter the Path /api/ping, the Initial delay 10 and the Timeout 5. Keep the Port at 8080.
3

Save

Click Update process.

Deploy AnythingLLM

Click the Overview tab, click Deploy now, then click Deploy. The deployment pulls the image (about 1.1 GB) and starts the web process; there’s no build. The first start creates the database on the disk.

Verify the deployment

  1. On the Deployments tab, the deployment’s status is Deployed. With the API, Get deployment (getDeploymentV3) returns "status": "success".
  2. The application’s runtime logs show Primary server in HTTP mode listening on port 8080.
  3. The health endpoint answers:
  4. Without the password, AnythingLLM’s APIs refuse requests:
  5. Open the domain. AnythingLLM shows Welcome and a password field. Enter AUTH_TOKEN and click Login.

Chat with a document

A workspace is a set of documents and a chat. AnythingLLM splits each document into chunks, embeds them, and sends the chunks that match a question to the model with it.
1

Create a workspace

In AnythingLLM’s sidebar, click New Workspace, enter the Workspace Name Support, and click Save.
2

Upload a document

Click the upload icon next to the workspace’s name (Upload documents to this workspace for RAG indexing). Click Click to upload or drag and drop and pick a file, for example a shipping policy.
3

Embed it

Select the file under My Documents, click Move to Workspace, then click Save and Embed.
4

Ask about it

Close the window, type a question about the document in Send a message, and press Enter. The answer streams in and lists the document under its sources.
The upload, the chunks, the vectors and the chat history are on the disk, so they survive deployments. In the tests, a workspace, its documents and an API key were all still there after a deployment, and the same session stayed signed in.

Use the API

AnythingLLM’s developer API manages workspaces and documents and runs chats. Every request needs an API key in the Authorization header.
1

Create an API key

Click Open settings (the wrench icon at the bottom of the sidebar), then Tools > Developer API. Click Generate New API Key, optionally enter a Name, click Create API Key, and copy the key.
2

Create a workspace

3

Upload and embed a document

Write a small document, upload it, and add it to the workspace in the same request with addToWorkspaces:
4

Ask about it

"mode":"query" answers only from the workspace’s documents:
The model writes the answer in its own words, so yours can differ. In the test, the answer took 2.6 seconds. Without a key, or with a wrong one, the API answers 403 with {"error":"No valid api key found."}. Store the key in the calling application’s environment variables, never in front-end code: anyone with it can read every workspace and spend from your model account. /api/v1/workspace/support/stream-chat takes the same body and returns the answer as Server-Sent Events (Content-Type: text/event-stream, Cache-Control: no-cache): one textResponseChunk event per piece of text, then finalizeResponseStream. Through Cloudflare and Sevalla’s proxy, the first chunk arrived after 1.19 s and the last after 1.68 s, without buffering. The full API reference is at /api/docs on your domain.

Embed large documents

The built-in embedder runs on the web process’s CPU, and the upload request waits until the document is embedded. On S1, a 618 KB text file became 666 chunks and took 3 minutes 20 seconds to embed. Through Cloudflare, a request that sends nothing for about 125 seconds ends with 524: in the test, the upload request returned 524 with the page title 524: A timeout occurred after 125.7 seconds. AnythingLLM kept embedding, and the document was in the workspace about a minute later. For large documents, upload them in the interface, which shows the progress, or check the workspace after the 524. A larger size embeds faster.

Use LiteLLM or Ollama on Sevalla

AnythingLLM can send chats to a model provider that runs on Sevalla instead of Anthropic. Both stay private: their HTTP traffic is off, and AnythingLLM reaches them over an internal connection.
1

Connect the provider

On the anythingllm application’s Networking tab, under Connected services, click Add internal connection and choose litellm or ollama. It needs no variables. Copy the provider’s hostname from Internal hostnames on its own Networking tab, for example litellm-q9w3e-web.litellm-q9w3e.svc.cluster.local.
2

Change the variables

On the Environment variables tab, change LLM_PROVIDER and add the provider’s variables:
.env
3

Deploy

Click the Overview tab, click Deploy now, then click Deploy.
LITE_LLM_MODEL_PREF is a model_name from LiteLLM’s config.yaml, and LITE_LLM_API_KEY is a virtual key for AnythingLLM, so LiteLLM tracks its spend. In the test, with LiteLLM’s HTTP traffic off and claude-haiku-5-5 behind it, the document question got the same answer through LiteLLM in 6.1 seconds. AnythingLLM sends no temperature unless you set one on the workspace, which matters for claude-haiku-5-5: LiteLLM rejects any value other than 1 for it with claude-haiku-5-5 does not support temperature=0.7. Only temperature=1 is supported. Without LITE_LLM_MODEL_TOKEN_LIMIT or OLLAMA_MODEL_TOKEN_LIMIT, AnythingLLM assumes the model reads 4,096 tokens and sends fewer document chunks with each question. Set it to the model’s context window to send more. The embedder stays the built-in one; to embed with Ollama too, set EMBEDDING_ENGINE=ollama, EMBEDDING_BASE_PATH and EMBEDDING_MODEL_PREF , and embed your documents again.

Keep settings in environment variables

AnythingLLM’s settings pages, such as AI Providers > LLM, save their values into the running process and into /app/server/.env, a file in the image, not on the disk. Each deployment starts from the image again, and the environment variables on Sevalla win. In the tests, changing the model to claude-sonnet-5-5 in AnythingLLM worked until the next deployment, which set it back to ANTHROPIC_MODEL_PREF. Change the model, the provider and the keys on the application’s Environment variables tab, then deploy. Settings that AnythingLLM keeps in its database, such as workspaces, users, a workspace’s own model and the system prompt, live on the disk and stay.

Let your team sign in

With AUTH_TOKEN alone, everyone shares one password and one set of chats. Multi-user mode gives each person an account with a role.
1

Turn on multi-user mode

Click Open settings, then Security. Under Multi-User Mode, turn on Enable Multi-User Mode, enter the Admin account username and Admin account password, and click Save changes.
2

Add people

Click Admin > Users, then Add user for each person. Give each one a role, and add them to workspaces on the workspace’s settings.
Only someone with AUTH_TOKEN can turn multi-user mode on. After that, AnythingLLM asks for a username and a password: AUTH_TOKEN alone returns [001] Invalid login credentials., and existing sessions end with {"error":"Invalid auth token."}. Developer API keys keep working. The accounts are in the database on the disk, so they survive deployments; keep JWT_SECRET, SIG_KEY and SIG_SALT unchanged: in the tests, a new JWT_SECRET signed everyone out, and old sessions got {"error":"Token expired or failed validation."}.

Update AnythingLLM

1

Read the release notes

Read the notes of every version between yours and the new one on AnythingLLM’s GitHub releases.
2

Change the image

On the application’s Settings tab, in the Source section, click Update source, change the tag in Docker image, and click Update source.
3

Deploy

Deploy. AnythingLLM migrates its database on the disk when the new version starts.
Sevalla backs up the disk every day and keeps each backup for 7 days. To go back after a failed update, set the previous tag and contact support to restore the disk from the backup before the update.

Add a custom domain

To serve AnythingLLM from your own domain, such as docs-chat.example.com, add the domain on the application’s Domains tab and create the DNS records Sevalla shows. S1 is larger than H1, so custom domains work. See Custom domains.

Troubleshoot

Deploy with the API

These calls create the same deployment without the dashboard: the application, the disk, the probe, the variables and the first deployment, then a document chat through AnythingLLM’s own API. They need:
  • An API key that can create applications, in SEVALLA_API_KEY, and your Anthropic key in ANTHROPIC_API_KEY.
  • curl, jq and openssl.
The application comes from a public Docker image, so no Git provider needs to be connected.
1

Choose the location and sizes

List clusters (listClustersV3), List process resource types (listProcessResourceTypesV3) and List disk resource types (listDiskResourceTypesV3). Pick the location by its region ID, here Frankfurt:
CLI: sevalla resources clusters, sevalla resources process-types and sevalla resources disk-types. MCP: get_platform_options, and sevalla_api_read with listDiskResourceTypesV3.
2

Create the application

Create application (createApplicationV3) with the Docker image source. It doesn’t start a deployment:
The last two calls are List application domains (listApplicationDomainsV3) and List processes (listProcessesV3). CLI: sevalla apps create --name anythingllm --cluster $CLUSTER_ID --source dockerImage --docker-image mintplexlabs/anythingllm:1.19.0. MCP: create_application with deploy: false, then get_application.
3

Add the disk

Attach or update process disk (updateProcessDiskV3):
CLI: sevalla apps processes disk update $WEB_ID --app-id $APP_ID --resource-type $DISK_10GB --mount-path /app/server/storage. MCP: sevalla_api_update with updateProcessDiskV3.
4

Size the web process and add the readiness probe

Update process (updateProcessV3). The API creates the web process at S1 already; send the size anyway if you change it. Always send timeoutSeconds: without it, the probe gets 120 seconds instead of the dashboard’s 1:
CLI: sevalla apps processes update $WEB_ID --app-id $APP_ID --resource-type $S1 (the CLI can’t set a probe). MCP: sevalla_api_update with updateProcessV3 and the same body.
5

Add the environment variables

Create environment variable (createEnvironmentVariableV3), once per variable. Store AUTH_TOKEN and the three secrets somewhere safe before you send them:
Each call prints the key it added. CLI: sevalla apps env-vars create $APP_ID --key SERVER_PORT --value 8080, once per variable; the CLI makes each variable available during runtime and build, which makes no difference for an image. MCP: set_environment_variables with all of them in variables.
6

Deploy

Trigger deployment (triggerDeploymentV3):
CLI: sevalla apps deployments trigger $APP_ID. MCP: trigger_deployment.
7

Wait for the deployment

Get deployment (getDeploymentV3) until status is final (success, failed, cancelled or skipped), for up to 30 minutes; Get application deployment logs (getApplicationDeploymentLogsV3, with ?limit=5000) shows why one failed:
CLI: sevalla apps deployments get $DEPLOYMENT_ID --app-id $APP_ID and sevalla apps deployments logs $DEPLOYMENT_ID --app-id $APP_ID. MCP: get_deployment and get_deployment_logs.
8

Check AnythingLLM

The health endpoint answers, and Get application runtime logs (getApplicationRuntimeLogsV3) shows the port:
CLI: sevalla apps logs runtime $APP_ID --search "listening on port". MCP: search_runtime_logs.
9

Create an AnythingLLM API key

AnythingLLM’s own API: sign in with AUTH_TOKEN for a session token, then create a developer API key with it:
10

Chat with a document

Create the workspace, upload the document into it, and ask about it:
The last line is the source the answer used; .textResponse holds the answer itself.
Then open https://$APP_HOST in a browser and sign in with AUTH_TOKEN.