mintplexlabs/anythingllm: the web process serves the chat interface, the developer API and the document collector, and a disk keeps everything AnythingLLM stores, from its SQLite database and your uploaded documents to the vectors in LanceDB and the embedding model. AnythingLLM embeds documents on the CPU with its built-in embedder, so a document chat needs only a model provider: Anthropic in this guide, or LiteLLM or Ollama on Sevalla over an internal connection.
At a glance
Environment variables
Before you begin
- A Sevalla company with a payment method.
- An API key from Anthropic, or a LiteLLM proxy or an Ollama application in the location where AnythingLLM will run.
- OpenSSL on your computer, to generate the secrets (
openssl versionprints its version).
How AnythingLLM runs on Sevalla
Create the application
Choose the image
mintplexlabs/anythingllm:1.19.0 in Docker image. Keep the Registry credential at None (public image).Name, place and size it
anythingllm, choose the Data center, and choose the Size S1. To use LiteLLM or Ollama, choose the location where they run.Create it without deploying
Add the disk
/app/server/storage, pick the Size 10 GB, and click Create disk.1.19.0 is the current one. AnythingLLM migrates its database when a new version starts, so pin a version and update on purpose.
Set the environment variables
Generate the password and the three secrets on your computer, and store them in your password manager:Add a readiness probe
A readiness probe tells Sevalla when the new version is ready to receive requests.Open the web process
Turn on the probe
/api/ping, the Initial delay 10 and the Timeout 5. Keep the Port at 8080.Save
Deploy AnythingLLM
Click the Overview tab, click Deploy now, then click Deploy. The deployment pulls the image (about 1.1 GB) and starts the web process; there’s no build. The first start creates the database on the disk.Verify the deployment
-
On the Deployments tab, the deployment’s status is Deployed. With the API, Get deployment (
getDeploymentV3) returns"status": "success". -
The application’s runtime logs show
Primary server in HTTP mode listening on port 8080. -
The health endpoint answers:
-
Without the password, AnythingLLM’s APIs refuse requests:
-
Open the domain. AnythingLLM shows Welcome and a password field. Enter
AUTH_TOKENand click Login.
Chat with a document
A workspace is a set of documents and a chat. AnythingLLM splits each document into chunks, embeds them, and sends the chunks that match a question to the model with it.Create a workspace
Support, and click Save.Upload a document
Embed it
Ask about it
Use the API
AnythingLLM’s developer API manages workspaces and documents and runs chats. Every request needs an API key in theAuthorization header.
Create an API key
Create a workspace
Upload and embed a document
addToWorkspaces:Ask about it
"mode":"query" answers only from the workspace’s documents:403 with {"error":"No valid api key found."}. Store the key in the calling application’s environment variables, never in front-end code: anyone with it can read every workspace and spend from your model account.
/api/v1/workspace/support/stream-chat takes the same body and returns the answer as Server-Sent Events (Content-Type: text/event-stream, Cache-Control: no-cache): one textResponseChunk event per piece of text, then finalizeResponseStream. Through Cloudflare and Sevalla’s proxy, the first chunk arrived after 1.19 s and the last after 1.68 s, without buffering. The full API reference is at /api/docs on your domain.
Embed large documents
The built-in embedder runs on the web process’s CPU, and the upload request waits until the document is embedded. On S1, a 618 KB text file became 666 chunks and took 3 minutes 20 seconds to embed. Through Cloudflare, a request that sends nothing for about 125 seconds ends with524: in the test, the upload request returned 524 with the page title 524: A timeout occurred after 125.7 seconds. AnythingLLM kept embedding, and the document was in the workspace about a minute later. For large documents, upload them in the interface, which shows the progress, or check the workspace after the 524. A larger size embeds faster.
Use LiteLLM or Ollama on Sevalla
AnythingLLM can send chats to a model provider that runs on Sevalla instead of Anthropic. Both stay private: their HTTP traffic is off, and AnythingLLM reaches them over an internal connection.Connect the provider
anythingllm application’s Networking tab, under Connected services, click Add internal connection and choose litellm or ollama. It needs no variables. Copy the provider’s hostname from Internal hostnames on its own Networking tab, for example litellm-q9w3e-web.litellm-q9w3e.svc.cluster.local.Change the variables
LLM_PROVIDER and add the provider’s variables:- LiteLLM
- Ollama
Deploy
LITE_LLM_MODEL_PREF is a model_name from LiteLLM’s config.yaml, and LITE_LLM_API_KEY is a virtual key for AnythingLLM, so LiteLLM tracks its spend. In the test, with LiteLLM’s HTTP traffic off and claude-haiku-5-5 behind it, the document question got the same answer through LiteLLM in 6.1 seconds. AnythingLLM sends no temperature unless you set one on the workspace, which matters for claude-haiku-5-5: LiteLLM rejects any value other than 1 for it with claude-haiku-5-5 does not support temperature=0.7. Only temperature=1 is supported.
Without LITE_LLM_MODEL_TOKEN_LIMIT or OLLAMA_MODEL_TOKEN_LIMIT, AnythingLLM assumes the model reads 4,096 tokens and sends fewer document chunks with each question. Set it to the model’s context window to send more. The embedder stays the built-in one; to embed with Ollama too, set EMBEDDING_ENGINE=ollama, EMBEDDING_BASE_PATH and EMBEDDING_MODEL_PREF , and embed your documents again.
Keep settings in environment variables
AnythingLLM’s settings pages, such as AI Providers > LLM, save their values into the running process and into/app/server/.env, a file in the image, not on the disk. Each deployment starts from the image again, and the environment variables on Sevalla win. In the tests, changing the model to claude-sonnet-5-5 in AnythingLLM worked until the next deployment, which set it back to ANTHROPIC_MODEL_PREF.
Change the model, the provider and the keys on the application’s Environment variables tab, then deploy. Settings that AnythingLLM keeps in its database, such as workspaces, users, a workspace’s own model and the system prompt, live on the disk and stay.
Let your team sign in
WithAUTH_TOKEN alone, everyone shares one password and one set of chats. Multi-user mode gives each person an account with a role.
Turn on multi-user mode
Add people
AUTH_TOKEN can turn multi-user mode on. After that, AnythingLLM asks for a username and a password: AUTH_TOKEN alone returns [001] Invalid login credentials., and existing sessions end with {"error":"Invalid auth token."}. Developer API keys keep working. The accounts are in the database on the disk, so they survive deployments; keep JWT_SECRET, SIG_KEY and SIG_SALT unchanged: in the tests, a new JWT_SECRET signed everyone out, and old sessions got {"error":"Token expired or failed validation."}.
Update AnythingLLM
Read the release notes
Change the image
Deploy
Add a custom domain
To serve AnythingLLM from your own domain, such asdocs-chat.example.com, add the domain on the application’s Domains tab and create the DNS records Sevalla shows. S1 is larger than H1, so custom domains work. See Custom domains.
Troubleshoot
Deploy with the API
These calls create the same deployment without the dashboard: the application, the disk, the probe, the variables and the first deployment, then a document chat through AnythingLLM’s own API. They need:- An API key that can create applications, in
SEVALLA_API_KEY, and your Anthropic key inANTHROPIC_API_KEY. curl,jqandopenssl.
Choose the location and sizes
listClustersV3), List process resource types (listProcessResourceTypesV3) and List disk resource types (listDiskResourceTypesV3). Pick the location by its region ID, here Frankfurt:sevalla resources clusters, sevalla resources process-types and sevalla resources disk-types. MCP: get_platform_options, and sevalla_api_read with listDiskResourceTypesV3.Create the application
createApplicationV3) with the Docker image source. It doesn’t start a deployment:listApplicationDomainsV3) and List processes (listProcessesV3). CLI: sevalla apps create --name anythingllm --cluster $CLUSTER_ID --source dockerImage --docker-image mintplexlabs/anythingllm:1.19.0. MCP: create_application with deploy: false, then get_application.Add the disk
updateProcessDiskV3):sevalla apps processes disk update $WEB_ID --app-id $APP_ID --resource-type $DISK_10GB --mount-path /app/server/storage. MCP: sevalla_api_update with updateProcessDiskV3.Size the web process and add the readiness probe
updateProcessV3). The API creates the web process at S1 already; send the size anyway if you change it. Always send timeoutSeconds: without it, the probe gets 120 seconds instead of the dashboard’s 1:sevalla apps processes update $WEB_ID --app-id $APP_ID --resource-type $S1 (the CLI can’t set a probe). MCP: sevalla_api_update with updateProcessV3 and the same body.Add the environment variables
createEnvironmentVariableV3), once per variable. Store AUTH_TOKEN and the three secrets somewhere safe before you send them:sevalla apps env-vars create $APP_ID --key SERVER_PORT --value 8080, once per variable; the CLI makes each variable available during runtime and build, which makes no difference for an image. MCP: set_environment_variables with all of them in variables.Deploy
triggerDeploymentV3):sevalla apps deployments trigger $APP_ID. MCP: trigger_deployment.Wait for the deployment
getDeploymentV3) until status is final (success, failed, cancelled or skipped), for up to 30 minutes; Get application deployment logs (getApplicationDeploymentLogsV3, with ?limit=5000) shows why one failed:sevalla apps deployments get $DEPLOYMENT_ID --app-id $APP_ID and sevalla apps deployments logs $DEPLOYMENT_ID --app-id $APP_ID. MCP: get_deployment and get_deployment_logs.Check AnythingLLM
getApplicationRuntimeLogsV3) shows the port:sevalla apps logs runtime $APP_ID --search "listening on port". MCP: search_runtime_logs.Create an AnythingLLM API key
AUTH_TOKEN for a session token, then create a developer API key with it:Chat with a document
.textResponse holds the answer itself.https://$APP_HOST in a browser and sign in with AUTH_TOKEN.