---
name: deploy-litellm-free-vps
description: Set up or update a containerized LiteLLM gateway on a Linux VPS using the F24 Sales free-model YAML, locally supplied aggregator keys, Caddy, Tailscale and/or Cloudflare Tunnel, and twice-daily systemd configuration refreshes. Use for an authorized gateway deployment, not for scanning providers or modifying the F24 Sales website.
---

# Deploy a LiteLLM free-model gateway

Build a working OpenAI-compatible API for the user's own clients or agent worker.
Use the user's available VPS and container runtime; inspect the existing services
before choosing paths and ports. The examples below use Docker Compose on a Linux
host with systemd. Adapt them to existing Podman/Quadlet conventions when needed;
do not install a second runtime just to follow an example.

## Inputs and source of truth

- Current tested routes: https://www.f24-sales.com/litellm-config.yaml
- Static blank credentials template: https://www.f24-sales.com/env_example
- This skill: https://www.f24-sales.com/SKILL.md
- Ask only for missing deployment choices: target host, Tailscale and/or Cloudflare,
  and the desired Cloudflare hostname if applicable. Do not ask for provider keys
  in chat. The user fills the private `.env` directly on the VPS.
- Keep all aggregator slots in `.env`; absence from the current YAML does not
  remove a slot. Never overwrite an existing `.env` when refreshing the catalog.
- The downloaded YAML owns model IDs, upstream IDs, API bases, order and model
  parameters. Do not hardcode a model list, invent metadata, change reasoning
  settings, or exclude a working route because descriptive metadata is missing.
- Hosting credits and free API quotas are independent. Availability, limits and
  regional restrictions can change. Do not promise unlimited requests or bypass
  limits of an orchestrator such as Grok Bot.

## Files and credentials

Use one deployment directory, for example `/opt/litellm-free`, containing
`compose.yaml`, `Caddyfile`, `.env`, `update_catalog.py`, and `config/`.
Put the downloaded original in `config/source.yaml` and the selected runtime
configuration in `config/config.yaml`. Keep all state under this directory.

Download `env_example` once and copy it to `.env` only if `.env` does not exist.
Set mode `0600`. Let the user fill their available aggregator keys; the remaining
keys stay blank. Generate a strong `LITELLM_MASTER_KEY` directly into the private
file without printing it. It authenticates this gateway, not the providers.

Before starting, choose maintained prebuilt LiteLLM, Caddy and optionally
cloudflared releases. Verify that their manifests support the VPS architecture
(`linux/amd64` or `linux/arm64`), and record version or digest references in
`LITELLM_IMAGE`, `CADDY_IMAGE` and `CLOUDFLARED_IMAGE`. Respect any existing image
management policy. Catalog refreshes must not pull or upgrade images.

For a personal API-only gateway, use LiteLLM's file-based mode without a database.
Virtual keys, database-backed budgets and spend tracking require a separate
Postgres deployment; add that only when the user needs those features. A
database-free gateway does not enforce a monetary budget.

## Containers and Caddy

Start with this Compose layout after generating a nonempty runtime config:

```yaml
name: litellm-free
services:
  litellm:
    image: ${LITELLM_IMAGE:?Set a verified LiteLLM release image}
    restart: unless-stopped
    env_file: .env
    command: ["--config", "/etc/litellm/config.yaml", "--host", "0.0.0.0", "--port", "4000"]
    volumes:
      - ./config:/etc/litellm:ro
  caddy:
    image: ${CADDY_IMAGE:?Set a verified Caddy release image}
    restart: unless-stopped
    ports:
      - "127.0.0.1:8080:8080"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile:ro
      - caddy-data:/data
      - caddy-config:/config
volumes:
  caddy-data:
  caddy-config:
```

Both containers use the Compose network. LiteLLM's port 4000 is not published.
The **directory** mount for `config/` is intentional: replacing a single
file atomically must be visible inside the container. A single-file bind mount
can remain attached to the old inode after a replacement.

Use this Caddyfile for the API entrance:

```caddyfile
:8080 {
    handle /v1/* {
        reverse_proxy litellm:4000
    }
    handle {
        respond "Not found" 404
    }
}
```

Preserve `/v1` and the client's Authorization header. Do not use `handle_path`,
which strips the matched prefix. Caddy forwards chat streaming; verify streaming
with the chosen model. Its HTTP listener is restricted to the local machine and
container network. HTTPS is provided by the selected entrance below.

## Entrance 1: Tailscale

Install Tailscale from its official repository on the VPS, enable
`tailscaled.service`, and join the user's tailnet. Complete any interactive
login with the user; reuse an existing registration rather than resetting it.
Enable MagicDNS/HTTPS as needed. Apply the user's tailnet access rules.

With Caddy listening on host loopback port 8080:

```bash
sudo tailscale serve --bg http://127.0.0.1:8080
tailscale serve status
```

The client base URL is `https://<machine>.<tailnet>.ts.net/v1`.
Serve supplies the HTTPS certificate and forwards to Caddy. Keep it private to
the tailnet; do not enable Funnel for this path. If port 443 already serves
another app, reuse its routing or select a free HTTPS port without resetting
existing routes. Keep the working administration connection until the new
route is verified. Confirm persistence after the relevant services restart.

## Entrance 2: Cloudflare Tunnel

Use a domain controlled by the user. Create or reuse a remotely managed tunnel,
then store its connector token as `TUNNEL_TOKEN` in the private `.env`.
Add this service to the same Compose network:

```yaml
  cloudflared:
    image: ${CLOUDFLARED_IMAGE:?Set a verified cloudflared release image}
    restart: unless-stopped
    command: ["tunnel", "--no-autoupdate", "run"]
    environment:
      TUNNEL_TOKEN: ${TUNNEL_TOKEN:?Set the private tunnel token}
```

Configure the chosen hostname, for example `llm.example.com`, with origin service
`http://caddy:8080`. Cloudflare provides public HTTPS. The connector makes outbound
connections; no public ingress port on the VPS is required. If cloudflared runs
on the host instead, the origin is `http://127.0.0.1:8080`.

Protect a private gateway with a Cloudflare Access application. Browser users
can use an email allowlist/OTP policy. Automated clients need a Service Auth
policy and the `CF-Access-Client-Id` and `CF-Access-Client-Secret` headers in
addition to their LiteLLM `Authorization: Bearer ...` header; an OTP login page
is not API authentication. Keep these client secrets outside version control.
The client base URL is `https://llm.example.com/v1`.

Tailscale and Cloudflare can coexist, both forwarding to the same Caddy and
LiteLLM containers. Do not add a duplicate LiteLLM instance for the second URL.

## Safe catalog refresh

Implement `/opt/litellm-free/update_catalog.py` using the host's Python and a
safe YAML parser (for example the distribution's `python3-yaml`). Give it a
`--prepare-only` mode for initial setup. Its normal mode must:

1. Lock the deployment's refresh lock to prevent overlapping updates. Fetch the
   fixed HTTPS YAML URL with certificate verification, a timeout and a bounded
   response size. Reject a non-200 response, HTML, invalid YAML or an empty
   `model_list`. Preserve the last working configuration on download failure.
2. Select routes whose required `os.environ/VARIABLE` key references have nonblank
   values in the service environment. Inspect presence locally without logging
   the values. Keep references as references, never substitute keys into YAML.
   Log skipped route IDs and missing variable **names**. A missing key or invalid
   route only skips that route; keep the other valid routes and their order.
3. Keep exact `model_name` and `litellm_params` values, including reasoning options.
   Retain usable `model_info`; missing descriptive fields are not an error.
   Set `general_settings.master_key: os.environ/LITELLM_MASTER_KEY` in the runtime
   configuration. Do not allow downloaded settings to disable authentication.
4. If no configured routes remain, preserve the previous runtime file and report
   the reason. On first setup, report which key names need filling and do not
   start an empty gateway. Never replace the user's `.env`.
5. Write validated source and runtime candidates in the mounted `config/`
   directory, then atomically replace the corresponding files. If the runtime
   bytes are unchanged, avoid an unnecessary container restart. In prepare-only
   mode, stop here without starting or restarting containers.
6. Apply changed configuration by recreating **only** LiteLLM with the already
   installed image, for example `docker compose up -d --no-deps --force-recreate
   --pull never --no-build litellm`. Verify readiness inside the running container
   and authenticated `/v1/models` through Caddy. On failure, restore the previous
   runtime file, recreate LiteLLM once and report the failure. Do not loop forever.
7. Log the fetch time in UTC, selected/skipped counts and reload outcome to the
   journal without credentials. A provider's inference failure must not remove
   unrelated models or prevent subsequent scheduled refreshes.

After editing `.env`, recreate LiteLLM explicitly even if YAML bytes are
unchanged: an ordinary container restart does not reload its environment.
Never use the catalog job to recreate Caddy or disconnect either entrance.

## systemd schedule: 00:30 and 12:30 UTC

Install `/etc/systemd/system/litellm-free-catalog.service`:

```ini
[Unit]
Description=Refresh the F24 Sales model YAML and reload LiteLLM
Wants=network-online.target
After=network-online.target docker.service

[Service]
Type=oneshot
WorkingDirectory=/opt/litellm-free
EnvironmentFile=/opt/litellm-free/.env
ExecStart=/usr/bin/python3 /opt/litellm-free/update_catalog.py
TimeoutStartSec=5min
UMask=0077
```

Install `/etc/systemd/system/litellm-free-catalog.timer`:

```ini
[Unit]
Description=Fetch the free-model YAML at 00:30 and 12:30 UTC

[Timer]
OnCalendar=*-*-* 00,12:30:00 UTC
Persistent=true
AccuracySec=1s
RandomizedDelaySec=0
Unit=litellm-free-catalog.service

[Install]
WantedBy=timers.target
```

Use this **systemd timer**, not a second crontab entry. UTC scheduling must not
shift with the VPS timezone or daylight saving. `Persistent=true` catches up
after downtime. On a rootless Podman deployment, install equivalent user units,
use the deployment owner's paths and restart only its LiteLLM Quadlet service;
enable lingering when appropriate for running without an interactive login.

For the Compose example: generate the first config, start the selected containers,
enable Docker at boot, then validate and enable the timer:

```bash
sudo systemd-analyze verify /etc/systemd/system/litellm-free-catalog.service /etc/systemd/system/litellm-free-catalog.timer
sudo systemctl daemon-reload
sudo systemctl enable --now litellm-free-catalog.timer
sudo systemctl start litellm-free-catalog.service
systemctl list-timers litellm-free-catalog.timer
journalctl -u litellm-free-catalog.service -n 30 --no-pager
```

## Verification, handoff and GitHub sync

Verify that `/v1/models` rejects unauthenticated requests and returns 200 with
the gateway bearer. Pick an ID from that response and send one user message
whose content is exactly `howdy!`, with `max_tokens: 16384`. Check the actual
answer and streaming, not just HTTP acceptance. Report rate limits or regional
restrictions accurately; never claim every listed route is unlimited or worked
unless it was tested.

Give the user's existing agent/worker the selected base URL and a local secret
reference. LiteLLM serves inference; it is not itself a job queue or shell
executor. For a delegation workflow, keep the orchestrator's original task,
worker status, results and failures explicit. Use the already chosen handoff
interface rather than inventing an unauthenticated execution endpoint.

If GitHub sync is requested, commit the skill, blank template, Compose/Caddy
configuration, updater and units. Exclude `.env`, connector credentials, live
state and any private results. Do not run blanket `git add .`; inspect staged
content. Push/pull direction and publication of results require the user's
chosen workflow, not an assumption that every result should be public.

Finish with the working client URL(s), exact deployment paths, image versions,
authentication check, test outcome, next UTC refresh and recovery command. Name
any missing provider credentials by variable name only. Do not claim deployment
success based solely on written files or passing unit tests.

## Official references

- [LiteLLM container quickstart and database-free mode](https://docs.litellm.ai/docs/proxy/docker_quick_start)
- [LiteLLM deployment and image selection](https://docs.litellm.ai/docs/proxy/deploy)
- [Caddy reverse proxy](https://caddyserver.com/docs/caddyfile/directives/reverse_proxy)
- [Tailscale Serve](https://tailscale.com/docs/reference/tailscale-cli/serve)
- [Cloudflare tunnel tokens](https://developers.cloudflare.com/tunnel/reference/tunnel-tokens/)
- [Cloudflare Access service tokens](https://developers.cloudflare.com/cloudflare-one/access-controls/service-credentials/service-tokens/)