Self-Host Phi-3 with Ollama and Open WebUI on a VPS

Run Phi-3 on a virtual private server and use it through Open WebUI in a browser, without relying on a hosted chat service. You’ll install Ollama, download a Phi-3 model, and connect the web interface to it; a Linux VPS with enough memory and a secure way to access it is essential.

What You’ll Need

• A VPS running a current 64-bit Linux distribution, such as Ubuntu, with a public IP address and administrator access.
• At least 8 GB of RAM for a smaller quantized Phi-3 model; 16 GB or more is preferable for smoother use and larger model variants. Check the chosen model’s requirements before downloading it.
• Enough disk space for the model and containers. Allow several gigabytes for a small model, plus space for operating-system files, logs, and future downloads.
• Docker Engine and the Docker Compose plugin, installed from the official instructions for your distribution.
• A domain name and HTTPS reverse proxy are recommended if the interface will be reachable over the internet. Otherwise, plan to restrict access to a trusted IP address or use an SSH tunnel.

Step-by-Step

1. Connect to the VPS and update it. From your computer, connect with SSH using the VPS address and account supplied by your provider. On Ubuntu or Debian, run sudo apt update && sudo apt upgrade -y. Reboot if the update requires it, then reconnect. Use a non-root account with sudo for routine administration.
2. Install Docker. Follow Docker’s official instructions for your Linux distribution to install Docker Engine and the Compose plugin. Verify both are available with docker --version and docker compose version. Avoid downloading installation scripts from unfamiliar sources.
3. Create a project directory. Run mkdir -p ~/phi-webui && cd ~/phi-webui. Create a file named compose.yaml there. The following configuration uses Ollama as the model service and Open WebUI as the browser interface:

services:
  ollama:
    image: ollama/ollama:latest
    restart: unless-stopped
    volumes:
      - ollama_data:/root/.ollama

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    restart: unless-stopped
    depends_on:
      - ollama
    ports:
      - "127.0.0.1:3000:8080"
    environment:
      - OLLAMA_BASE_URL=http://ollama:11434
    volumes:
      - open_webui_data:/app/backend/data

volumes:
  ollama_data:
  open_webui_data:

4. Start the services. In the project directory, run docker compose up -d, then check status with docker compose ps. The Ollama port is intentionally not published to the public network. Open WebUI can reach it over the private Compose network.
5. Download Phi-3 through Ollama. Run docker compose exec ollama ollama pull phi3. This downloads the default Phi-3 model tag, which may change as Ollama updates its catalog. To see available model names and variants, check Ollama’s model library and use the exact tag you want, for example phi3:mini if listed. Confirm the download with docker compose exec ollama ollama list.
6. Open the interface privately first. From your own computer, create an SSH tunnel: ssh -L 3000:127.0.0.1:3000 user@YOUR_VPS_IP, replacing the account and address. Keep that SSH session open, then visit http://localhost:3000 in your browser. This avoids exposing the login page to the public internet while you configure it.
7. Create an Open WebUI account. On first visit, register the initial account. Treat it as an administrator account: choose a unique, long password and do not reuse a password from another service. In the chat model selector, choose the downloaded Phi-3 model. Send a short test prompt and wait for the response.
8. Configure public access only if needed. For a public site, put a maintained reverse proxy such as Caddy or Nginx in front of Open WebUI, configure HTTPS with a valid certificate, and proxy to 127.0.0.1:3000. Keep Ollama’s port 11434 private. Allow only required inbound ports in the VPS firewall and provider firewall—typically SSH and HTTPS, and HTTP only if needed for certificate setup. Do not expose port 3000 directly to the internet.
9. Check persistence and updates. The named volumes keep model files and Open WebUI data when containers are recreated. Back up important data before changing images or deleting volumes. Update deliberately with docker compose pull && docker compose up -d, then check the logs and test a chat.

Common Mistakes

• Choosing an undersized VPS: model loading and generation need memory, and low RAM can cause slowdowns or process termination. Match RAM and disk to the exact model variant, context size, and expected number of users.
• Publishing Ollama’s port: a public Ollama endpoint can let others use your compute. Do not add a public port mapping for 11434; keep it on the Compose network.
• Using the wrong service address: inside the Compose network, Open WebUI should connect to http://ollama:11434, not localhost. In a container, localhost means that same container.
• Removing volumes during cleanup: docker compose down -v deletes named volumes, including downloaded models and WebUI data. Use docker compose down when you only mean to stop and remove containers.
• Opening the WebUI port without protection: a login screen alone is not a substitute for HTTPS, strong credentials, and restricted network access.

Tips

• Start with a smaller Phi-3 variant to confirm the setup works before trying a larger one.
• Use docker compose logs -f ollama or docker compose logs -f open-webui to watch startup and diagnose errors.
• Check free disk space with df -h and memory with free -h before pulling another model.
• Use a VPN or SSH tunnel instead of public access if only you need the service. If you do publish it, keep the operating system, Docker images, and reverse proxy maintained.

Troubleshooting

If the page does not load through the SSH tunnel, confirm that the tunnel command is still running, that the Compose service is up, and that the published mapping is 127.0.0.1:3000:8080. Check docker compose ps and the Open WebUI logs. If you use a reverse proxy, verify its upstream is the VPS’s local port 3000 and that the DNS record points to the right server.

If Phi-3 is missing from the model selector, check the exact downloaded name with docker compose exec ollama ollama list. If no model appears, rerun the pull command and inspect Ollama’s logs. If downloads fail, check the VPS’s outbound internet access and available disk space.

If responses are very slow or a container restarts, inspect docker compose logs, memory, and disk usage. A model may exceed the VPS’s available RAM. Try a smaller supported variant, reduce simultaneous use, or move to a VPS with more memory. After changing Compose settings, apply them with docker compose up -d.

Frequently Asked Questions

Can I run Phi-3 on a CPU-only VPS?
Yes, if the selected model fits in memory, but generation may be slower than on a suitable GPU. Performance depends on the model variant and VPS hardware.

Does Ollama require Open WebUI?
No. Ollama can run models on its own. Open WebUI provides a convenient browser-based chat interface.

Will my model remain after restarting the VPS?
Yes, provided the Compose volume is retained and Docker starts the services again. The configuration uses restart: unless-stopped.

Can I use a different Phi-3 variant?
Yes. Pull a variant available in Ollama’s model library and select its exact name in Open WebUI. Check its resource needs first.

Is it safe to expose Open WebUI publicly?
Do not expose it without HTTPS, strong unique credentials, firewall restrictions, and regular maintenance. Keep Ollama’s API port private; a VPN or SSH tunnel is safer for personal use.

How do I stop the services?
Run docker compose down from the project directory. This removes the containers but preserves the named model and WebUI data volumes.

Related

• Best Farming Games: Top Picks for Every Kind of Player
• Best AI Game Development Tools for Indie Developers
• AI-Generated Code: Does It Work, and Do You Understand Why?