How-To Expose Ollama (LLaMA 4) Service on Crusoe Cloud

This article explains how to set up and expose the Ollama (LLaMA 4) inference service running on a Crusoe Cloud virtual machine. By following these instructions, users will be able to make the Ollama API accessible externally over the internet on port 11434, enabling integration with external applications while securing the service using firewall rules.

Prerequisites

• Crusoe Cloud virtual machine instance. The steps below were performed on a100.1x pcie running Ubuntu 22.04.
• Ollama installed on the VM (instructions included).
• Access to the Crusoe Cloud Firewall configuration console.
• Public IP address assigned to the VM.

Step-by-Step Instructions

Step 1: Install Ollama on the VM

1. Connect to your VM via SSH.

2. Create a directory for Ollama and change into it:

   mkdir -p ~/ollama && cd ~/ollama

3. Download and run the Ollama install script:

   curl -fsSL https://ollama.com/install.sh | sh

4. Verify Ollama is installed:

   ollama --version

5. Check if Ollama is running

   ubuntu@ollama-test:~/ollama$ sudo systemctl status ollama
   ● ollama.service - Ollama Service
        Loaded: loaded (/etc/systemd/system/ollama.service; enabled; vendor preset: enabled)
        Active: active (running) since Tue 2025-07-29 14:54:20 UTC; 1min 3s ago
      Main PID: 43773 (ollama)
         Tasks: 10 (limit: 144845)
        Memory: 17.1M
           CPU: 1.388s
        CGroup: /system.slice/ollama.service
                └─43773 /usr/local/bin/ollama serve
   
   Jul 29 14:54:20 ollama-test systemd[1]: Started Ollama Service.
   Jul 29 14:54:20 ollama-test ollama[43773]: Couldn't find '/usr/share/ollama/.ollama/id_ed25519'. Generating new private key.
   Jul 29 14:54:20 ollama-test ollama[43773]: Your new public key is:
   Jul 29 14:54:20 ollama-test ollama[43773]: ssh-ed25519 AAAAC3NzaxxxxxNKeKI
   Jul 29 14:54:20 ollama-test ollama[43773]: time=2025-07-29T14:54:20.060Z level=INFO source=routes.go:1235 msg="server config" env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_CONTEXT_LENGTH:4096 OLLAMA_DEBUG:INFO OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA>
   Jul 29 14:54:20 ollama-test ollama[43773]: time=2025-07-29T14:54:20.060Z level=INFO source=images.go:476 msg="total blobs: 0"
   Jul 29 14:54:20 ollama-test ollama[43773]: time=2025-07-29T14:54:20.060Z level=INFO source=images.go:483 msg="total unused blobs removed: 0"
   Jul 29 14:54:20 ollama-test ollama[43773]: time=2025-07-29T14:54:20.060Z level=INFO source=routes.go:1288 msg="Listening on 127.0.0.1:11434 (version 0.9.6)"
   Jul 29 14:54:20 ollama-test ollama[43773]: time=2025-07-29T14:54:20.061Z level=INFO source=gpu.go:217 msg="looking for compatible GPUs"
   Jul 29 14:54:21 ollama-test ollama[43773]: time=2025-07-29T14:54:21.439Z level=INFO source=types.go:130 msg="inference compute" id=GPU-7e17ebbb-d9fa-f548-aeee-2583bda9ad4f library=cuda variant=v12 compute=8.0 driver=12.9 name="NVIDIA A100 80GB PCIe" total="79.3 GiB" available="78.8 GiB"

6. Check Ollama is running on port 11434

   ubuntu@ollama-test:~/ollama$ sudo lsof -i :11434
   COMMAND   PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
   ollama  43773 ollama    3u  IPv4  59090      0t0  TCP ollama-test.us-east1-a.compute.internal:11434 (LISTEN)

7. Test locally:

   ubuntu@ollama-test:~/ollama$ curl http://127.0.0.1:11434/
   Ollama is running

Step 2: Configure Ollama Systemd Service to Listen Publicly

By default, Ollama listens only on localhost. You need to configure it to listen on all interfaces (0.0.0.0) to allow external connections.

1. Open the Ollama systemd service file:

   sudo vi /etc/systemd/system/ollama.service

2. Add the following line under the [Service] section:

   Environment="OLLAMA_HOST=0.0.0.0:11434"

   The [Service] section should look like this:

   [Unit]
   Description=Ollama Service
   After=network-online.target
   
   [Service]
   ExecStart=/usr/local/bin/ollama serve
   User=ollama
   Group=ollama
   Restart=always
   RestartSec=3
   Environment="PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin"
   Environment="OLLAMA_HOST=0.0.0.0:11434".   <---- I have added this line to expose it to public
   [Install]
   WantedBy=default.target

3. Save and exit the editor 

4. Reload systemd and restart Ollama service to apply changes:

   sudo systemctl daemon-reload
   sudo systemctl restart ollama

5. Confirm Ollama is listening on all interfaces:

   ubuntu@ollama-test:~/ollama$ sudo ss -tulpn | grep 11434
   tcp   LISTEN 0      4096                   *:11434            *:*    users:(("ollama",pid=44909,fd=3)) 

Step 3: Configure Crusoe Cloud Firewall to Allow External Traffic

1. Log in to Crusoe Cloud Console and navigate to the Firewall Rules section.

2. Create or update an ingress firewall rule with: (ollama-test in the screenshot below is the VM_name)

   
   

Step 4: Test External Access to Ollama API

From your local machine or any external system:

curl http://<YOUR_VM_PUBLIC_IP>:11434/

Expected output:

Ollama is running

Common Issues and Resolutions

1. Ollama only listens on 127.0.0.1: Make sure Environment="OLLAMA_HOST=0.0.0.0:11434" is set in ollama.service systemd service and restart it.
2. Confirm Crusoe Cloud firewall allows TCP port 11434 ingress. Check VM subnet and target correctly selected.
3. If UFW enabled, run sudo ufw allow 11434/tcp and reload.
4. Verify VM’s public IP via curl ifconfig.me or Crusoe Console. Confirm VM is in running state.

References

1. https://ollama.readthedocs.io/en/quickstart/
2. https://ollama.readthedocs.io/en/api/
3. https://ollama.com/