How-To Run DeepSeek-R1: 70b on a Crusoe Cloud VM using Ollama

Introduction

This article provides a step-by-step guide on how to run the DeepSeek-R1: 70B large language model locally using Ollama. DeepSeek-R1: 70B is a high-parameter reasoning model designed for advanced tasks such as complex problem solving, code generation, and long-form reasoning.

Ollama simplifies the process of downloading, configuring, and running large language models by handling model management and exposing a local inference interface. By using Ollama, you can run DeepSeek-R1: 70B on your own infrastructure, enabling greater control over data privacy, performance, and customization without relying on external APIs.

This guide walks through the required prerequisites, model setup, and execution steps to help you get DeepSeek-R1: 70B up and running efficiently in a local or self-hosted environment.

Prerequisites 

• Crusoe Cloud account
• Access to the Crusoe Cloud UI or Crusoe CLI
• A linux system with sufficient resources. It's recommended to use a system with greater than 40 GB of VRAM (H100, H200, B200, etc.)

Step-by-Step Instructions

1. To get started, connect to your compute instance using SSH and ensure Docker is installed and running on the system before proceeding. 

2. From your terminal, launch the Ollama container with GPU support enabled:

   sudo docker run -d \
     --gpus=all \
     -v ollama:/root/.ollama \
     -p 11434:11434 \
     --name ollama \
     ollama/ollama

   1. Docker will pull the Ollama image if it is not already present.

   2. The image download is complete when Docker reports that a newer image has been successfully retrieved.

      Status: Downloaded newer image for ollama/ollama:latest

3. Check the status to verify the Ollama Container Is Running by running sudo docker ps. You should see something like the following:

   CONTAINER ID   IMAGE           COMMAND               CREATED          STATUS          PORTS                                           NAMES
   8d815762dd9c   ollama/ollama   "/bin/ollama serve"   19 seconds ago   Up 13 seconds   0.0.0.0:11434->11434/tcp, :::11434->11434/tcp   ollama

   1. Note: The Container ID will appear different for you.
4. You are now ready to download and run the DeepSeek-R1 70B model.

5. Execute the following command to pull and start the DeepSeek-R1: 70B model inside the running Ollama container:

   sudo docker exec -it ollama ollama run deepseek-r1:70b

   1. Note: The model download is large (over 40 GB) and may take up to 15 minutes, depending on network speed.

   2. Once the download and initialization are complete, Ollama will present an interactive prompt indicating the model is ready to accept input.

      success
      >>> Send a message (/? for help)

6. You are now ready to test the model! 

7. At the prompt, enter a test query such as:

   >>> What is Infiniband?

8. The model will generate a response to the submitted prompt. Internally, the model performs reasoning to produce the answer, though only the final response is returned to the user.

   InfiniBand is a high-speed interconnect technology used primarily in data centers, clusters, and high-performance computing (HPC) environments. It is designed to provide low latency, high bandwidth, and
   scalable communication between servers, storage systems, and other network devices. InfiniBand is often used in applications where fast data transfer and minimal delays are critical, such as scientific
   simulations, financial modeling, and large-scale data processing.
   
   Key features of InfiniBand include:
   
   1. **High Bandwidth**: InfiniBand supports multiple data rates, with the latest versions offering up to 200 Gbps per port (in dual-port configurations), enabling extremely fast data transfer rates.
   
   2. **Low Latency**: The technology is optimized for minimal latency, making it suitable for real-time and time-sensitive applications.
   
   3. **Scalability**: InfiniBand supports a large number of devices in a single fabric, making it scalable for large-scale computing environments.
   
   4. **Reliability**: It includes advanced reliability features such as error detection and correction mechanisms to ensure data integrity.
   
   5. **Quality of Service (QoS)**: InfiniBand allows for prioritization of traffic, ensuring that critical applications receive the necessary bandwidth and low latency.
   
   InfiniBand is commonly used in:
   
   - **High-Performance Computing (HPC)**: For connecting supercomputers and clusters.
   - **Cloud Data Centers**: To provide high-speed connectivity between servers and storage.
   - **Storage Area Networks (SANs)**: For fast access to shared storage resources.
   - **Real-Time Analytics**: Where rapid data processing is essential.
   
   InfiniBand competes with other interconnect technologies like Ethernet, Fibre Channel, and Intel's Omni-Path. Its adoption is particularly strong in environments where performance and scalability are
   paramount.

9. Congrats! If you see this output, it confirms that the DeepSeek-R1: 70B model is running successfully and responding to prompts as expected.

Additional Resources:

• Docker Installation
• Ollama Documentation – Installation, configuration, and usage
• DeepSeek-R1 Model Overview
• Crusoe Instance Hardware and Performance Overview