Introduction
By default, VMs are created with a placement policy of unspecified, which places no constraint on which physical host each VM lands on. For instance types smaller than a full host, two or more of your VMs can end up on the same host.
If those VMs are replicas of something critical, such as Slurm controllers or the control plane of a Kubernetes cluster you run yourself on Crusoe VMs (RKE2, kubeadm), co-location silently defeats the redundancy: a single host reboot or hardware failure takes down every replica running on that host.
The spread placement policy prevents this. When you set placement_policy to spread on an Instance Template, every VM created together from that template is scheduled onto a different physical host. The scheduler enforces this at placement time and skips any host that already holds a VM from the group, including VMs that are still starting up. This holds even when the VMs are created concurrently in a single bulk request.
Spread applies to VMs created together from the same template, in one creation run. It is not a general anti-affinity rule: it cannot be attached to arbitrary existing VMs, and VMs created in a later run from the same template are not guaranteed to avoid the hosts used by an earlier run.
The policy matters for instance types smaller than a full host, such as the c1a and s1a CPU types. Full-host GPU instance types such as h100-80gb-sxm-ib.8x already run one VM per host, so spreading is inherent there and the setting is ignored.
Prerequisites
- Crusoe Cloud Account with an Active Project
- Crusoe CLI Installed and Configured (CLI Steps Only)
- Terraform with the crusoecloud/crusoe Provider (Terraform Steps Only)
- An Instance Type Smaller Than a Full Host Such as c1a or s1a (Spread Has No Effect on Full-Host GPU Types)
- Crusoe Managed Kubernetes Cluster and Crusoe Cloud API Access (CMK Node Pool Steps Only)
Instructions
Step 1: Create an Instance Template with the Spread Placement Policy
The placement policy is a property of the Instance Template, not of individual VMs. Using Terraform:
resource "crusoe_instance_template" "ha_controllers" {
name = "ha-controller-template"
type = "c1a.8x"
image = "ubuntu22.04:latest"
location = "us-east1-a"
subnet = crusoe_vpc_subnet.example.id
ssh_key = file("~/.ssh/id_ed25519.pub")
placement_policy = "spread"
}The field accepts spread or unspecified (the default). The same placement_policy field is available when creating an Instance Template through the Crusoe Cloud API.
From the CLI, pass --placement-policy spread to crusoe compute templates create:
crusoe compute templates create \ --name <YOUR_TEMPLATE_NAME> \ --type <VM_TYPE> \ --location <YOUR_LOCATION> \ --vpc-subnet-id <YOUR_SUBNET_ID> \ --image <IMAGE_NAME> \ --keyfile <PATH_TO_YOUR_PUBLIC_SSH_KEY> \ --placement-policy spread
List available VM types with crusoe compute vms types, locations with crusoe locations list, and images with crusoe compute images list.
ℹ️ Note: A location-bound template requires
--vpc-subnet-id. Omit both--locationand--vpc-subnet-idto create a global template instead.
You can also set this in the Console: go to Compute > Instance Templates > Create Template, and on the Template Details step, set the Placement Policy dropdown to Spread. The dropdown defaults to Unspecified.
ℹ️ Note: If you set
spreadon a template that uses a full-host GPU instance type, the policy is ignored rather than rejected. Those VMs already run one per host, so there is nothing to spread.
Step 2: Create Your VMs Together from the Template
Spread placement applies to VMs created as a group from the template. Use bulk-create so all VMs are created in one atomic operation:
crusoe compute vms bulk-create \ --name-prefix controller \ --template-id <YOUR_TEMPLATE_ID> \ --count 3 \ --location us-east1-a
This creates controller-1, controller-2, and controller-3, each on a different physical host. The operation is atomic. Either all VMs are created and spread, or none are.
ℹ️ Note: The host-separation guarantee applies within one
bulk-createrun. If you need more spread VMs later, a second run from the same template spreads its own VMs across distinct hosts, but they are not guaranteed to avoid the hosts used by the first run. Plan your group size up front where full separation matters.If you manage the template in Terraform, still create the group from it with one
bulk-createrequest (CLI or API) rather than with separate per-VM resources.
⚠️ Warning: If the location does not have enough distinct hosts with free capacity for your instance type, the entire bulk create fails with a capacity error. No partial group is left behind. Reduce the count or contact support about capacity in your region.
Step 3: Verify Your Placement
A successful bulk create is itself the confirmation. With placement_policy: spread, the request only succeeds if every VM in the group landed on a distinct host.
Crusoe does not expose physical host identifiers to customers. If you need explicit confirmation, open a support ticket with the VM IDs and the team can confirm whether they are on distinct hosts.
ℹ️ Note: Spread cannot be applied retroactively. If you have existing VMs that were created individually and need them separated onto different hosts, open a support ticket. This requires a support-assisted migration and typically a stop and start of the affected VMs.
Step 4: Set the Placement Policy on a CMK Node Pool (Optional)
For Crusoe Managed Kubernetes, the same policy is available on node pool creation through the Crusoe Cloud REST API. Include placement_policy in the POST request body when creating the node pool:
{
"name": "system-pool",
"cluster_id": "<YOUR_CLUSTER_ID>",
"product_name": "c1a.8x",
"count": 3,
"ssh_public_key": "<YOUR_SSH_PUBLIC_KEY>",
"placement_policy": "spread"
}This field is currently API-only. It is not yet available in the crusoe kubernetes nodepools create CLI command, the Terraform crusoe_kubernetes_node_pool resource, or the Console's Create Node Pool form.
💡 Tip: This is most useful for CPU node pools that run cluster-critical components such as ingress controllers, CoreDNS, or monitoring, where you do not want all replicas scheduled onto nodes that share one host.
Example
A team runs a self-managed Slurm cluster on Crusoe with two controller VMs and three login nodes, all on c1a instances. The VMs were created individually, and Crusoe Support confirmed that the two controllers shared a physical host. A single host reboot could take down both controllers at once, which is the situation the replicas were supposed to prevent.
To fix this, the team defined one Instance Template with placement_policy = "spread" and recreated the controllers with bulk-create --count 2, then created the login nodes from a second spread template with --count 3. Each group now spans distinct hosts. Any single host event costs at most one controller and one login node.
Related Articles
- How To Deploy Multiple Crusoe VMs Using Instance Templates and Terraform
- FAQ: How Node Pool Configuration Changes Affect Existing Nodes
- FAQ: How Do I Identify the Physical Hardware Behind My GPU VM?