How-To Create /etc/hosts file Using Crusoe CLI

Last Updated: Dec 09, 2025

Introduction

For large clusters of compute, it may be necessary to bypass internal DNS altogether for node to node network communications. This can help boost performance and reliability of long running training jobs. This guide walks through using the Crusoe CLI to generate a json file of your compute nodes, and a python script to generate /etc/hosts  to distribute across the instances in your VPC. 

Step-by-Step Instructions

1. Use Crusoe CLI to generate JSON metdata

   • Use the Crusoe CLI to generate a json file of all the Instances in your project.

     #crusoe compute vms list -f json  > vms.json

   • Run Python Script to generate /etc/hosts file 
   • Copy hosts file to all instances
2. Run Python Script to generate /etc/hosts file 

   • Save the following script to a file called etc.py 

     import json
     
     def generate_etc_hosts(json_data):
         """
         Generate /etc/hosts format file from JSON data
     
         Args:
             json_data (str): JSON string containing machine information
         """
         data = json.loads(json_data)
     
         # Start with standard localhost entries
         hosts_content = [
             "127.0.0.1\tlocalhost",
             "::1\tlocalhost ip6-localhost ip6-loopback",
             "fe00::0\tip6-localnet",
             "ff00::0\tip6-mcastprefix",
             "ff02::1\tip6-allnodes",
             "ff02::2\tip6-allrouters",
             "\n# Cluster nodes"
         ]
     
         # Add entries for each machine
         for machine in data:
             for interface in machine['network_interfaces']:
                 for ip in interface['ips']:
                     if 'private_ipv4' in ip:
                         private_ip = ip['private_ipv4']['address']
                         name = machine['name']
                         hosts_content.append(f"{private_ip}\t{name}")
     
         # Write to /etc/hosts format file
         with open('etc-hosts', 'w') as f:
             f.write('\n'.join(hosts_content))
             f.write('\n')  # Add final newline
     
     if __name__ == "__main__":
         # Read JSON from file
         with open('vms.json', 'r') as f:
             json_data = f.read()
     
         # Generate the hosts file
         generate_etc_hosts(json_data)
     
         # Print confirmation and show contents
         print("Generated etc-hosts file. Contents:")
         print("-" * 40)
         with open('etc-hosts', 'r') as f:
             print(f.read())        

   • Run the script in the same directory as your vms.json output from the previous step. This should return something like the following:

     > python3 etc.py
     
     -------------------------------------------------------------------------
     127.0.0.1	localhost
     ::1	localhost ip6-localhost ip6-loopback
     fe00::0	ip6-localnet
     ff00::0	ip6-mcastprefix
     ff02::1	ip6-allnodes
     ff02::2	ip6-allrouters
     
     # Cluster nodes
     
     172.27.60.152	slurm-compute-node-0
     172.27.52.2	    slurm-head-node-0
     172.27.62.2  	slurm-login-node-0
     172.27.50.79	slurm-nfs-node-0
     

3. Copy hosts file to all instances
   • Copy the /etc/hosts output to all instances in your VPC. You can leverage shell tools like pssh or clush for easy administration.
   • Once copied, each Instance should bypass DNS to resolve internal IPs of other Instances in your VPC directly

Additional Resources 

PSSH Documentation 

CLUSH Documentation