Introduction

SSH connection failures with errors like "Connection reset by peer" or "kex_exchange_identification: read: Connection reset by peer" can often be caused by SSH MaxStartups throttling. This security feature in the SSH daemon limits concurrent unauthenticated connections to prevent resource exhaustion from brute-force attacks or excessive connection attempts. This guide will help you identify, troubleshoot, and resolve SSH connectivity issues related to MaxStartups throttling.

Prerequisites

• Access to the affected VM via serial console or SSH into the VM.
• Root or sudo privileges on the affected system
• Basic understanding of SSH configuration
• Access to system logs

Step-by-Step Instructions

Step 1: Verify SSH Service Status

• Connect to the VM via serial console or SSH into the VM
• Check if SSH service is running:

  sudo systemctl status sshd
  

• If the service is inactive, start it:

  sudo systemctl start sshd
  

• Note: If SSH service is running but connections are still failing, proceed to the next step

Step 2: Collect and Analyze SSH Logs

• Examine SSH service logs for MaxStartups throttling messages:

  sudo journalctl -u ssh.service | grep -i maxstartups
  

• Check system logs for SSH-related errors:

  sudo grep -i "maxstartups\|drop connection" /var/log/syslog
  

• Review authentication logs for failed login attempts:

  sudo grep -i ssh /var/log/auth.log | tail -50
  

• Look for kernel messages related to SSH:

  dmesg | grep -i ssh
  

• Key indicators of MaxStartups throttling:
  • "beginning MaxStartups throttling" messages
  • "drop connection #X from [IP]:port" messages
  • Multiple failed login attempts from various IP addresses

Step 3: Identify Brute-Force Attack Patterns

• Analyze failed login attempts to identify potential attacks:

  sudo grep "Failed password\|Invalid user" /var/log/auth.log | tail -20
  

• Check for repeated connection attempts from specific IPs:

  sudo grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -nr
  

• Common attack patterns:
  • Multiple attempts with invalid usernames
  • High frequency of connection attempts from single IP addresses
  • Attempts targeting common service accounts

Step 4: Check Current SSH Configuration

• View current MaxStartups settings:

  sudo grep -i maxstartups /etc/ssh/sshd_config
  

• If no MaxStartups line exists, SSH uses default values (typically 10:30:100)
• Default MaxStartups format: start:rate:full
  • start: Number of unauthenticated connections before throttling begins
  • rate: Percentage of connections to drop once throttling starts
  • full: Maximum number of unauthenticated connections allowed

Step 5: Adjust MaxStartups Configuration

• Edit the SSH configuration file:

  sudo nano /etc/ssh/sshd_config
  

• Add or modify the MaxStartups line:

  MaxStartups 20:50:100
  

• Recommended values:
  • For light usage: MaxStartups 10:30:60
  • For moderate usage: MaxStartups 20:50:100
  • For heavy usage: MaxStartups 50:30:200
• Save the file and restart SSH service:

  sudo systemctl restart sshd
  

Step 6: Implement Security Hardening

• Restrict SSH access to trusted IP addresses by configuring firewall rules
• Consider changing the default SSH port:

  Port 2222
  

• Disable root login if not required:

  PermitRootLogin no
  

• Enable key-based authentication and disable password authentication:

  PasswordAuthentication noPubkeyAuthentication yes
  

• Important: Test configuration changes before disconnecting from current session

Step 7: Monitor and Verify Resolution

• Test SSH connectivity from your client machine:

  ssh user@vm-ip-address
  

• Monitor logs for continued MaxStartups messages:

  sudo tail -f /var/log/syslog | grep -i maxstartups
  

• Check for successful connections in auth.log:

  sudo tail -f /var/log/auth.log | grep "Accepted"
  

Example

A user reported inability to SSH into VMs ib-vm-21 through ib-vm-27 with the error:

kex_exchange_identification: read: Connection reset by peer
Connection reset by 216.86.170.69 port 22

Log Analysis Results:

May 23 05:59:49 ib-vm-27 sshd[12345]: beginning MaxStartups throttling
May 23 05:59:50 ib-vm-27 sshd[12346]: drop connection #12 from [195.178.110.232]:56954
May 23 05:59:51 ib-vm-27 sshd[12347]: Invalid user admin from 185.156.73.233
May 23 05:59:52 ib-vm-27 sshd[12348]: Failed password for invalid user admin from 185.156.73.233

Resolution Applied:

1. Identified MaxStartups throttling in logs
2. Configured MaxStartups 20:50:100 in /etc/ssh/sshd_config
3. Restarted SSH service
4. Recommended implementing firewall rules in Crusoe Cloud to only allow Trusted IPs
5. SSH connectivity restored successfully

Troubleshooting Common Issues

Issue: SSH service restarts but connections still fail

Resolution:

• Verify firewall rules are not blocking SSH traffic
• Check if fail2ban or similar security tools are blocking your IP
• Ensure MaxStartups changes were saved and applied

Issue: Unable to access VM via serial console

Resolution:

• Use VM reset command if available: crusoe compute vms reset <vm-name>
• Contact support for assistance with serial console access
• Consider using alternative management interfaces

Issue: MaxStartups throttling continues after configuration change

Resolution:

• Verify SSH configuration syntax with: sudo sshd -t
• Check if multiple SSH daemons are running
• Increase MaxStartups values further if legitimate traffic is high
• Implement rate limiting at firewall level

Additional Resources

• SSH MaxStartups Configuration Reference
• Stack Overflow: MaxStartups Explanation
• How-To Troubleshoot SSH Issues with a VM
• Linux SSH Security Best Practices
• Fail2ban Configuration for SSH Protection