How-To Install Istio Service Mesh and Istio Ingress Gateway on Crusoe Managed Kubernetes

Introduction

Istio is a popular service mesh for Kubernetes which enables features such as L7 ingress routing, pod-to-pod traffic encryption (mTLS), zero-trust authentication and observability at the platform level, rather than the application level. In other words, you no longer need to build TLS into your application pods, or inspect HTTP headers of requests to decide how to route them, or export request metrics to Prometheus. Istio transparently implements all these features for you. In this example, we'll install Istio, point a wildcard subdomain at a CMK load balancer and use it to reach 2 independent back-end services in a CMK cluster using Istio Gateway and Virtual Service objects.

Prerequisites

• A local bash/zsh terminal with kubectl access to a healthy CMK cluster (run kubectl config current-context to be sure that you are working with your intended cluster)
• Helm (for installing the CMK Load Balancer chart)
• (optional) Access to the Crusoe Cloud Load Balancers feature (follow steps 1 and 2 of this article).

Limitations

Each Istio version is tested with numerous recent Kubernetes versions, so we don't anticipate any compatibility problems with currently-supported CMK versions. However, we recommend checking the official Istio support matrix to be certain. Important: Don't enable Istio sidecar injection on namespaces in which you will run pods that perform collective GPU communications between each other (for example, a namespace where you will create a multi-gpu Ray cluster). We have found that Istio sidecars can block NCCL communications between pods.

Step-by-step instructions

Step 1: Download istioctl

Download istioctl and add its bin folder to your path.

curl -L https://istio.io/downloadIstio | sh -
cd istio-1.27.1 # or whichever version you downloaded
export PATH=$PWD/bin:$PATH
istioctl version

#expected output
client version: 1.27.1

Step 2: Install Istio on your cluster

Install istio with the default profile (control plane and Ingress gateway only) - or select another profile as described in the official Istio docs.

istioctl install --set profile=default -y
watch kubectl get po -n istio-system

When the istiod and istio-ingressgateway pods have come to a 'ready' state, cancel the watch command.

Step 3: Enable sidecar injection

Label a namespace to enable automatic injection of Istio sidecars into each of that namespace's pods.

kubectl label namespace default istio-injection=enabled

Note: You can also configure Istio to automatically label all newly-created namespaces for automatic sidecar injection. See the Istio documentation for further details. As mentioned above, don't enable Istio sidecar injection on pods that will use NCCL.

Step 4: Create test services 

Create a couple of example back-end services for testing Istio Ingress Gateway functionality. Skip this step if you already have a service that you want to test with.

kubectl create deployment echo-1 --image=hashicorp/http-echo:latest \
  -- /http-echo -text="You have reached service 1"
kubectl create deployment echo-2 --image=hashicorp/http-echo:latest \
  -- /http-echo -text="You have reached service 2"
  
kubectl expose deployment echo-1 --port=5678
kubectl expose deployment echo-2 --port=5678

Note that the resulting pods each have 2 containers. The additional container is the istio-proxy sidecar. The sidecar communicates with the Istio control plane ('istiod'), enforces mTLS and service authn/authz, and exposes a metrics endpoint for Prometheus to scrape.

Optionally, run a curl pod within the cluster to test one of the services:

kubectl run test-curl-pod --image=curlimages/curl -it --rm \
 --restart=Never -- curl echo-1:5678
#expected output
You have reached service 1

Step 5: Configure DNS and firewall rules for your CMK Load Balancer

kubectl -n istio-system get svc istio-ingressgateway
NAME                   TYPE           CLUSTER-IP     EXTERNAL-IP      PORT(S)                                                                      AGE
istio-ingressgateway   LoadBalancer   10.233.***.***   <External IP>   15021:31366/TCP,80:31004/TCP,443:31039/TCP,31400:31193/TCP,15443:31661/TCP   26h

The EXTERNAL-IP address is the public IP address of the CMK Load Balancer that was assigned to handle traffic for your Istio ingress service. If this field shows as 'pending', revisit the steps in the Prerequisites section to check and fix your Load Balancer installation. If you have a DNS domain that you want to use for external access to your service, configure an A record in your DNS provider to point subdomains (for example, service-1.example.com and service-2.example.com) at the EXTERNAL-IP address. If your DNS provider allows it, add just a single wildcard (e.g *.example.com). As a quick alternative for testing purposes only, edit your local /etc/hosts file to map test hostnames to the load balancer's public IP address.

Configure a firewall rule in your Crusoe Cloud project to allow access to the NodePort corresponding to the target port 443 in the istio-ingressgateway service. In the example above, we see 443:31039/TCP, so we need to add a firewall rule allowing TCP access to port 31039. Confirm the exact NodePort used in your installation and add the rule as described in Step 5 of the Load Balancer instructions.

Step 6: Create a TLS secret containing a certificate for your subdomain and its private key

Obtain a valid TLS certificate and key for the domain name (or wildcard) that you used in step 5. With Let's Encrypt/Certbot, obtain the 'fullchain.pem' and 'privkey.pem' output files and save them locally. For testing purposes, create a self-signed certificate and key:

#create the self signed cert and private key

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout key.pem -out cert.pem \
  -subj "/CN=service-1.example.com" \
      -addext "subjectAltName = DNS:service-1.example.com,DNS:service-2.mycluster.example.com"
      
#create a Kubernetes secret containing the key and cert (replace key.pem and cert.pem with
#the real names of your private key and cert files e.g privkey.pem and fullchain.pem

kubectl -n istio-system create secret tls crusoe-tls-secret --key=key.pem --cert=cert.pem

For production, use cert-manager to manage and update the TLS certificate as described here. This means that you don't need to remember to manually update your certificate every 3 months.

Step 7: Create a gateway and a virtual service.

Edit the yaml to match the hostnames, service names, port numbers and secret name that you want to test with, if different to the examples above.

cat <<EOF|kubectl apply -f -
apiVersion: networking.istio.io/v1
kind: Gateway
metadata:
  name: crusoe-gateway
  namespace: istio-system
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - '*'
    port:
      name: https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: crusoe-tls-secret
      mode: SIMPLE
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: echo-1
  namespace: default
spec:
  gateways:
  - istio-system/crusoe-gateway
  hosts:
  - service-1.example.com
  http:
  - route:
    - destination:
        host: echo-1
        port:
          number: 5678
---
apiVersion: networking.istio.io/v1
kind: VirtualService
metadata:
  name: echo-2
  namespace: default
spec:
  gateways:
  - istio-system/crusoe-gateway
  hosts:
  - service-2.example.com
  http:
  - route:
    - destination:
        host: echo-2
        port:
          number: 5678

Step 8: Test 

Test that you can reach the services in your browser and that the content matches the expected deployment's output

Using Istio Ingress Gateway as a Kubernetes IngressClass

Optionally, Istio's Ingress Gateway can be added as an IngressClass controller, as an alternative to Nginx ingress. When you do this, you can then assign 'istio' as the spec.ingressClassName of a standard Kubernetes Ingress object. Istiod recognizes these ingresses and creates corresponding gateway and virtualservice configurations inside the Istio IngressGateway itself (gateways and virtualservices created in this manner are not visible to the Kubernetes API server, unlike those described in step 7 above). You need to configure DNS and TLS secrets as shown in steps 5 and 6 - the DNS name and subject name in the TLS secret need to match the 'host' property of your Ingress object. Requests arriving at the Istio IngressGateway that match an ingress host and backend route configuration will then be forwarded to the service defined in that configuration.
 

To configure Istio as an IngressClass, run the following command:

cat <<EOF | kubectl apply -f -
apiVersion: networking.k8s.io/v1
kind: IngressClass
metadata:
  name: istio
spec:
  controller: istio.io/ingress-controller
EOF

Ingresses that use 'istio' as their ingressClass will then display the public IP address details of the Istio Ingressgateway:

kubectl get ingress -A
NAMESPACE   NAME         CLASS   HOSTS                   ADDRESS          PORTS
flyte       flyte-core   istio   flyte.crusoe.internal   216.86.162.221   80, 443

Conclusion

By now, you should have successfully installed Istio and exposed 2 Kubernetes deployments to the outside world. You can continue to add an unlimited number of back-end services to your gateway in this manner; or, you can create Ingresses for those services and set 'istio' as the ingress class. You can also experiment with the various other Istio add-ons available. Integrating Istio with Prometheus is a great way to monitor your applications at Layer 7 in terms of request latency, HTTP response codes etc. You can do this with your existing Prometheus stack, or install Prometheus and Grafana as Istio add-ons. Full instructions are given in the Istio documentation.