How-To Secure Crusoe Managed Kubernetes (CMK) Cluster with OpenID Connect and RBAC

Last Updated: Oct 27, 2025

Introduction

Integrating OpenID Connect (OIDC) with Kubernetes RBAC provides a secure and centralized way to granularly manage user authentication and authorization within your Kubernetes environment. This article walks you through the process of configuring a Crusoe Managed Kubernetes (CMK) cluster to integrate with an OIDC provider e.g. Okta.

Note: Currently only Okta is supported as an identity provider. 

Prerequisites

• Access to a Crusoe Cloud project with appropriate permissions
• Access to create a Crusoe Managed Kubernetes (CMK) cluster
• Pre-configured OIDC connection in Okta with access to OIDC details

Step-by-Step Instructions

1. Create the CMK cluster with OIDC
   First, create a new CMK cluster, specifying your OIDC provider's details during creation. OIDC can only be enabled when you first create the cluster.

   • --oidc-provider: This flag takes three comma-separated values:
     • client-id: Your OIDC application's Client ID.
     • issuer-url: The URL of your OIDC provider.
     • groups-claim: The name of the claim in your OIDC token that contains group membership (e.g., groups).

   $ crusoe kubernetes clusters create 
     --name new-oidc-test 
     --project-id <project_id>
     --location eu-iceland1-a # see `crusoe locations list` to list all locations
     --cluster-version 1.31.7-cmk.7 
     --oidc-provider client-id=<client_id>,issuer-url=<issuer_url>,groups-claim=groups

   Note: OIDC support is only available on specific minimum CMK versions:

   • 1.31.7-cmk.7
   • 1.32.7-cmk.3
   • 1.33.4-cmk.2

2. Get Admin credentials
   To configure permissions inside the new cluster, you first need to get administrative credentials. Use the admin_cert authentication type for this. This command updates your local kubeconfig with administrative access to the new-oidc-test cluster.

   $ crusoe kubernetes clusters get-credentials new-oidc-test --auth-type=admin_cert

3. Create and apply RBAC roles
   Next, define the Kubernetes RBAC roles and bindings that grant permissions to your OIDC groups. In this example, we'll create a ClusterRole named k8s-oidc-limited that only grants read access (get, list, watch) to nodes and pods.

   We then bind this role to our OIDC group (named k8s-CMK-OIDC in this example) using a ClusterRoleBinding.

   Create a file named rbac.yaml:

   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRole
   metadata:
     name: k8s-oidc-limited
   rules:
   - apiGroups: [""]
     resources: ["nodes", "pods"]
     verbs: ["get", "list", "watch"]
   
   ---
   apiVersion: rbac.authorization.k8s.io/v1
   kind: ClusterRoleBinding
   metadata:
     name: k8s-oidc-limited-binding
   subjects:
   - kind: Group
     name: k8s-CMK-OIDC    # This MUST match the group name from your OIDC provider
     apiGroup: rbac.authorization.k8s.io
   roleRef:
     kind: ClusterRole
     name: k8s-oidc-limited
     apiGroup: rbac.authorization.k8s.io

   Apply the RBAC configuration to your cluster:

   $ kubectl apply -f rbac.yaml
   clusterrole.rbac.authorization.k8s.io/k8s-oidc-limited created
   clusterrolebinding.rbac.authorization.k8s.io/k8s-oidc-limited-binding created

4. Get OIDC user credentials
   Now that the cluster is configured for OIDC and the RBAC policies are in place, retrieve the kubeconfig for end-users. This time, specify oidc as the authentication type. The following command configures the kubeconfig to use the OIDC authentication flow.

   $ crusoe kubernetes clusters get-credentials new-oidc-test --auth-type=oidc

5. Verify access as an OIDC user
   Test the configuration by running kubectl commands as a user who has authenticated via OIDC and is a member of the k8s-CMK-OIDC group.

   You may need to explicitly point to the correct kubeconfig file if it's not your default.

   $ kubectl get nodes --kubeconfig=~/.kube/config
   NAME                                           STATUS   ROLES    AGE   VERSION
   np-f3fae51e-1.eu-iceland1-a.compute.internal   Ready    <none>   18h   v1.31.7
   
   $ kubectl get pods -A
   NAMESPACE     NAME                                                      READY   STATUS    RESTARTS      AGE
   kube-system   cilium-bxv9t                                              1/1     Running   0             18h
   kube-system   cilium-operator-7ffb994f45-5vx7g                          1/1     Running   0             25h
   ...

   To confirm the RBAC policy is working correctly, try to access a resource not specified in the k8s-oidc-limited role (like storageclass). This should result in a "Forbidden" error.

   $ kubectl get storageclass
   Error from server (Forbidden): storageclasses.storage.k8s.io is forbidden: User "https://trial-2140803.okta.com/oauth2/austsycy0eVcAm78k697#00utsvrqddeJl3uR0697" cannot list resource "storageclasses" in API group "storage.k8s.io" at the cluster scope

Example OIDC Auth Workflow (Okta as the authentication provider)

This is the step-by-step process of how a user authenticates to a Kubernetes cluster using OIDC and the kubelogin helper.

1. Request Initiation: A user runs a kubectl command. The user's kubeconfig is configured to use OIDC, which automatically triggers the kubelogin plugin.
2. Browser Redirect: The kubelogin plugin opens the user's default web browser, redirecting them to the OIDC provider (e.g., Okta) to sign in.
3. User Authentication: The user authenticates with the OIDC provider in their browser. Upon success, the provider generates a signed ID Token containing claims, including the user's identity and their groups membership.
4. Token Caching: The OIDC provider sends the ID Token back to the kubelogin plugin through the browser. The plugin validates the token's signature and securely caches it on the user's local machine for future kubectl commands.
5. API Request: kubelogin provides the cached ID Token to kubectl. kubectl attaches this token (as a Bearer token) to the API request and sends it to the Kubernetes API Server.
6. Server-Side Validation: The Kubernetes API Server receives the request. It validates the ID Token's signature using the OIDC provider's public certificate. Once the token is confirmed as valid, the server reads the groups claim from within it.
7. RBAC Enforcement: The API Server checks its internal Role-Based Access Control (RBAC) policies to determine if a group listed in the token is bound to a role with the necessary permissions. If the request is authorized, the API server executes the command and sends the result back to the user's kubectl client.

Additional Resources

• Okta - OpenID Connect and RBAC
• Crusoe CMK cluster