Skip to main content

K3s

K3s is available for Edge host deployments as well as virtual clusters that you can create from cluster groups. Refer to the Create an Infrastructure Profile guide and the Create and Manage Cluster Groups guide for more information.

info

In order to use K3s as part of an Edge deployment, you need to go through the EdgeForge process and specify K3s as your intended Kubernetes distribution when you build your OS image. For more information, refer to the EdgeForge Workflow guide.

Support Lifecycle​

We support different Kubernetes distributions such as K3s, Microk8s, and RKE2 until their official EOL. The EOL is set by the respective owner. Once we stop supporting the minor version, we initiate the deprecation process. Refer to the Kubernetes Support Lifecycle guide to learn more.

warning

Once you upgrade your cluster to a new Kubernetes version, you will not be able to downgrade. We recommend that, before upgrading, you review the information provided in the Kubernetes Upgrades section.

Versions Supported​

Configure OIDC Identity Provider for Edge​

You can modify the configuration file to configure your Edge cluster to use an OpenID Connect (OIDC) Identity Provider (IDP) for authentication. You can use a custom third-party IDP, such as Okta, or use Palette as your IDP.

When you add the K3s pack to a cluster profile, Palette displays the OIDC IDP options listed below:

  • None: This setting does not require OIDC configuration for the cluster. It displays in the YAML file as noauth.
  • Custom: This is the default setting and does not require OIDC configuration. However, if desired, it allows you to specify a third-party OIDC provider by configuring OIDC statements in the YAML file as described in Configure Custom OIDC. This setting displays in the YAML file as none.

  • Palette: This setting makes Palette the IDP. Any user with a Palette account in the tenant and the proper permissions to view and access the project's resources is able to use kubectl CLI to access cluster. This setting displays in the YAML file as palette. When you select Palette, all you have to do to enable OIDC for your cluster is create role bindings to configure authorization. You do not need to provide extra parameters such as oidc-issuer-url as you need to when you configure a custom OIDC provider.

  • Inherit from Tenant: This setting allows you to apply RBAC to multiple clusters and requires you to configure OpenID Connect (OIDC) in Tenant Settings. In Tenant Admin scope, navigate to Tenant Settings > SSO, choose OIDC, and provide your third-party IDP details. This setting displays in the YAML file as tenant. For more information, check out the SSO Setup guide.

All the options require you to map a set of users or groups to a Kubernetes RBAC role. To learn how to map a Kubernetes role to users and groups, refer to Create Role Bindings.

warning

If your IDP uses Security Assertion Markup Language (SAML) authentication, then the Inherit from Tenant option will not work, and you will need to use the Custom option instead. This is because Kubernetes supports only OIDC authentication and not SAML authentication.

To configure a custom OIDC IDP, choose Custom when adding the K3s pack to your profile, and then follow these steps:

  1. Add the following OIDC parameters to the kube-apiserver-arg section of your configuration file for your Kubernetes layer when creating a cluster profile.

    cluster:
      config:
        kube-apiserver-arg:
          - oidc-issuer-url="provider URL"
          - oidc-client-id="client-id"
          - oidc-groups-claim="groups"
          - oidc-username-claim="email"

  2. Add the following clientConfig section that contains OIDC parameters to your Kubernetes YAML file and replace the placeholders with your third-party OIDC IDP details. The clientConfig section must be placed at the root level of the YAML file.

    clientConfig:
      oidc-issuer-url: "OIDC-ISSUER-URL"
      oidc-client-id: "OIDC-CLIENT-ID"
      oidc-client-secret: "OIDC-CLIENT-SECRET"
      oidc-extra-scope: profile,email,openid

After you have configured the IDP for authentication, you can proceed to create role bindings to configure authorization in your cluster. Refer to Create Role Bindings for more guidance.

Configure OIDC Identity Provider for Palette Virtual Clusters​

If you are using K3s in a virtual cluster inside of a cluster group, you can also configure OIDC for your cluster. Refer to Configure OIDC for a Virtual Cluster for more guidance.

Add a Certificate for Reverse Proxy​

You can use a reverse proxy with a K3s Kubernetes cluster. The reverse proxy allows you to connect to the cluster API of a Palette-managed Kubernetes cluster in private networks or clusters configured with private API endpoints. For more information, refer to the Spectro Proxy pack guide.

Terraform​

You can retrieve details about the K3s pack by using the following Terraform code.

data "spectrocloud_registry" "public_registry" {
  name = "Public Repo"
}

data "spectrocloud_pack_simple" "k8s" {
  name    = "edge-k3s"
  version = "1.27.5"
  type = "helm"
  registry_uid = data.spectrocloud_registry.public_registry.id
}