Tanzu Kubernetes Grid includes signed binaries for Dex, so that you can enable authentication services in your clusters. Dex is an OpenID Connect (OIDC) provider, that enables authentication for Kubernetes clusters by connecting to an external identity provider (IDP), such as an LDAP server or to an OIDC provider, such as Okta.

IMPORTANT: Tanzu Kubernetes Grid v1.3 introduces user authentication with Pinniped and Dex, that run automatically in management clusters if you enable identity management during deployment. Due to this change in authentication mechanism in v1.3, if you have not already manually deployed the Dex and Gangway extensions in v1.2.x, it is strongly recommended not to do so. To implement cluster authentication, it is strongly recommended to upgrade to Tanzu Kubernetes Grid v1.3 and to use the built-in Pinniped and Dex identity management service that the newer version provides.

Prerequisites

IMPORTANT:

  • In this release of Tanzu Kubernetes Grid, the provided authentication implementation assumes that you use self-signed certificates. To configure Dex and Gangway with your own certificates, engage with VMware Tanzu support.
  • Tanzu Kubernetes Grid does not support IPv6 addresses. This is because upstream Kubernetes only provides alpha support for IPv6. Always provide IPv4 addresses in the procedures in this section.
  • Tanzu Kubernetes Grid uses NodePort to expose the Dex service when running on vSphere and the LoadBalancer service type to expose the Dex service on Amazon EC2 and Azure.
  • The extensions folder tkg-extensions-v1.2.0+vmware.1 contains subfolders for each type of extension, for example, authentication, ingress, registry, and so on. At the top level of the folder there is an additional subfolder named extensions. The extensions folder also contains subfolders for authentication, ingress, registry, and so on. Take care to run commands from the location provided in the instructions. Commands are usually run from within the extensions folder.

Prepare the Management Cluster for Dex Deployment

Before you can deploy Dex on a management cluster, you must install the tools that the Dex extension requires.

This procedure applies to all management clusters, running on vSphere, Amazon EC2, and Azure.

  1. In a terminal, navigate to the folder that contains the unpacked Tanzu Kubernetes Grid extension manifest files, tkg-extensions-v1.2.0+vmware.1/extensions.

    cd <path>/tkg-extensions-v1.2.0+vmware.1/extensions
    

    You should see folders for authentication, ingress, logging, monitoring, registry, and some YAML files. Run all of the commands in these procedures from this location.

  2. Set the context of kubectl to the context of your management cluster.

    For example, if your cluster is named auth-mgmt-cluster, run the following command.

    kubectl config use-context auth-mgmt-cluster-admin@auth-mgmt-cluster
    
  3. If you haven't already, install needed components on the management cluster by following the procedure Installing Extension Prerequisite Components to a Cluster.

  4. Create a namespace for the Dex service on the management cluster.

    kubectl apply -f authentication/dex/namespace-role.yaml
    

    You should see confirmation that a tanzu-system-auth namespace, service account, and RBAC role bindings are created.

    namespace/tanzu-system-auth created
    serviceaccount/dex-extension-sa created
    role.rbac.authorization.k8s.io/dex-extension-role created
    rolebinding.rbac.authorization.k8s.io/dex-extension-rolebinding created
    clusterrole.rbac.authorization.k8s.io/dex-extension-cluster-role created
    clusterrolebinding.rbac.authorization.k8s.io/dex-extension-cluster-rolebinding created
    
  5. Check that the new services are running by listing all of the pods that are running in the management cluster.

    kubectl get pods -A
    

    In the tanzu-system-tmc namespace, you should see the extension-manager and kapp-controller services running in a pod with names similar to extension-manager-7cbdf7cbf9-xzrbn and kapp-controller-cd55bbd6b-vt2c4.

    NAMESPACE              NAME                                  READY   STATUS    RESTARTS   AGE
    [...]
    vmware-system-tmc      extension-manager-7cbdf7cbf9-xzrbn    1/1     Running   0          52s
    vmware-system-tmc      kapp-controller-cd55bbd6b-vt2c4       1/1     Running   0          40s
    

The management cluster is ready for you to deploy the Dex service. For the next steps, see the procedure that corresponds to the infrastructure in which your management cluster is running:

Update the Dex Configuration File to Deploy Dex on a Management Cluster Running on vSphere

After you have set up the management cluster, you must update the configuration file that is used when you deploy Dex on a management cluster that is running on vSphere. For management clusters that are running on vSphere, you can configure Dex to connect to either an LDAP server or to an OIDC provider.

  1. Make a copy of the dex-data-values.yaml.example file and name it dex-data-values.yaml.

    The file to copy depends on whether you are connecting Dex to an LDAP server or to an OIDC provider.

    LDAP:

    cp authentication/dex/vsphere/ldap/dex-data-values.yaml.example authentication/dex/vsphere/ldap/dex-data-values.yaml
    

    OIDC:

    cp authentication/dex/vsphere/oidc/dex-data-values.yaml.example authentication/dex/vsphere/oidc/dex-data-values.yaml
    
  2. Open dex-data-values.yaml in a text editor.
  3. Update dex-data-values.yaml with information about your management cluster and LDAP server or OIDC provider.

    LDAP:

    For most deployments, you only need to update <LDAP_HOST>, the baseDN values, and <MGMT_CLUSTER_IP>.

    • dex.config.ldap.host: Replace <LDAP_HOST> with the IP or DNS address of your LDAP server. If the LDAP server is listening on the default port 636, which is the secured configuration, you do not need to specify the port. If the LDAP server is listening on a different port, provide the address and port of the LDAP server.
    • dex.config.ldap.userSearch.baseDN and dex.config.ldap.groupSearch.baseDN: Update dc=vmware with your company name.
    • If necessary, update the userSearch and groupSearch values with parameters that match your LDAP server configuration. If you need to enable LDAP user and group search, see this KB article.
    • dex.config.staticClients: Do not update WORKLOAD_CLUSTER_NAME, WORKLOAD_CLUSTER_IP, or CLIENT_SECRET yet. You will update these later.
    • dns.vsphere.ipAddresses: In the square brackets, replace <MGMT_CLUSTER_IP> with the static virtual IP address that you provided when you deployed the management cluster. Alternatively, add the dns.vsphere.dnsNames option to specify the FQDN of the management cluster.

    For example:

    #@data/values
    #@overlay/match-child-defaults missing_ok=True
    ---
    infrastructure_provider: "vsphere"
    dex:
      config:
        connector: ldap
        ldap:
          host: ldaps.example.com:636
          userSearch:
            baseDN: 'ou=people,dc=example,dc=com'
            filter: '(objectClass=posixAccount)'
            username: uid
            idAttr: uid
            emailAttr: mail
            nameAttr: 'givenName'
          groupSearch:
            baseDN: 'ou=group,dc=example,dc=com'
            filter: '(objectClass=posixGroup)'
            userAttr: uid
            groupAttr: memberUid
            nameAttr: cn
        #@overlay/replace
        staticClients:
        - id: WORKLOAD_CLUSTER_NAME
          redirectURIs:
          - 'https://WORKLOAD_CLUSTER_IP:30166/callback'
          name: WORKLOAD_CLUSTER_NAME
          secret: CLIENT_SECRET
    dns:
      vsphere:
        #@overlay/replace
        ipAddresses: [192.168.10.66]
    

    OIDC:

    Update <OIDC_CLIENT_ID>, <OIDC_CLIENT_SECRET>, <OIDC_IDP_URL>, and <MGMT_CLUSTER_IP>.

    • dex.config.oidc.CLIENT_ID: Replace <OIDC_CLIENT_ID> with the Base64 client_id value that you obtain from your OIDC provider.
    • dex.config.oidc.CLIENT_SECRET: Replace <OIDC_CLIENT_SECRET> with the Base64 secret vallue that you obtain from your OIDC provider.

      For example, if your provider is Okta, log in to Okta, create a Web application, and select the Client Credentials options in order to get a client_id and secret.

    • dex.config.oidc.issuer: Replace <OIDC_IDP_URL> with the IP or DNS address of your OIDC server.
    • dex.config.staticClients: Do not update WORKLOAD_CLUSTER_NAME, WORKLOAD_CLUSTER_IP, or CLIENT_SECRET yet. You will update these later.
    • dns.vsphere.ipAddresses: In the square brackets, replace <MGMT_CLUSTER_IP> with the static virtual IP address that you provided when you deployed the management cluster. Alternatively, add the dns.vsphere.dnsNames option to specify the FQDN of the management cluster.
    • If you need to enable user and group search, open the file ../authentication/dex/values.yaml in a text editor and set dex.config.oidc.insecureEnableGroups: true.

    For example:

    #@data/values
    #@overlay/match-child-defaults missing_ok=True
    ---
    infrastructure_provider: "vsphere"
    dex:
      config:
        connector: oidc
        oidc:
          CLIENT_ID: 0oa5exxxxxxnoiO4x6
          CLIENT_SECRET: Ay5cCxxxxxxxxxxxxhhTpli7mV1Nmf3N3J3dKVPM
          issuer: https://my-server.okta.com
        #@overlay/replace
        staticClients:
        - id: <WORKLOAD_CLUSTER_NAME>
          redirectURIs:
          - 'https://<WORKLOAD_CLUSTER_IP>:30166/callback'
          name: <WORKLOAD_CLUSTER_NAME>
          secret: <CLIENT_SECRET>
    dns:
      vsphere:
        #@overlay/replace
        ipAddresses: [192.168.100.44]
    
  4. Save the updated dex-data-values.yaml file.
  5. Create a Kubernetes secret named dex-data-values with the values that you set in dex-data-values.yaml.

    LDAP:

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/ldap/dex-data-values.yaml -n tanzu-system-auth
    

    OIDC:

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/oidc/dex-data-values.yaml -n tanzu-system-auth
    

    You should see the confirmation secret/dex-data-values created.

For the next steps, see Deploy Dex on the Management Cluster.

Update the Dex Configuration File to Deploy Dex on a Management Cluster Running on Amazon EC2

After you have set up the management cluster, you must update the configuration file that is used when you deploy Dex on a management cluster that is running on Amazon EC2. For management clusters that are running on Amazon EC2, you configure Dex to connect to an OIDC provider.

  1. Make a copy of the dex-data-values.yaml.example file and name it dex-data-values.yaml.

    cp authentication/dex/aws/oidc/dex-data-values.yaml.example authentication/dex/aws/oidc/dex-data-values.yaml
    
  2. Open dex-data-values.yaml in a text editor.
  3. Update dex-data-values.yaml with information about your management cluster and OIDC provider.

    • dex.config.oidc.CLIENT_ID: Replace <OIDC_CLIENT_ID> with the Base64 client_id value that you obtain from your OIDC provider.
    • dex.config.oidc.CLIENT_SECRET: Replace <OIDC_CLIENT_SECRET> with the Base64 secret vallue that you obtain from your OIDC provider.

      For example, if your provider is Okta, log in to Okta, create a Web application, and select the Client Credentials options in order to obtain the Client ID and Client secret.

    • dex.config.oidc.issuer: Replace <OIDC_PROVIDER_ADDRESS> with the IP or DNS address of your OIDC server.
    • dex.config.staticClients: Do not update GANGWAY_SVC_LB_HOSTNAME, WORKLOAD_CLUSTER_NAME, or CLIENT_SECRET yet. You will update these later.
    • dns.aws.DEX_SVC_LB_HOSTNAME: Do not update sample-elb yet. The Dex service load balancer is created when you deploy the Dex service, so you must update this value after you have deployed Dex.
    • If you need to enable user and group search, open the file ../authentication/dex/values.yaml in a text editor and set dex.config.oidc.insecureEnableGroups: true.

    For example:

    #@data/values
    #@overlay/match-child-defaults missing_ok=True
    ---
    infrastructure_provider: "aws"
    dex:
      config:
        connector: oidc
        oidc:
          CLIENT_ID: 0oa5exxxxxxxxxxiO4x6
          CLIENT_SECRET: Ay5cCjxxxxxxxxxxxxxxxxxxxx1Nmf3N3J3dKVPM
          issuer: https://my-account.okta.com/
        #! Deploy dex first with dummy staticClients....
        #@overlay/replace
        staticClients:
        - id: WORKLOAD_CLUSTER_NAME
          redirectURIs:
          - 'https://GANGWAY_SVC_LB_HOSTNAME/callback'
          name: WORKLOAD_CLUSTER_NAME
          secret: CLIENT_SECRET
    #! Deploy dex first and update sample-elb with real name once dex service loadbalancer is created, dnsNames is optional.
    dns:
      aws:
        DEX_SVC_LB_HOSTNAME: sample-elb
        dnsNames: []
    
  4. Save the updated dex-data-values.yaml file.
  5. Create a Kubernetes secret named dex-data-values with the values that you set in dex-data-values.yaml.

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/aws/oidc/dex-data-values.yaml -n tanzu-system-auth
    

    You should see the confirmation secret/dex-data-values created.

For the next steps, see Deploy Dex on the Management Cluster.

Update the Dex Configuration File to Deploy Dex on a Management Cluster Running on Azure

After you have set up the management cluster, you must update the configuration file that is used when you deploy Dex on a management cluster that is running on Azure. For management clusters that are running on Azure, you configure Dex to connect to a Secure LDAP server.

  1. Make a copy of the dex-data-values.yaml.example file and name it dex-data-values.yaml.

    cp authentication/dex/azure/ldap/dex-data-values.yaml.example authentication/dex/azure/ldap/dex-data-values.yaml
    
  2. Open dex-data-values.yaml in a text editor.
  3. Update dex-data-values.yaml with information about your management cluster and Secure LDAP server.

    For most deployments, you only need to update <LDAP_HOST> and the baseDN values.

    • dex.config.ldap.host: Replace <LDAP_HOST> with the IP or DNS address of your LDAP server. If the LDAP server is listening on the default port 636, which is the secured configuration, you do not need to specify the port. If the LDAP server is listening on a different port, provide the address and port of the LDAP server.
    • dex.config.ldap.userSearch.baseDN and dex.config.ldap.groupSearch.baseDN: Update dc=vmware with your company name.
    • If necessary, update the userSearch and groupSearch values with parameters that match your LDAP server configuration. If you need to enable LDAP user and group search, see this KB article.
    • dex.config.staticClients: Do not update WORKLOAD_CLUSTER_NAME, WORKLOAD_CLUSTER_IP, or CLIENT_SECRET yet. You will update these later.
    • dns.azure.DEX_SVC_LB_HOSTNAME: Do not update sample-elb yet. The Dex service load balancer is created when you deploy the Dex service, so you must update this value after you have deployed Dex.

    For example:

    #@data/values
    #@overlay/match-child-defaults missing_ok=True
    ---
    infrastructure_provider: "azure"
    dex:
      config:
        connector: ldap
        ldap:
          host: host: ldaps.example.com:636
          userSearch:
            baseDN: 'ou=people,dc=example,dc=com'
            filter: '(objectClass=person)'
            username: userPrincipalName
            idAttr: DN
            emailAttr: userPrincipalName
            nameAttr: 'cn'
          groupSearch:
            baseDN: 'ou=group,dc=example,dc=com'
            filter: '(objectClass=group)'
            userAttr: DN
            groupAttr: "member:1.2.840.113556.1.4.1941:"
            nameAttr: cn
        #@overlay/replace
        staticClients:
        - id: WORKLOAD_CLUSTER_NAME
          redirectURIs:
          - 'https://WORKLOAD_CLUSTER_IP:30166/callback'
          name: WORKLOAD_CLUSTER_NAME
          secret: CLIENT_SECRET
    dns:
      azure:
        DEX_SVC_LB_HOSTNAME: sample-lb
        dnsNames: []
    
  4. Save the updated dex-data-values.yaml file.
  5. Create a Kubernetes secret named dex-data-values with the values that you set in dex-data-values.yaml.

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/azure/ldap/dex-data-values.yaml -n tanzu-system-auth
    

    You should see the confirmation secret/dex-data-values created.

For the next steps, see Deploy Dex on the Management Cluster.

Deploy Dex on the Management Cluster

After you have prepared the management cluster and updated the appropriate configuration file for your platform, you can deploy Dex on the management cluster.

This procedure applies to all management clusters, running on vSphere, Amazon EC2, and Azure.

  1. Deploy the Dex extension.

    kubectl apply -f authentication/dex/dex-extension.yaml
    

    You should see the confirmation extension.clusters.tmc.cloud.vmware.com/dex created.

  2. View the status of the Dex extension.

    kubectl get extension dex -n tanzu-system-auth
    

    You should see information about the Dex extension.

    NAME   STATE   HEALTH   VERSION
    dex    3
    
  3. View the status of the Dex service itself.

    kubectl get app dex -n tanzu-system-auth
    

    The status of the Dex app should show Reconcile Succeeded when Dex has deployed successfully.

    NAME   DESCRIPTION                    SINCE-DEPLOY   AGE
    dex    Reconcile Succeeded                           53m
    

    If the status is not Reconcile Succeeded, view the full status details of the Dex service.

    Viewing the full status can help you to troubleshoot the problem.

    kubectl get app dex -n tanzu-system-auth -o yaml
    
  4. Check that the Dex service is running by listing all of the pods that are running in the management cluster.

    kubectl get pods -A
    

    In the tanzu-system-auth namespace, you should see the dex service running in a pod with a name similar to dex-56687dcdbc-qjkrm.

    NAMESPACE              NAME                                  READY   STATUS    RESTARTS   AGE
    [...]
    tanzu-system-auth      dex-56687dcdbc-qjkrm                  1/1     Running   0          6m11s
    vmware-system-tmc      extension-manager-d7cc7fcbb-t77m7     1/1     Running   0          13m
    vmware-system-tmc      kapp-controller-7c98dff676-lbt6p      1/1     Running   0          11m
    

What to Do Next

If you deployed Dex on a management cluster that is running on Amazon EC2 or Azure, you have additional steps to perform. See Obtain the Dex Load Balancer for Amazon EC2 or Obtain the Dex Load Balancer for Azure below.

If you deployed Dex on a management cluster that is running on vSphere, your Dex deployment is complete. Next, you must deploy a Tanzu Kubernetes cluster with an embedded OIDC endpoint that can connect to your LDAP server or OIDC provider, via the Dex service that is running in the management cluster. Go to Deploy an Authentication-Enabled Tanzu Kubernetes Cluster.

Obtain the Dex Load Balancer for Amazon EC2

If you deployed Dex on a management cluster that is running on Amazon EC2, you must perform additional steps to obtain the address of the Dex service load balancer that is running in the management cluster. You then need to update the Dex configuration to use it.

  1. Obtain the host name of the Dex service load balancer.

    kubectl get svc dexsvc -n tanzu-system-auth -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
    
  2. Open authentication/dex/aws/oidc/dex-data-values.yaml in a text editor and update the DEX_SVC_LB_HOSTNAME parameter with the address that you obtained in the preceding step.

    Add the load balancer address in the dns.aws.DEX_SVC_LB_HOSTNAME parameter. The address will resemble the following:

    dns:
      aws:
        DEX_SVC_LB_HOSTNAME: a708e794809d54af699834323c33a0a-463181988.us-west-2.elb.amazonaws.com
    
  3. Update the dex-data-values secret to include the load balancer address.

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/aws/oidc/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
    

    You should see the confirmation secret/dex-data-values replaced. The changes should show up in five minutes or less. This is handled by the Kapp controller, which synchronizes every five minutes.

Now, the Dex service is ready and you can deploy a Tanzu Kubernetes cluster with an embedded OIDC endpoint that can connect to your LDAP server or OIDC provider via the Dex service. Go to Deploy an Authentication-Enabled Tanzu Kubernetes Cluster.

Obtain the Dex Load Balancer for Azure

If you deployed Dex on a management cluster that is running on Azure, you must perform additional steps to obtain the address of the Dex service load balancer that is running in the management cluster. You then need to update the Dex configuration to use it.

  1. Obtain the external IP address of the Dex service load balancer.

     kubectl get svc dexsvc -n tanzu-system-auth -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
    
  2. Open authentication/dex/azure/ldap/dex-data-values.yaml in a text editor and update the DEX_SVC_LB_HOSTNAME parameter with the address that you obtained in the preceding step.

    Add the load balancer address in the dns.azure.DEX_SVC_LB_HOSTNAME parameter. The address will resemble the following:

    dns:
      azure:
        DEX_SVC_LB_HOSTNAME: 20.54.226.42
    
  3. Update the dex-data-values secret to include the load balancer address.

    This command assumes that you are running it from tkg-extensions-v1.2.0+vmware.1/extensions.

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/azure/ldap/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
    

    You should see the confirmation secret/dex-data-values replaced.

  4. Redeploy the Dex extension.

    kubectl apply -f authentication/dex/dex-extension.yaml
    

    You should see the confirmation extension.clusters.tmc.cloud.vmware.com/dex configured.

Now, the Dex service is ready and you can deploy a Tanzu Kubernetes cluster with an embedded OIDC endpoint that can connect to your LDAP server or OIDC provider via the Dex service. Go to Deploy an Authentication-Enabled Tanzu Kubernetes Cluster.

Update a Dex Deployment

If you need to make changes to the configuration of the Dex extension, you can update the extension.

Perform the steps in this procedure if you modify either of the extensions/authentication/dex/<INFRA_PROVIDER>/<AUTH>/dex-data-values.yaml or authentication/dex/values.yaml files after the initial deployment of Dex.

  1. Obtain the Dex data values from the Kubernetes secret.

    kubectl get secret dex-data-values -n tanzu-system-auth -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > dex-data-values.yaml
    
  2. Modify either or both of the extensions/authentication/dex/<INFRA_PROVIDER>/<AUTH>/dex-data-values.yaml or authentication/dex/values.yaml files to update your configuration.
  3. Update the Kubernetes secret.

    This command assumes that you are running it from tkg-extensions-v1.2.0+vmware.1/extensions.

    vSphere with LDAP:

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/ldap/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
    

    vSphere with OIDC:

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/oidc/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
    

    Amazon EC2:

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/aws/oidc/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
    

    Azure:

    kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/azure/ldap/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
    

    Note that the final - on the kubectl replace command above is necessary to instruct kubectl to accept the input being piped to it from the kubectl create secret command.

    The Dex extension will be reconciled using the new values you just added. The changes should show up in five minutes or less. This is handled by the Kapp controller, which synchronizes every five minutes.

check-circle-line exclamation-circle-line close-line
Scroll to top icon