Configure and Manage TLS Certificates for Pinniped and Dex

This topic explains how to configure custom TLS certificates for Pinniped and Dex in Tanzu Kubernetes Grid. It also explains how to update the Dex certificates in Tanzu Kubernetes Grid.

Configuring Custom TLS Certificates

By default, Tanzu Kubernetes Grid uses a self-signed Issuer to generate the TLS certificates that secure HTTPS traffic to Pinniped and Dex. You can optionally update the default configuration after deploying the management cluster, as follows:

  1. Set a custom ClusterIssuer resource or your own TLS secret. See Set a ClusterIssuer Resource or a TLS Secret below.
  2. Update your Pinniped configuration by redeploying the Pinniped add-on secret for the management cluster. See Update Your Pinniped Configuration below.

Note: Tanzu Kubernetes Grid deploys Dex only for LDAP identity providers. If you configure an OIDC identity provider when you create the management cluster or configure it as a post-creation step, Dex is not deployed.

Set a ClusterIssuer Resource or a TLS Secret

If you want to use a custom ClusterIssuer resource to generate the TLS certificates:

  1. Verify that cert-manager is running in your management cluster. This component runs by default in all management clusters.
  2. Obtain the name of an existing ClusterIssuer resource in the management cluster. For more information, see Issuer Configuration in the cert-manager documentation.
  3. Specify the ClusterIssuer name in the custom_cluster_issuer field of the values.yaml section in the Pinniped add-on secret for the management cluster and then apply your changes. For instructions, see Update Your Pinniped Configuration below. After you complete the steps in this section, both the Pinniped the Dex certificate chains will be signed by your ClusterIssuer.

If you want to specify your own TLS secret directly:

  1. Retrieve the IP address or DNS hostname of the Pinniped service, pinniped-supervisor, and if you are using an LDAP identity provider, of the Dex service, dexsvc:

    • The pinniped-supervisor service:

      • If the type of the service is set to LoadBalancer (vSphere with a load balancer, for example, NSX Advanced Load Balancer, Amazon EC2, or Azure), retrieve the external address of the service by running:

        kubectl get service pinniped-supervisor -n pinniped-supervisor
        
      • If the type of the service is set to NodePort (vSphere without a load balancer), the IP address of the service is the same as the vSphere control plane endpoint. To retrieve the IP address, you can run the following command:

        kubectl get configmap cluster-info -n kube-public -o yaml
        
    • (LDAP only) The dexsvc service:

      • If the type of the service is set to LoadBalancer (vSphere with a load balancer, for example, NSX Advanced Load Balancer, Amazon EC2, or Azure), retrieve the external address of the service by running:

        kubectl get service dexsvc -n tanzu-system-auth
        
      • If the type of the service is set to NodePort (vSphere without a load balancer), the IP address of the service is the same as the vSphere control plane endpoint. To retrieve the IP address, you can run the following command:

        kubectl get configmap cluster-info -n kube-public -o yaml
        
  2. Create a kubernetes.io/tls secret in the pinniped-supervisor namespace if you are using an OIDC identity provider. If you are using an LDAP identity provider, create two kubernetes.io/tls secrets with the same name, one, for the Pinniped service, in the pinniped-supervisor namespace and one, for the Dex service, in the tanzu-system-auth namespace. To create a TLS secret, run:

    kubectl create secret generic SECRET-NAME -n SECRET-NAMESPACE --type kubernetes.io/tls --from-file tls.crt=FILENAME-1.crt --from-file tls.key=FILENAME-2.pem --from-file ca.crt=FILENAME-3.pem
    

    Replace the placeholder text as follows:

    • SECRET-NAME is the name you choose for the secret. For example, my-secret.
    • SECRET-NAMESPACE is the namespace in which to create the secret. This must be pinniped-supervisor for Pinniped and tanzu-system-auth for Dex.
    • FILENAME-* is the name of your tls.crt, tls.key, or ca.crt. The TLS certificate that you specify in the TLS secret for Pinniped must include the IP or DNS hostname of the Pinniped service from the step above. Similarly, the TLS certificate for Dex must include the IP address or DNS hostname of the Dex service that you retrieved above. The ca.crt contains the CA bundle that is used to verify the TLS certificate.

    For example, the resulting secret for Pinniped looks similar to the following:

    apiVersion: v1
    kind: Secret
    metadata:
     name: my-secret
     namespace: pinniped-supervisor
    type: kubernetes.io/tls
    data:
     tls.crt: |       MIIC2DCCAcCgAwIBAgIBATANBgkqh ...
     tls.key: |       MIIEpgIBAAKCAQEA7yn3bRHQ5FHMQ ...
     ca.crt:  |       MIIEpgIBAAKCAQEA7yn3bRHQ5FHMQ ...
    

    If the secret has been generated correctly, decoding tls.crt with openssl x509 -in tls.crt -text shows the IP address or DNS hostname in the Subject Alternative Name field.

  3. Specify the secret name in the custom_tls_secret field of the values.yaml section in the Pinniped add-on secret for the management cluster and then apply your changes. For instructions, see Update Your Pinniped Configuration below.

Update Your Pinniped Configuration

To apply your changes, update the Pinniped configuration by following the steps below:

  1. Save the Pinniped add-on secret for the management cluster to a file:

    kubectl get secret CLUSTER-NAME-pinniped-addon -n tkg-system -o yaml > FILENAME.yaml
    

    Replace the placeholder text as follows:

    • CLUSTER-NAME is the name of your management cluster.
    • FILENAME is the file name that you want to use for the secret. For example, pinniped-addon-secret.yaml.
  2. Decode the Base64-encoded string in the values.yaml section of the secret using the decoding tool of your choice. For example:

    pbpaste | base64 -d > pinniped-configuration.yaml
    
  3. In the decoded text, do one of the following:

    • If you prepared a ClusterIssuer resource above, specify the name of the resource in the custom_cluster_issuer field. For example:

      #@data/values
      #@overlay/match-child-defaults missing_ok=True
      ---
      infrastructure_provider: vsphere
      tkg_cluster_role: management
      custom_cluster_issuer: "my-cluster-issuer-name"
      pinniped:
       cert_duration: 2160h
       cert_renew_before: 360h
       supervisor_svc_endpoint: https://10.168.217.220:31234
       supervisor_ca_bundle_data: LS0tLS1CRUdJTiBDRVJUSUZJQ0F……
      ...
      
    • If you prepared your own TLS secret above, specify the name of the secret in the custom_tls_secret field. For example:

      #@data/values
      #@overlay/match-child-defaults missing_ok=True
      ---
      infrastructure_provider: vsphere
      tkg_cluster_role: management
      custom_tls_secret: "my-tls-secret-name"
      pinniped:
       cert_duration: 2160h
       cert_renew_before: 360h
       supervisor_svc_endpoint: https://10.168.217.220:31234
       supervisor_ca_bundle_data: LS0tLS1CRUdJTiBDRVJUSUZJQ0F……
      ...
      

    If you configure the custom_tls_secret field, the custom_cluster_issuer is ignored.

  4. Encode the values.yaml section again using the Base64 scheme. For example:

    cat pinniped-configuration.yaml | base64 > encoded.txt
    
  5. Copy the encoded string and paste it into the Pinniped add-on secret file that you retrieved from the cluster in step 1. For example, pinniped-addon-secret.yaml.

  6. Redeploy the Pinniped add-on secret by running the kubectl apply -f command. For example:

    kubectl apply -f pinniped-addon-secret.yaml
    
  7. Confirm that the changes have been applied successfully:

    1. Get the status of the pinniped app:

      kubectl get app pinniped -n tkg-system
      

      If the returned status is Reconcile failed, run the following command to get details on the failure:

      kubectl get app pinniped -n tkg-system -o yaml
      
    2. Generate a kubeconfig file for the management cluster by running the tanzu mc kubeconfig get command and then run kubectl get pods using the kubeconfig. Additionally, if your management cluster manages any workload clusters, run tanzu cluster kubeconfig get and then kubectl get pods against each of the existing clusters.

Update Dex Serving Certificate and Dex CA certificate

With clusters that use LDAP identity management, you might want to update the Dex certificates if:

  • The Dex CA certificate has been updated or expired.
  • The private key associated with the Dex CA is compromised.

Prerequisites

Before performing this procedure, ensure that you have:

  • Configured the Pinniped addon with LDAP IDP. For more information, see Identity Providers - LDAP.
  • Obtained the address of the Dex service by using the kubectl get service dexsvc -n tanzu-system-auth command.

Update the Dex Serving Certificate

  1. Change the kubectl context to the management cluster, if you are currently in the workload cluster context. For more information, See Retrieve Tanzu Kubernetes Cluster kubeconfig.

  2. Download the current Dex serving certificate:

    openssl s_client -connect ADDRESS -showcerts </dev/null  | openssl x509 -noout -text > /tmp/OLD-FILE.txt
    

    Where:

    • ADDRESS is the Dex service address in Tanzu Kubernetes Grid.
    • OLD-FILE is the name in which you want to save the serving certificate text file, for example, before.txt.
  3. Delete the current Dex serving certificate secret so that cert-manager re-creates it:

    kubectl delete secret -n tanzu-system-auth  dex-cert-tls
    

    The following is a sample output:

    secret "dex-cert-tls" deleted
    
  4. Verify that a new Dex serving certificate is created:

    kubectl get secret -n tanzu-system-auth
    

    The following is a sample output:

    kubectl get secret -n tanzu-system-auth
    NAME                  TYPE                                  DATA   AGE
    default-token-cg8f2   kubernetes.io/service-account-token   3      6m5s
    dex-ca-key-pair       kubernetes.io/tls                     3      5m50s
    dex-cert-tls          kubernetes.io/tls                     3      2s
    dex-token-p96gl       kubernetes.io/service-account-token   3      6m3s
    
  5. Restart the Dex pods:

    kubectl delete pod -n tanzu-system-auth -l app=dex
    

    The following is a sample output:

    pod DEX-POD deleted
    
  6. Download the new Dex serving certificate:

    openssl s_client -connect ADDRESS -showcerts </dev/null  | openssl x509 -noout -text > /tmp/NEW-FILE.txt
    

    Where:

    • ADDRESS is the Dex service address in Tanzu Kubernetes Grid.
    • NEW-FILE is the name in which you want to save the new serving certificate text file, for example, after.txt.
  7. Compare the old and the new certificates to ensure that the new certificate has been created:

    diff /tmp/OLD-FILE /tmp/NEW-FILE
    

    Where:

    • OLD-FILE is the name of the old Dex serving certificate.
    • NEW-FILE is the name of the new Dex serving certificate.

Update the Dex CA Certificate

  1. Download the current Dex CA certificate:

    kubectl get secret -n tanzu-system-auth dex-cert-tls -o jsonpath={.data.ca\\.crt} | base64 -d | openssl x509 -noout -text > /tmp/OLD-FILE.txt
    

    Where OLD-FILE is the name in which you want to save the serving certificate text file, for example, ca-before.txt.

  2. Download the current Dex serving certificate:

    openssl s_client -connect ADDRESS -showcerts </dev/null  | openssl x509 -noout -text > /tmp/OLD-FILE.txt
    

    Where:

    • ADDRESS is the Dex service address in Tanzu Kubernetes Grid.

    • OLD-FILE is the name in which you want to save the serving certificate text file, for example, cert-before.txt.

  3. Download the current Dex CA data:

    kubectl get oidcidentityprovider upstream-oidc-identity-provider -n pinniped-supervisor -o jsonpath={.spec.tls.certificateAuthorityData} | base64 -d  | openssl x509 -noout -text > /tmp/OLD-FILE.txt
    

    Where OLD-FILE is the name in which you want to save the serving certificate text file, for example, ca-data-before.txt.

  4. Delete the Dex CA certificate secret so that cert-manager re-creates it:

    kubectl delete secret -n tanzu-system-auth dex-ca-key-pair
    

    The following is a sample output:

    secret "dex-ca-key-pair" deleted
    
  5. Verify that a new Dex CA certificate is created:

    kubectl get secret -n tanzu-system-auth
    

    The following is a sample output:

    kubectl get secret -n tanzu-system-auth
    NAME                  TYPE                                  DATA   AGE
    default-token-cg8f2   kubernetes.io/service-account-token   3      25m
    dex-ca-key-pair       kubernetes.io/tls                     3      18s
    dex-cert-tls          kubernetes.io/tls                     3      17s
    dex-token-p96gl       kubernetes.io/service-account-token   3      25m
    
    
  6. Delete the current Dex serving certificate secret so that cert-manager re-creates it:

    kubectl delete secret -n tanzu-system-auth  dex-cert-tls
    

    The following is a sample output:

    secret "dex-cert-tls" deleted
    
  7. Verify that a new Dex serving certificate is created:

    kubectl get secret -n tanzu-system-auth
    

    The following is a sample output:

    kubectl get secret -n tanzu-system-auth
    NAME                  TYPE                                  DATA   AGE
    default-token-cg8f2   kubernetes.io/service-account-token   3      6m5s
    dex-ca-key-pair       kubernetes.io/tls                     3      5m50s
    dex-cert-tls          kubernetes.io/tls                     3      2s
    dex-token-p96gl       kubernetes.io/service-account-token   3      6m3s
    
  8. Delete the Pinniped post-deploy job:

    kubectl delete job -n pinniped-supervisor pinniped-post-deploy-job
    

    The following is a sample output:

    job.batch "pinniped-post-deploy-job" deleted
    
  9. Update the Pinniped add-on to sync your changes quickly, and wait a few minutes for the add-on to reconcile the changes:

    kubectl patch app pinniped -n tkg-system -p '{"spec":{"syncPeriod":"30s"}}' --type merge
    
    kubectl wait app pinniped -n tkg-system --for condition=ReconcileSucceeded --timeout 5m
    
  10. Download the new Dex CA certificate:

    kubectl get secret -n tanzu-system-auth dex-cert-tls -o jsonpath={.data.ca\\.crt} | base64 -d | openssl x509 -noout -text > /tmp/NEW-FILE.txt
    

    Where NEW-FILE is the name in which you want to save the new Dex CA certificate text file, for example, ca-after.txt.

  11. Download the new Dex serving certificate:

    openssl s_client -connect ADDRESS -showcerts </dev/null  | openssl x509 -noout -text > /tmp/NEW-FILE.txt
    

    Where:

    • ADDRESS is the Dex service address in Tanzu Kubernetes Grid.
    • NEW-FILE is the name in which you want to save the new Dex serving certificate text file, for example, after.txt.

    • Download the new Dex CA data:

    kubectl get oidcidentityprovider upstream-oidc-identity-provider -n pinniped-supervisor -o jsonpath={.spec.tls.certificateAuthorityData} | base64 -d  | openssl x509 -noout -text > /tmp/NEW-FILE.txt
    

    Where NEW-FILE is the name in which you want to save the new Dex CA data text file, for example, ca-data-after.txt.

  12. Compare the old and the new CA certificates to ensure that the new CA certificate has been created:

    diff /tmp/OLD-FILE /tmp/NEW-FILE
    

    Where:

    • OLD-FILE is the name of the old Dex CA certificate.
    • NEW-FILE is the name of the old Dex CA certificate.
  13. Compare the old and the new serving certificates to ensure that the new serving certificate has been created:

    diff /tmp/OLD-FILE /tmp/NEW-FILE
    

    Where:

    • OLD-FILE is the name of the old Dex serving certificate.
    • NEW-FILE is the name of the old Dex serving certificate.
  14. Compare the old and the new CA data to ensure that the new CA data has been created:

    diff /tmp/OLD-FILE /tmp/NEW-FILE
    

    Where:

    • OLD-FILE is the name of the old Dex CA data file.
    • NEW-FILE is the name of the old Dex CA data file.
check-circle-line exclamation-circle-line close-line
Scroll to top icon