Tanzu Kubernetes Grid includes signed binaries for Dex, so that you can enable authentication services in your clusters. Dex is an OpenID Connect (OIDC) provider, that enables authentication for Kubernetes clusters by connecting to an external identity provider (IDP), such as an LDAP server or to an OIDC provider, such as Okta.
auth-mgmt-cluster.IMPORTANT:
NodePort to expose the Dex service when running on vSphere and the LoadBalancer service type to expose the Dex service on Amazon EC2 and Azure.Before you can deploy Dex on a management cluster, you must install the tools that the Dex extension requires.
This procedure applies to all management clusters, running on vSphere, Amazon EC2, and Azure.
In a terminal, navigate to the folder that contains the unpacked Tanzu Kubernetes Grid extension manifest files, tkg-extensions-v1.2.0+vmware.1/extensions.
You should see folders for authentication, ingress, logging, monitoring, registry, and some YAML files. Run all of the commands in these procedures from this location.
Set the context of kubectl to the context of your management cluster.
For example, if your cluster is named auth-mgmt-cluster, run the following command.
kubectl config use-context auth-mgmt-cluster-admin@auth-mgmt-cluster
Install the VMware Tanzu Mission Control extension manager on the management cluster.
The Tanzu Kubernetes Grid extensions and Tanzu Mission Control both use the same extensions manager service. You must install the extensions manager even if you do not intend to use Tanzu Mission Control.
kubectl apply -f tmc-extension-manager.yaml
You should see confirmation that a namespace, resource definitions, a service account, an RBAC role, and a role binding for the extension-manager service are all created.
namespace/vmware-system-tmc created
customresourcedefinition.apiextensions.k8s.io/agents.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensions.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionresourceowners.clusters.tmc.cloud.vmware.com created
customresourcedefinition.apiextensions.k8s.io/extensionconfigs.intents.tmc.cloud.vmware.com created
serviceaccount/extension-manager created
clusterrole.rbac.authorization.k8s.io/extension-manager-role created
clusterrolebinding.rbac.authorization.k8s.io/extension-manager-rolebinding created
service/extension-manager-service created
deployment.apps/extension-manager created
Install the Kapp controller on the management cluster.
kubectl apply -f kapp-controller.yaml
You should see confirmation that a service account, resource definition, and RBAC role are created for the kapp-controller service.
serviceaccount/kapp-controller-sa created
customresourcedefinition.apiextensions.k8s.io/apps.kappctrl.k14s.io created
deployment.apps/kapp-controller created
clusterrole.rbac.authorization.k8s.io/kapp-controller-cluster-role created
clusterrolebinding.rbac.authorization.k8s.io/kapp-controller-cluster-role-binding created
Create a namespace for the Dex service on the management cluster.
kubectl apply -f authentication/dex/namespace-role.yaml
You should see confirmation that a tanzu-system-auth namespace, service account, and RBAC role bindings are created.
namespace/tanzu-system-auth created
serviceaccount/dex-extension-sa created
role.rbac.authorization.k8s.io/dex-extension-role created
rolebinding.rbac.authorization.k8s.io/dex-extension-rolebinding created
clusterrole.rbac.authorization.k8s.io/dex-extension-cluster-role created
clusterrolebinding.rbac.authorization.k8s.io/dex-extension-cluster-rolebinding created
Check that the new services are running by listing all of the pods that are running in the management cluster.
kubectl get pods -A
In the tanzu-system-tmc namespace, you should see the extension-manager and kapp-controller services running in a pod with names similar to extension-manager-7cbdf7cbf9-xzrbn and kapp-controller-cd55bbd6b-vt2c4.
NAMESPACE NAME READY STATUS RESTARTS AGE
[...]
vmware-system-tmc extension-manager-7cbdf7cbf9-xzrbn 1/1 Running 0 52s
vmware-system-tmc kapp-controller-cd55bbd6b-vt2c4 1/1 Running 0 40s
The management cluster is ready for you to deploy the Dex service. For the next steps, see the procedure that corresponds to the infrastructure in which your management cluster is running:
After you have set up the management cluster, you must update the configuration file that is used when you deploy Dex on a management cluster that is running on vSphere. For management clusters that are running on vSphere, you can configure Dex to connect to either an LDAP server or to an OIDC provider.
Make a copy of the dex-data-values.yaml.example file and name it dex-data-values.yaml.
The file to copy depends on whether you are connecting Dex to an LDAP server or to an OIDC provider.
LDAP:
cp authentication/dex/vsphere/ldap/dex-data-values.yaml.example authentication/dex/vsphere/ldap/dex-data-values.yaml
OIDC:
cp authentication/dex/vsphere/oidc/dex-data-values.yaml.example authentication/dex/vsphere/oidc/dex-data-values.yaml
dex-data-values.yaml in a text editor.Update dex-data-values.yaml with information about your management cluster and LDAP server or OIDC provider.
LDAP:
For most deployments, you only need to update <LDAP_HOST>, the baseDN values, and <MGMT_CLUSTER_IP>.
dex.config.ldap.host: Replace <LDAP_HOST> with the IP or DNS address of your LDAP server. If the LDAP server is listening on the default port 636, which is the secured configuration, you do not need to specify the port. If the LDAP server is listening on a different port, provide the address and port of the LDAP server.dex.config.ldap.userSearch.baseDN and dex.config.ldap.groupSearch.baseDN: Update dc=vmware with your company name.userSearch and groupSearch values with parameters that match your LDAP server configuration. If you need to enable LDAP user and group search, see this KB article.dex.config.staticClients: Do not update WORKLOAD_CLUSTER_NAME, WORKLOAD_CLUSTER_IP, or CLIENT_SECRET yet. You will update these later.dns.vsphere.ipAddresses: In the square brackets, replace <MGMT_CLUSTER_IP> with the static virtual IP address that you provided when you deployed the management cluster. Alternatively, add the dns.vsphere.dnsNames option to specify the FQDN of the management cluster.For example:
#@data/values
#@overlay/match-child-defaults missing_ok=True
---
infrastructure_provider: "vsphere"
dex:
config:
connector: ldap
ldap:
host: ldaps.example.com:636
userSearch:
baseDN: 'ou=people,dc=example,dc=com'
filter: '(objectClass=posixAccount)'
username: uid
idAttr: uid
emailAttr: mail
nameAttr: 'givenName'
groupSearch:
baseDN: 'ou=group,dc=example,dc=com'
filter: '(objectClass=posixGroup)'
userAttr: uid
groupAttr: memberUid
nameAttr: cn
#@overlay/replace
staticClients:
- id: WORKLOAD_CLUSTER_NAME
redirectURIs:
- 'https://WORKLOAD_CLUSTER_IP:30166/callback'
name: WORKLOAD_CLUSTER_NAME
secret: CLIENT_SECRET
dns:
vsphere:
#@overlay/replace
ipAddresses: [192.168.10.66]
OIDC:
Update <OIDC_CLIENT_ID>, <OIDC_CLIENT_SECRET>, <OIDC_IDP_URL>, and <MGMT_CLUSTER_IP>.
dex.config.oidc.CLIENT_ID: Replace <OIDC_CLIENT_ID> with the Base64 client_id value that you obtain from your OIDC provider.dex.config.oidc.CLIENT_SECRET: Replace <OIDC_CLIENT_SECRET> with the Base64 secret vallue that you obtain from your OIDC provider.
For example, if your provider is Okta, log in to Okta, create a Web application, and select the Client Credentials options in order to get a client_id and secret.
dex.config.oidc.issuer: Replace <OIDC_IDP_URL> with the IP or DNS address of your OIDC server.dex.config.staticClients: Do not update WORKLOAD_CLUSTER_NAME, WORKLOAD_CLUSTER_IP, or CLIENT_SECRET yet. You will update these later.dns.vsphere.ipAddresses: In the square brackets, replace <MGMT_CLUSTER_IP> with the static virtual IP address that you provided when you deployed the management cluster. Alternatively, add the dns.vsphere.dnsNames option to specify the FQDN of the management cluster.../authentication/dex/values.yaml in a text editor and set dex.config.oidc.insecureEnableGroups: true.For example:
#@data/values
#@overlay/match-child-defaults missing_ok=True
---
infrastructure_provider: "vsphere"
dex:
config:
connector: oidc
oidc:
CLIENT_ID: 0oa5exxxxxxnoiO4x6
CLIENT_SECRET: Ay5cCxxxxxxxxxxxxhhTpli7mV1Nmf3N3J3dKVPM
issuer: https://my-server.okta.com
#@overlay/replace
staticClients:
- id: <WORKLOAD_CLUSTER_NAME>
redirectURIs:
- 'https://<WORKLOAD_CLUSTER_IP>:30166/callback'
name: <WORKLOAD_CLUSTER_NAME>
secret: <CLIENT_SECRET>
dns:
vsphere:
#@overlay/replace
ipAddresses: [192.168.100.44]
dex-data-values.yaml file.Create a Kubernetes secret named dex-data-values with the values that you set in dex-data-values.yaml.
LDAP:
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/ldap/dex-data-values.yaml -n tanzu-system-auth
OIDC:
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/oidc/dex-data-values.yaml -n tanzu-system-auth
You should see the confirmation secret/dex-data-values created.
For the next steps, see Deploy Dex on the Management Cluster.
After you have set up the management cluster, you must update the configuration file that is used when you deploy Dex on a management cluster that is running on Amazon EC2. For management clusters that are running on Amazon EC2, you configure Dex to connect to an OIDC provider.
Make a copy of the dex-data-values.yaml.example file and name it dex-data-values.yaml.
cp authentication/dex/aws/oidc/dex-data-values.yaml.example authentication/dex/aws/oidc/dex-data-values.yaml
dex-data-values.yaml in a text editor.Update dex-data-values.yaml with information about your management cluster and OIDC provider.
dex.config.oidc.CLIENT_ID: Replace <OIDC_CLIENT_ID> with the Base64 client_id value that you obtain from your OIDC provider.dex.config.oidc.CLIENT_SECRET: Replace <OIDC_CLIENT_SECRET> with the Base64 secret vallue that you obtain from your OIDC provider.
For example, if your provider is Okta, log in to Okta, create a Web application, and select the Client Credentials options in order to obtain the Client ID and Client secret.
dex.config.oidc.issuer: Replace <OIDC_PROVIDER_ADDRESS> with the IP or DNS address of your OIDC server.dex.config.staticClients: Do not update GANGWAY_SVC_LB_HOSTNAME, WORKLOAD_CLUSTER_NAME, or CLIENT_SECRET yet. You will update these later.dns.aws.DEX_SVC_LB_HOSTNAME: Do not update sample-elb yet. The Dex service load balancer is created when you deploy the Dex service, so you must update this value after you have deployed Dex.../authentication/dex/values.yaml in a text editor and set dex.config.oidc.insecureEnableGroups: true.For example:
#@data/values
#@overlay/match-child-defaults missing_ok=True
---
infrastructure_provider: "aws"
dex:
config:
connector: oidc
oidc:
CLIENT_ID: 0oa5exxxxxxxxxxiO4x6
CLIENT_SECRET: Ay5cCjxxxxxxxxxxxxxxxxxxxx1Nmf3N3J3dKVPM
issuer: https://my-account.okta.com/
#! Deploy dex first with dummy staticClients....
#@overlay/replace
staticClients:
- id: WORKLOAD_CLUSTER_NAME
redirectURIs:
- 'https://GANGWAY_SVC_LB_HOSTNAME/callback'
name: WORKLOAD_CLUSTER_NAME
secret: CLIENT_SECRET
#! Deploy dex first and update sample-elb with real name once dex service loadbalancer is created, dnsNames is optional.
dns:
aws:
DEX_SVC_LB_HOSTNAME: sample-elb
dnsNames: []
dex-data-values.yaml file.Create a Kubernetes secret named dex-data-values with the values that you set in dex-data-values.yaml.
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/aws/oidc/dex-data-values.yaml -n tanzu-system-auth
You should see the confirmation secret/dex-data-values created.
For the next steps, see Deploy Dex on the Management Cluster.
After you have set up the management cluster, you must update the configuration file that is used when you deploy Dex on a management cluster that is running on Azure. For management clusters that are running on Azure, you configure Dex to connect to a Secure LDAP server.
Make a copy of the dex-data-values.yaml.example file and name it dex-data-values.yaml.
cp authentication/dex/azure/ldap/dex-data-values.yaml.example authentication/dex/azure/ldap/dex-data-values.yaml
dex-data-values.yaml in a text editor.Update dex-data-values.yaml with information about your management cluster and Secure LDAP server.
For most deployments, you only need to update <LDAP_HOST> and the baseDN values.
dex.config.ldap.host: Replace <LDAP_HOST> with the IP or DNS address of your LDAP server. If the LDAP server is listening on the default port 636, which is the secured configuration, you do not need to specify the port. If the LDAP server is listening on a different port, provide the address and port of the LDAP server.dex.config.ldap.userSearch.baseDN and dex.config.ldap.groupSearch.baseDN: Update dc=vmware with your company name.userSearch and groupSearch values with parameters that match your LDAP server configuration. If you need to enable LDAP user and group search, see this KB article.dex.config.staticClients: Do not update WORKLOAD_CLUSTER_NAME, WORKLOAD_CLUSTER_IP, or CLIENT_SECRET yet. You will update these later.dns.azure.DEX_SVC_LB_HOSTNAME: Do not update sample-elb yet. The Dex service load balancer is created when you deploy the Dex service, so you must update this value after you have deployed Dex.For example:
#@data/values
#@overlay/match-child-defaults missing_ok=True
---
infrastructure_provider: "azure"
dex:
config:
connector: ldap
ldap:
host: host: ldaps.example.com:636
userSearch:
baseDN: 'ou=people,dc=example,dc=com'
filter: '(objectClass=person)'
username: userPrincipalName
idAttr: DN
emailAttr: userPrincipalName
nameAttr: 'cn'
groupSearch:
baseDN: 'ou=group,dc=example,dc=com'
filter: '(objectClass=group)'
userAttr: DN
groupAttr: "member:1.2.840.113556.1.4.1941:"
nameAttr: cn
#@overlay/replace
staticClients:
- id: WORKLOAD_CLUSTER_NAME
redirectURIs:
- 'https://WORKLOAD_CLUSTER_IP:30166/callback'
name: WORKLOAD_CLUSTER_NAME
secret: CLIENT_SECRET
dns:
azure:
DEX_SVC_LB_HOSTNAME: sample-lb
dnsNames: []
dex-data-values.yaml file.Create a Kubernetes secret named dex-data-values with the values that you set in dex-data-values.yaml.
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/azure/ldap/dex-data-values.yaml -n tanzu-system-auth
You should see the confirmation secret/dex-data-values created.
For the next steps, see Deploy Dex on the Management Cluster.
After you have prepared the management cluster and updated the appropriate configuration file for your platform, you can deploy Dex on the management cluster.
This procedure applies to all management clusters, running on vSphere, Amazon EC2, and Azure.
Deploy the Dex extension.
kubectl apply -f authentication/dex/dex-extension.yaml
You should see the confirmation extension.clusters.tmc.cloud.vmware.com/dex created.
View the status of the Dex extension.
kubectl get extension dex -n tanzu-system-auth
You should see information about the Dex extension.
NAME STATE HEALTH VERSION
dex 3
View the status of the Dex service itself.
kubectl get app dex -n tanzu-system-auth
The status of the Dex app should show Reconcile Succeeded when Dex has deployed successfully.
NAME DESCRIPTION SINCE-DEPLOY AGE
dex Reconcile Succeeded 53m
If the status is not Reconcile Succeeded, view the full status details of the Dex service.
Viewing the full status can help you to troubleshoot the problem.
kubectl get app dex -n tanzu-system-auth -o yaml
Check that the Dex service is running by listing all of the pods that are running in the management cluster.
kubectl get pods -A
In the tanzu-system-auth namespace, you should see the dex service running in a pod with a name similar to dex-56687dcdbc-qjkrm.
NAMESPACE NAME READY STATUS RESTARTS AGE
[...]
tanzu-system-auth dex-56687dcdbc-qjkrm 1/1 Running 0 6m11s
vmware-system-tmc extension-manager-d7cc7fcbb-t77m7 1/1 Running 0 13m
vmware-system-tmc kapp-controller-7c98dff676-lbt6p 1/1 Running 0 11m
If you deployed Dex on a management cluster that is running on Amazon EC2 or Azure, you have additional steps to perform. See Obtain the Dex Load Balancer for Amazon EC2 or Obtain the Dex Load Balancer for Azure below.
If you deployed Dex on a management cluster that is running on vSphere, your Dex deployment is complete. Next, you must deploy a Tanzu Kubernetes cluster with an embedded OIDC endpoint that can connect to your LDAP server or OIDC provider, via the Dex service that is running in the management cluster. Go to Deploy an Authentication-Enabled Tanzu Kubernetes Cluster.
If you deployed Dex on a management cluster that is running on Amazon EC2, you must perform additional steps to obtain the address of the Dex service load balancer that is running in the management cluster. You then need to update the Dex configuration to use it.
Obtain the host name of the Dex service load balancer.
kubectl get svc dexsvc -n tanzu-system-auth -o jsonpath='{.status.loadBalancer.ingress[0].hostname}'
Open authentication/dex/aws/oidc/dex-data-values.yaml in a text editor and update the DEX_SVC_LB_HOSTNAME parameter with the address that you obtained in the preceding step.
Add the load balancer address in the dns.aws.DEX_SVC_LB_HOSTNAME parameter. The address will resemble the following:
dns:
aws:
DEX_SVC_LB_HOSTNAME: a708e794809d54af699834323c33a0a-463181988.us-west-2.elb.amazonaws.com
Update the dex-data-values secret to include the load balancer address.
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/aws/oidc/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
You should see the confirmation secret/dex-data-values replaced. The changes should show up in five minutes or less. This is handled by the Kapp controller, which synchronizes every five minutes.
Now, the Dex service is ready and you can deploy a Tanzu Kubernetes cluster with an embedded OIDC endpoint that can connect to your LDAP server or OIDC provider via the Dex service. Go to Deploy an Authentication-Enabled Tanzu Kubernetes Cluster.
If you deployed Dex on a management cluster that is running on Azure, you must perform additional steps to obtain the address of the Dex service load balancer that is running in the management cluster. You then need to update the Dex configuration to use it.
Obtain the external IP address of the Dex service load balancer.
kubectl get svc dexsvc -n tanzu-system-auth -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
Open authentication/dex/azure/ldap/dex-data-values.yaml in a text editor and update the DEX_SVC_LB_HOSTNAME parameter with the address that you obtained in the preceding step.
Add the load balancer address in the dns.azure.DEX_SVC_LB_HOSTNAME parameter. The address will resemble the following:
dns:
azure:
DEX_SVC_LB_HOSTNAME: 20.54.226.42
Update the dex-data-values secret to include the load balancer address.
This command assumes that you are running it from tkg-extensions-v1.2.0+vmware.1/extensions.
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/azure/ldap/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
You should see the confirmation secret/dex-data-values replaced.
Redeploy the Dex extension.
kubectl apply -f authentication/dex/dex-extension.yaml
You should see the confirmation extension.clusters.tmc.cloud.vmware.com/dex configured.
Now, the Dex service is ready and you can deploy a Tanzu Kubernetes cluster with an embedded OIDC endpoint that can connect to your LDAP server or OIDC provider via the Dex service. Go to Deploy an Authentication-Enabled Tanzu Kubernetes Cluster.
If you need to make changes to the configuration of the Dex extension, you can update the extension.
Perform the steps in this procedure if you modify either of the extensions/authentication/dex/<INFRA_PROVIDER>/<AUTH>/dex-data-values.yaml or authentication/dex/values.yaml files after the initial deployment of Dex.
Obtain the Dex data values from the Kubernetes secret.
kubectl get secret dex-data-values -n tanzu-system-auth -o 'go-template={{ index .data "values.yaml" }}' | base64 -d > dex-data-values.yaml
extensions/authentication/dex/<INFRA_PROVIDER>/<AUTH>/dex-data-values.yaml or authentication/dex/values.yaml files to update your configuration.Update the Kubernetes secret.
This command assumes that you are running it from tkg-extensions-v1.2.0+vmware.1/extensions.
vSphere with LDAP:
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/ldap/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
vSphere with OIDC:
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/vsphere/oidc/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
Amazon EC2:
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/aws/oidc/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
Azure:
kubectl create secret generic dex-data-values --from-file=values.yaml=authentication/dex/azure/ldap/dex-data-values.yaml -n tanzu-system-auth -o yaml --dry-run | kubectl replace -f-
Note that the final - on the kubectl replace command above is necessary to instruct kubectl to accept the input being piped to it from the kubectl create secret command.
The Dex extension will be reconciled using the new values you just added. The changes should show up in five minutes or less. This is handled by the Kapp controller, which synchronizes every five minutes.
