This topic describes how to manage VMware Tanzu Kubernetes Grid Integrated Edition after changing a BOSH Director or Tanzu Kubernetes Grid Integrated Edition service account password.
When you installed Tanzu Kubernetes Grid Integrated Edition you created two service accounts:
You must update a tile’s copy of a service account password after changing the password on your network.
To update BOSH Director with a new BOSH/Ops Manager Service Account password, perform the following steps:
To update Tanzu Kubernetes Grid Integrated Edition with a new Master Node Service Account password, perform the following steps:
After updating an Ops Manager tile’s service account password you must also deploy the new password.
To deploy a new password to BOSH Director and Tanzu Kubernetes Grid Integrated Edition, perform the following steps:
Note: The Update all clusters errand must be enabled to update the Kubernetes cloud provider password stored in Kubernetes clusters.
If you are on vSphere or vSphere with NSX-T only, you also configured the NSX Manager Account and password when you installed Tanzu Kubernetes Grid Integrated Edition. This service account is configured in the BOSH Director tile.
After changing the password on your network, you must also update the BOSH Director tile’s copy of the NSX Manager Account password.
To update the BOSH Director with the new NSX Manager password, perform the following steps:
Symptom
Your cluster control plane node does not authenticate with your vCenter even though the cluster’s /var/vcap/jobs/kube-controller-manager/config/cloud-provider.ini
file includes the correct vCenter credentials.
You see errors similar to the following in your logs:
Service account errors in the TKGI logs:
error ... Failed to authenticate user ...
Authentication errors in the BOSH tasks logs:
WARN -- [req_id ...]: Error running method 'Login'. Failed with message 'Cannot complete login due to an incorrect user name or password.'.
Rescued Unknown: Cannot complete login due to an incorrect user name or password.. backtrace: /var/vcap/data/packages/vsphere_cpi/.../lib/cloud/vsphere/retryer.rb:13:in `try'
Additionally, the cluster control plane repeatedly attempts to authenticate with vCenter, degrading the state of the STS service, and your vCenter becomes inaccessible.
Solution
To resolve this issue:
Confirm that the correct credentials are included in your cluster’s /var/vcap/jobs/kube-controller-manager/config/cloud-provider.ini
file. If the credentials are incorrect, update the configuration with the valid credentials.
If the configured credentials are correct, review your cluster’s actual vCenter authentication credentials:
bosh ssh -d DEPLOYMENT-NAME master/0 "sudo cat /var/vcap/jobs/csi-controller/config/csi-vsphere.conf"
Where DEPLOYMENT-NAME
is the BOSH deployment name for the cluster that does not authenticate.
If the configured credentials and actual credentials are different, you must upgrade the cluster.