If you’re having trouble registering a Supervisor Cluster running in vSphere with Tanzu with Tanzu Mission Control, there are a number of steps you can take to identify the root cause and fix the problem.
Problem
You might suspect you’re having trouble registering your Supervisor Cluster if:
When registering your Supervisor Cluster, the verification step fails.
If you’re registering a Supervisor Cluster through the CLI, the operation times out.
If you’re registering a Supervisor Cluster through the API, the status of the cluster remains pending for longer than 7 minutes.
Cause
Cause
There are a few common reasons you might have trouble registering your Supervisor Cluster with Tanzu Mission Control. You can follow the solution described to identify which of these causes is the issue and help you correct it.
You might need to rerun the registration step if the agent installation failed. Common reasons for failure:
You might have tried to attach instead of register the Supervisor Cluster.
You might have tried to register a Supervisor Cluster using the Tanzu Kubernetes Grid procedure and seen a message like “no such resource found.”
You might have a resource scheduling issue or permission issue preventing the Tanzu Mission Control agent from installing successfully.
You might have a problem with connectivity or proxy configuration.
Solution
- Check the format of the registration URL to verify you’re using the procedure to register a vSphere with Tanzu. You should see
source=registration and type=tkgs. For example:
https://myorgname.tmc.cloud.vmware.com/installer?id=1234&source=registration&type=tkgs.
- Check if the problem is your access rights or network connection to the by running a simple kubectl command like get namespaces.
If you cannot access the cluster, contact your kubernetes administrator.
- Verify that the tmc namespace exists. The tmc namespace is in the form of
svc-tmc-N.
kubectl get namespaces | grep svc-tmc
svc-tmc-c8 Active 30h
If a tmc namespace doesn't exist, you might be registering an unsupported version of Tanzu Kubernetes Grid.
- Check if the problem is the agent installation failure. Reinstall the agent if the installation failed.
- Verify if the agent install succeeded by checking if AgentInstall exists.
kubectl get agentinstall -n svc-tmc-aN
NAME AGE
tmc-agent-installer-config 4s
- Check the AgentInstall status field to understand if there were any issues with the installation and how to correct them.
kubectl describe agentinstall -n svc-tmc-aN tmc-agent-installer-config
Name: tmc-agent-installer-config
Namespace: svc-tmc-aN
...
Status:
Message: Installation inprogress
Retry Count: 5
Status: INSTALLATION_IN_PROGRESS
Events: <none>
- Check if there was an issue applying the agent configuration.
kubectl get configmap -n svc-tmc-aN stack-config tmc-agent-installer-config
NAME DATA AGE
stack-config 9 9d
If the agent configuration doesn’t exist:
- Investigate the agent pods to locate the source of the problem.
- Check the status of the agent pods.
kubectl get pods -n svc-tmc-aN -l "tmc-extension-name in (extension-manager,extension-updater,agent-updater)"
NAME READY STATUS RESTARTS AGE
agent-updater-cc9c67b4d-sp6q9 0/1 CLBO 0 3m
agentupdater-workload-27478576-8rxvk 0/1 CLBO 0 60s
extension-manager-85f9cf5497-rk8ms 0/1 CLBO 0 3m
extension-updater-6c4c95c5bf-9wl6p 0/1 CLBO 0 3m
- If the pods are in a CLBO (crash loop backoff), retry the registration procedure and verify you’re using the vSphere with Tanzu registration procedure instead of the Tanzu Kubernetes Grid registration procedure.
- If any of the pods are in a failed state, fetch the logs for the failed pod to learn more information about the problem. For example:
kubectl logs -l "tmc-extension-name=extension-updater" -n svc-tmc-aN
For help solving pod failures, see Common Log Errors and Solutions for Pod Failure in the Examples section of this page.
- If the pods have not been scheduled (they appear as “pending” or not at all), inspect the deployments of extension-manager, extension-updater and agent-updater to find any deployments that are not in a ready state.
kubectl get deployments -n svc-tmc-aN -l "tmc-extension-name in (extension-manager,extension-updater,agent-updater)"
NAME READY UP-TO-DATE AVAILABLE AGE
agent-updater 1/11 1 3m
extension-manager 1/11 1 3m
extension-updater 1/11 1 3m
- If any of the deployments show as 0/1 , describe the deployment to understand how to correct the issue. For example:
kubectl describe deployment extension-updater -n svc-tmc-aN
These kubernetes errors are usually resource related. For example, you might get <output>OutOfDisk.
Ask your kubernetes administrator to solve these types of errors by expanding resources or relaxing policies.
- If the deployments are all showing 1/1 READY, look for Events messages in the replicasets to understand how to correct the issue.
- Look for replicasets that are not in the ready status. For example:
kubectl get replicaset -n svc-tmc-aN -l "tmc-extension-name in (extension-manager,extension-updater,agent-updater)"
- Inspect any replicasets that are not in ready status and check the events messages in the replicasets to understand how to correct the issue. For example:
kubectl describe replicaset -n vmware-system-tmc extension-updater-6c4c95c5bf
The Events messages should contain enough information for you to solve the issue.
Example
The following tables contain examples of common errors and how to solve them.
Table 1.
How to Solve Common AgentInstall Status Messages
Status |
Message |
Solutions |
INSTALLATION_IN_PROGRESS |
Installation in progress |
Wait until the installation completes. |
INSTALL_FAILED |
Registration link domain name verification failed |
Check the registration link. Make sure you see something like, https://myorgname.tmc.cloud.vmware.com/installer?id=1234&source=registration&type=tkgs. |
INSTALL_FAILED |
Registration link is not an HTTPS link |
Check the registration link. Make sure you see something like, https://myorgname.tmc.cloud.vmware.com/installer?id=1234&source=registration&type=tkgs. |
INSTALL_FAILED |
Get proxy options |
Verify the proxy configured in the cluster. |
INSTALL_FAILED |
Download manifest from link |
Make sure you downloaded and prepared the YAML from the Tanzu Mission Control Console as described in Complete the Registration of a Supervisor Cluster in vSphere with Tanzu. You might need to retry your registration procedure from the beginning. |
INSTALL_FAILED |
Parse manifest |
Make sure you downloaded and prepared the YAML from the Tanzu Mission Control Console as described in Complete the Registration of a Supervisor Cluster in vSphere with Tanzu. You might need to retry your registration procedure from the beginning. |
INSTALL_FAILED |
Apply manifest |
It's likely there aren't enough resources available for the agent to install successfully. Contact your vSphere administrator for help. |
Table 2.
Common Log Errors and Solutions for Pod Failure
Application |
Log |
Solutions |
Extension-Updater |
Failed to discover cluster metadata |
Make sure your cluster is CNCF conformant. You can check the CNCF site for a list of Platform Certified Kubernetes - Distribution Providers. |
Extension-Updater |
Failed to set up manager:get Tanzu Mission Control connection |
Make sure the cluster has internet connectivity. -
If you’re behind a firewall, set up or validate the configuration of your proxy. See Connecting through a Proxy for more information.
|
Extension-Updater |
Pre-flight checks failed Or something like tmc.extension-flag.register.tkg.version.block: 1.4.0 |
Make sure your cluster is using a version of kubernetes that is supported by Tanzu Mission Control. |
What to do next
If you can’t solve the issue on your own, you can create a log bundle to share with support. Run the following commands against your management cluster:
Get the logs of the extension-updater kubectl logs -n svc-tmc-aN -l "tmc-extension-name=extension-updater"
Get the logs of the extension-manager <userinput> kubectl logs -n svc-tmc-aN -l "tmc-extension-name=extension-manager”
Save the logs and create a bundle to share with support.
See How to Get Support if you need more information about contacting them.
