Replace Existing VMCA-Signed Certificates with New VMCA-Signed Certificates Using the CLI

If the VMware Certificate Authority (VMCA) root certificate expires in the near future, or if you want to replace it for other reasons, you can use the CLI to generate a new root certificate and add it to the VMware Directory Service. You can then generate new machine SSL certificates and solution user certificates using the new root certificate.

Use the vSphere Certificate Manager utility to replace certificates for most cases.

If you need fine-grained control, this scenario gives detailed step-by-step instructions for replacing the complete set of certificates using CLI commands. You can instead replace only individual certificates using the procedure in the corresponding task.

Prerequisites

Only [email protected] or other users in the CAAdmins group can perform certificate management tasks. See Add Members to a vCenter Single Sign-On Group.

Generate a New VMCA-Signed Root Certificate Using the CLI

You can generate new VMCA-signed certificates with the certool CLI and publish the certificates to vmdir.

Procedure

1. On the vCenter Server, generate a new self-signed certificate and private key.

   certool --genselfcacert --outprivkey <key_file_path> --outcert <cert_file_path> --config <config_file>

2. Replace the existing root certificate with the new certificate.

   certool --rootca --cert <cert_file_path> --privkey <key_file_path>

   The command generates the certificate, adds it to vmdir, and adds it to VECS.
3. Stop all services and start the services that handle certificate creation, propagation, and storage.

   service-control --stop --all
   service-control --start vmafdd
   service-control --start vmdird
   service-control --start vmcad
   

4. (Optional) Publish the new root certificate to vmdir.

   dir-cli trustedcert publish --cert newRoot.crt
   

   The command updates all instances of vmdir immediately. If you do not run the command, propagation of the new certificate to all nodes might take a while.
5. Restart all services.

   service-control --start --all

Example: Generate a New VMCA-Signed Root Certificate

The following example shows all the steps for verifying the current root CA information, and for regenerating the root certificate.

1. (Optional) On the vCenter Server, list the VMCA root certificate to make sure it is in the certificate store.

   /usr/lib/vmware-vmca/bin/certool --getrootca 

   The output looks similar to this:

   output:
   Certificate:
       Data:
           Version: 3 (0x2)
           Serial Number:
               cf:2d:ff:49:88:50:e5:af
       ...
   

2. (Optional) List the VECS TRUSTED_ROOTS store and compare the certificate serial number there with the output from Step 1.

   /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store TRUSTED_ROOTS --text
   

   In the simplest case with only one root certificate, the output looks like this:

   Number of entries in store :    1
   Alias : 960d43f31eb95211ba3a2487ac840645a02894bd
   Entry type :    Trusted Cert
   Certificate:
       Data:
           Version: 3 (0x2)
           Serial Number:
               cf:2d:ff:49:88:50:e5:af

3. Generate a new VMCA root certificate. The command adds the certificate to the TRUSTED_ROOTS store in VECS and in vmdir (VMware Directory Service).

   /usr/lib/vmware-vmca/bin/certool --selfca --config=/usr/lib/vmware-vmca/share/config/certool.cfg

Replace Machine SSL Certificates with VMCA-Signed Certificates Using the CLI

After you generate a new VMCA-signed root certificate, you can use the vecs-cli command to replace all machine SSL certificates in your environment.

Each machine must have a machine SSL certificate for secure communication with other services. When multiple vCenter Server instances are connected in Enhanced Linked Mode configuration, you must run the Machine SSL certificate generation commands on each node.

Prerequisites

Be prepared to stop all services and to start the services that handle certificate propagation and storage.

Procedure

1. Make one copy of certool.cfg for each machine that needs a new certificate.
   You can find the certool.cfg file in the /usr/lib/vmware-vmca/share/config/ directory.
2. Edit the custom configuration file for each machine to include that machine's FQDN.
   Run NSLookup against the machine’s IP address to see the DNS listing of the name, and use that name for the Hostname field in the file.
3. Generate a public/private key file pair and a certificate for each file, passing in the configuration file that you just customized.
   For example:

   certool --genkey --privkey=machine1.priv --pubkey=machine1.pub
   certool --gencert --privkey=machine1.priv --cert machine1.crt --Name=Machine1_Cert --config machine1.cfg

4. Stop all services and start the services that handle certificate creation, propagation, and storage.

   service-control --stop --all
   service-control --start vmafdd
   service-control --start vmdird
   service-control --start vmcad
   

5. Add the new certificate to VECS.
   All machines need the new certificate in the local certificate store to communicate over SSL. You first delete the existing entry, then add the new entry.

   vecs-cli entry delete --store MACHINE_SSL_CERT --alias __MACHINE_CERT  
   vecs-cli entry create --store MACHINE_SSL_CERT --alias __MACHINE_CERT --cert machine1.crt --key machine1.priv

6. Restart all services.

   service-control --start --all

Example: Replacing Machine Certificates with VMCA-Signed Certificates

1. Create a configuration file for the SSL certificate and save it as ssl-config.cfg in the current directory.

   Country = US
   Name = vmca-<FQDN-example>
   Organization = <my_company>
   OrgUnit = <my_company Engineering>
   State = <my_state> 
   Locality = <mytown>
   Hostname = <FQDN>

2. Generate a key pair for the machine SSL certificate. In a deployment of multiple vCenter Server instances connected in Enhanced Linked Mode configuration, run this command on each vCenter Server node.

   /usr/lib/vmware-vmca/bin/certool --genkey --privkey=ssl-key.priv --pubkey=ssl-key.pub
   

   The ssl-key.priv and ssl-key.pub files are created in the current directory.

3. Generate the new machine SSL certificate. This certificate is signed by VMCA. If you replaced the VMCA root certificate with custom certificate, VMCA signs all certificates with the full chain.

   /usr/lib/vmware-vmca/bin/certool --gencert --cert=new-vmca-ssl.crt --privkey=ssl-key.priv --config=ssl-config.cfg

   The new-vmca-ssl.crt file is created in the current directory.

4. (Optional) List the content of VECS.

   /usr/lib/vmware-vmafd/bin/vecs-cli store list

   • Sample output on vCenter Server:

     output (on vCenter):
     MACHINE_SSL_CERT
     TRUSTED_ROOTS
     TRUSTED_ROOT_CRLS
     machine
     vsphere-webclient
     vpxd
     vpxd-extension
     hvc
     data-encipherment
     APPLMGMT_PASSWORD
     SMS
     wcp
     KMS_ENCRYPTION

5. Replace the Machine SSL certificate in VECS with the new Machine SSL certificate. The --store and --alias values have to exactly match with the default names.
   • On each vCenter Server, run the following commands to update the Machine SSL certificate in the MACHINE_SSL_CERT store. You must update the certificate for each machine separately because each has a different FQDN.

     /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store MACHINE_SSL_CERT --alias __MACHINE_CERT
     /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store MACHINE_SSL_CERT --alias __MACHINE_CERT --cert new-vmca-ssl.crt --key ssl-key.priv
     

What to do next

You can also replace the certificates for your ESXi hosts. See the vSphere Security publication.

Replace Solution User Certificates with New VMCA-Signed Certificates Using the CLI

After you replace the machine SSL certificates, you can use the dir-cli command to replace all solution user certificates. Solution user certificates must be valid, that is, not expired, but none of the other information in the certificate is used by the certificate infrastructure.

Many VMware customers do not replace solution user certificates. They replace only the machine SSL certificates with custom certificates. This hybrid approach satisfies the requirements of their security teams.

• Certificates either sit behind a proxy, or they are custom certificates.
• No intermediate CAs are used.

You replace the machine solution user certificate and the solution user certificate on each vCenter Server system.

Note: When you list solution user certificates in large deployments, the output of /usr/lib/vmware-vmafd/bin/dir-cli list includes all solution users from all nodes. Run /usr/lib/vmware-vmafd/bin/vmafd-cli get-machine-id --server-name localhost to find the local machine ID for each host. Each solution user name includes the machine ID.

Prerequisites

Be prepared to stop all services and to start the services that handle certificate propagation and storage.

Procedure

1. Make one copy of certool.cfg, remove the Name, IP address, DNS name, and email fields, and rename the file, for example, to sol_usr.cfg.
   You can name the certificates from the command line as part of generation. The other information is not needed for solution users. If you leave the default information, the certificates that are generated are potentially confusing.
2. Generate a public/private key file pair and a certificate for each solution user, passing in the configuration file that you just customized.
   For example:

   /usr/lib/vmware-vmca/bin/certool --genkey --privkey=vpxd.priv --pubkey=vpxd.pub 
   /usr/lib/vmware-vmca/bin/certool --gencert --privkey=vpxd.priv --cert vpxd.crt --Name=VPXD_1 --config sol_usr.cfg

3. Find the name for each solution user.

   /usr/lib/vmware-vmafd/bin/dir-cli service list 
   

   You can use the unique ID that is returned when you replace the certificates. The input and output might look as follows.

   /usr/lib/vmware-vmafd/bin/dir-cli service list
   Enter password for [email protected]:
   1. machine-623bef28-0311-436e-b21f-6e0d39aa5179
   2. vsphere-webclient-623bef28-0311-436e-b21f-6e0d39aa5179
   3. vpxd-623bef28-0311-436e-b21f-6e0d39aa5179
   4. vpxd-extension-623bef28-0311-436e-b21f-6e0d39aa5179
   5. hvc-623bef28-0311-436e-b21f-6e0d39aa5179
   6. wcp-1cbe0a40-e4ce-4378-b5e7-9460e2b8200e

   In a deployment of multiple vCenter Server instances connected in Enhanced Linked Mode configuration, the output of /usr/lib/vmware-vmafd/bin/dir-cli service list includes all solution users from all nodes. Run /usr/lib/vmware-vmafd/bin/vmafd-cli get-machine-id --server-name localhost to find the local machine ID for each host. Each solution user name includes the machine ID.
4. Stop all services and start the services that handle certificate creation, propagation, and storage.

   service-control --stop --all
   service-control --start vmafdd
   service-control --start vmdird
   service-control --start vmcad
   

5. For each solution user, replace the existing certificate in vmdir and then in VECS.
   The following example shows how to replace the certificates for the vpxd service.

   /usr/lib/vmware-vmafd/bin/dir-cli service update --name <vpxd-xxxx-xxx-7c7b769cd9f4> --cert ./vpxd.crt
   /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store vpxd --alias vpxd
   /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store vpxd --alias vpxd --cert vpxd.crt --key vpxd.priv
   

   Note: Solution users cannot authenticate to vCenter Single Sign-On if you do not replace the certificate in vmdir.
6. Restart all services.

   service-control --start --all

Example: Using VMCA-Signed Solution User Certificates

1. Generate a public/private key pair for each solution user on each vCenter Server node in an Enhanced Linked Mode configuration. That includes a pair for the machine solution and a pair for each additional solution user (vpxd, vpxd-extension, vsphere-webclient, wcp).
   1. Generate a key pair for the machine solution user.

      /usr/lib/vmware-vmca/bin/certool --genkey --privkey=machine-key.priv --pubkey=machine-key.pub
      

   2. Generate a key pair for the vpxd solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --genkey --privkey=vpxd-key.priv --pubkey=vpxd-key.pub

   3. Generate a key pair for the vpxd-extension solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --genkey --privkey=vpxd-extension-key.priv --pubkey=vpxd-extension-key.pub

   4. Generate a key pair for the vsphere-webclient solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --genkey --privkey=vsphere-webclient-key.priv --pubkey=vsphere-webclient-key.pub

   5. Generate a key pair for the wcp solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --genkey --privkey=wcp-key.priv --pubkey=wcp-key.pub

2. Generate solution user certificates that are signed by the new VMCA root certificate for the machine solution user and for each additional solution user (vpxd, vpxd-extension, vsphere-webclient, wcp) on each vCenter Server node.
   Note: The --Name parameter has to be unique. Including the name of the solution user store name makes it easy to see which certificate maps to which solution user. The example includes the name, for example vpxd or vpxd-extension in each case.
   1. Make one copy of the /usr/lib/vmware-vmca/share/config/certool.cfg file, then modify or remove the Name, IP address, DNS name, and email fields as required, and rename the file, for example, to sol_usr.cfg.
   2. Generate a certificate for the machine solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --gencert --cert=new-machine.crt --privkey=machine-key.priv --Name=machine --config sol_usr.cfg

   3. Generate a certificate for the vpxd solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --gencert --cert=new-vpxd.crt --privkey=vpxd-key.priv --Name=vpxd --config sol_usr.cfg
      

   4. Generate a certificate for the vpxd-extensions solution user on each node.

      /usr/lib/vmware-vmca/bin/certool --gencert --cert=new-vpxd-extension.crt --privkey=vpxd-extension-key.priv --Name=vpxd-extension --config sol_usr.cfg

   5. Generate a certificate for the vsphere-webclient solution user on each node by running the following command.

      /usr/lib/vmware-vmca/bin/certool --gencert --cert=new-vsphere-webclient.crt --privkey=vsphere-webclient-key.priv --Name=vsphere-webclient --config sol_usr.cfg

   6. Generate a certificate for the wcp solution user on each node by running the following command.

      /usr/lib/vmware-vmca/bin/certool --gencert --cert=new-wcp.crt --privkey=wcp-key.priv --Name=wcp --config sol_usr.cfg

3. Replace the solution user certificates in VECS with the new solution user certificates.
   Note: The --store and --alias parameters have to exactly match the default names for services.
   1. Replace the machine solution user certificate on each node:

      /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store machine --alias machine
      /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store machine --alias machine --cert new-machine.crt --key machine-key.priv
      

   2. Replace the vpxd solution user certificate on each node.

      /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store vpxd --alias vpxd
      /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store vpxd --alias vpxd --cert new-vpxd.crt --key vpxd-key.priv
      

   3. Replace the vpxd-extension solution user certificate on each node.

      /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store vpxd-extension --alias vpxd-extension
      /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store vpxd-extension --alias vpxd-extension --cert new-vpxd-extension.crt --key vpxd-extension-key.priv
      

   4. Replace the vsphere-webclient solution user certificate on each node.

      /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store vsphere-webclient --alias vsphere-webclient
      /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store vsphere-webclient --alias vsphere-webclient --cert new-vsphere-webclient.crt --key vsphere-webclient-key.priv
      

   5. Replace the wcp solution user certificate on each node.

      /usr/lib/vmware-vmafd/bin/vecs-cli entry delete --store wcp --alias wcp
      /usr/lib/vmware-vmafd/bin/vecs-cli entry create --store wcp --alias wcp --cert new-wcp.crt --key wcp-key.priv
      

4. Update VMware Directory Service (vmdir) with the new solution user certificates. You are prompted for a vCenter Single Sign-On administrator password.
   1. Run /usr/lib/vmware-vmafd/bin/dir-cli service list to get the unique service ID suffix for each solution user. You run this command on a vCenter Server system.

      /usr/lib/vmware-vmafd/bin/dir-cli service list
      output:
      1. machine-623bef28-0311-436e-b21f-6e0d39aa5179
      2. vsphere-webclient-623bef28-0311-436e-b21f-6e0d39aa5179
      3. vpxd-623bef28-0311-436e-b21f-6e0d39aa5179
      4. vpxd-extension-623bef28-0311-436e-b21f-6e0d39aa5179
      5. hvc-623bef28-0311-436e-b21f-6e0d39aa5179
      6. wcp-1cbe0a40-e4ce-4378-b5e7-9460e2b8200e

      Note: When you list solution user certificates in large deployments, the output of /usr/lib/vmware-vmafd/bin/dir-cli list includes all solution users from all nodes. Run /usr/lib/vmware-vmafd/bin/vmafd-cli get-machine-id --server-name localhost to find the local machine ID for each host. Each solution user name includes the machine ID.
   2. Replace the machine certificate in vmdir on each vCenter Server node. For example, if machine-6fd7f140-60a9-11e4-9e28-005056895a69 is the machine solution user on the vCenter Server, run this command:

      /usr/lib/vmware-vmafd/bin/dir-cli service update --name machine-6fd7f140-60a9-11e4-9e28-005056895a69 --cert new-machine.crt

   3. Replace the vpxd solution user certificate in vmdir on each node. For example, if vpxd-6fd7f140-60a9-11e4-9e28-005056895a69 is the vpxd solution user ID, run this command:

      /usr/lib/vmware-vmafd/bin/dir-cli service update --name vpxd-6fd7f140-60a9-11e4-9e28-005056895a69 --cert new-vpxd.crt

   4. Replace the vpxd-extension solution user certificate in vmdir on each node. For example, if vpxd-extension-6fd7f140-60a9-11e4-9e28-005056895a69 is the vpxd-extension solution user ID, run this command:

      /usr/lib/vmware-vmafd/bin/dir-cli service update --name vpxd-extension-6fd7f140-60a9-11e4-9e28-005056895a69 --cert new-vpxd-extension.crt
      

   5. Replace the vsphere-webclient solution user certificate on each node. For example, if vsphere-webclient-6fd7f140-60a9-11e4-9e28-005056895a69 is the vsphere-webclient solution user ID, run this command:

      /usr/lib/vmware-vmafd/bin/dir-cli service update --name vsphere-webclient-6fd7f140-60a9-11e4-9e28-005056895a69 --cert new-vsphere-webclient.crt
      

   6. Replace the wcp solution user certificate on each node. For example, if wcp-1cbe0a40-e4ce-4378-b5e7-9460e2b8200e is the wcp solution user ID, run this command:

      /usr/lib/vmware-vmafd/bin/dir-cli service update --name wcp-1cbe0a40-e4ce-4378-b5e7-9460e2b8200e --cert new-wcp.crt
      

What to do next

Restart all services on each vCenter Server node.