Troubleshooting ESX GI Module (MUX)

If all virtual machines on an ESXi host are not working with Guest Introspection, or there are alarms on a particular host regarding communication to the GI SVA, then it could be a problem with the ESX GI Module on the ESXi host.

ESX GI Module (MUX)

Check to see if the service is running on the ESXi host by running the # /etc/init.d/vShield-Endpoint-Mux status command:

For example:
```
# /etc/init.d/vShield-Endpoint-Mux status
         vShield-Endpoint-Mux is running
```
If you see that the service is not running, restart it or start it with this command:

/etc/init.d/vShield-Endpoint-Mux start

or

/etc/init.d/vShield-Endpoint-Mux restart

Note that it is safe to restart this service during production hours as it does not have any great impact, and restarts in a couple of seconds.
To get a better idea of what the ESX GI Module is doing or check the communication status, you can check the logs on the ESXi host. ESX GI Module logs are written to the host /var/log/syslog file. This is also included in the ESXi host support logs.
For more information, see Collecting diagnostic information for ESX/ESXi hosts and vCenter Server using the vSphere Web Client (2032892)
The default logging option for ESX GI Module is info and can be raised to debug to gather more information:
For more information, see Guest Introspection Logs.
Re-installing the ESX GI Module module can also fix many issues, especially if the wrong version is installed, or the ESXi host was brought into the environment which previously had Endpoint installed on it. This needs to be removed and re-installed.
To remove the VIB, run this command: esxcli software vib remove -n epsec-mux
If you run into issues with the VIB installation, check the /var/log/esxupdate.log file on the ESXi host. This log shows the most clear information as to why the driver did not successfully get installed. This is a common issue for ESX GI Module installation issues. For more information, see Installing NSX Guest Introspection services (ESX GI Module VIB) on the ESXi host fails in VMware NSX for vSphere 6.x (2135278).

To check for a corrupt ESXi image look for a message similar to this:

esxupdate: esxupdate: ERROR: Installation Error:
 (None, 'No image profile is found on the host or image profile is empty.
 An image profile is required to install or remove VIBs. To install an image profile,
 use the esxcli image profile install command.')

To verify that the image is corrupted run the command cd /vmfs/volumes on the ESXi host.
1. Search for the imgdb.tgz file by running this command: find * | grep imgdb.tgz.
  This command normally results in two matches. For example:
  
  0ca01e7f-cc1ea1af-bda0-1fe646c5ceea/imgdb.tgz or edbf587b-da2add08-3185-3113649d5262/imgdb.tgz
2. On each match, run this command: ls -l match_result
  
  For example:
```
> ls -l 0ca01e7f-cc1ea1af-bda0-1fe646c5ceea/imgdb.tgz -rwx------  
 1 root root  26393 Jul 20 19:28 0ca01e7f-cc1ea1af-bda0-1fe646c5ceea/imgdb.tgz
   > ls -l edbf587b-da2add08-3185-3113649d5262/imgdb.tgz -rwx------ 
  1 root root   93 Jul 19 17:32 edbf587b-da2add08-3185-3113649d5262/imgdb.tgz
```
  The default size for the imgdb.tgz file is far greater than the other file or if one of the files is only a couple of bytes, it indicates that the file is corrupt. The only supported way to resolve this is to re-install ESXi for that particular ESXi host.