The Message Bus is used for NSX Edge communication when ESXi hosts are prepared.
When you encounter issues, the NSX Manager logs might contain entries similar to:
GMT ERROR taskScheduler-6 PublishTask:963 - Failed to configure VSE-vm index 0, vm-id vm-117, edge edge-5. Error: RPC request timed out
This issue occurs if:
- Edge is in a bad state
- Message Bus connection is broken
To diagnose the issue on the Edge:
- To check rmq connectivity, run this command:
nsxedge> show messagebus messages ----------------------- Message bus is enabled cmd conn state : listening init_req : 1 init_resp : 1 init_req_err : 0 ... -
To check vmci connectivity, run this command:
nsxedge> show messagebus forwarder ----------------------- Forwarder Command Channel vmci_conn : up app_client_conn : up vmci_rx : 3649 vmci_tx : 3648 vmci_rx_err : 0 vmci_tx_err : 0 vmci_closed_by_peer: 8 vmci_tx_no_socket : 0 app_rx : 3648 app_tx : 3649 app_rx_err : 0 app_tx_err : 0 app_conn_req : 1 app_closed_by_peer : 0 app_tx_no_socket : 0 ----------------------- Forwarder Event Channel vmci_conn : up app_client_conn : up vmci_rx : 1143 vmci_tx : 13924 vmci_rx_err : 0 vmci_tx_err : 0 vmci_closed_by_peer: 0 vmci_tx_no_socket : 0 app_rx : 13924 app_tx : 1143 app_rx_err : 0 app_tx_err : 0 app_conn_req : 1 app_closed_by_peer : 0 app_tx_no_socket : 0 ----------------------- cli_rx : 1 cli_tx : 1 cli_tx_err : 0 counters_reset : 0In the example, the output vmci_closed_by_peer: 8 indicates the number of times the connection has been closed by the host agent. If this number is increasing and vmci conn is down, the host agent cannot connect to the RMQ broker. In show log follow, look for repeated errors in the Edge logs: VmciProxy: [daemon.debug] VMCI Socket is closed by peer
To diagnose the issue on the ESXi host:
- To check if the ESXi host connects to the RMQ broker, run this command:
esxcli network ip connection list | grep 5671 tcp 0 0 10.32.43.4:43329 10.32.43.230:5671 ESTABLISHED 35854 newreno vsfwd tcp 0 0 10.32.43.4:52667 10.32.43.230:5671 ESTABLISHED 35854 newreno vsfwd tcp 0 0 10.32.43.4:20808 10.32.43.230:5671 ESTABLISHED 35847 newreno vsfwd tcp 0 0 10.32.43.4:12486 10.32.43.230:5671 ESTABLISHED 35847 newreno vsfwd
