top of page
  • Writer's pictureRichard

VMware HCX Troubleshooting

  • Log Locations:

  • HCX: /common/logs/admin

  • Check in app.log for application events

  • Check in web.log for web service events

  • IX:

  • /var/log/vmware/hbrsrv*.log on IX appliances


Check Firewall Port Connectivity

Use this command to check whether the port is open between HCX Manager and ESXi hosts or Appliances

  • curl -vv telnet://<esxihost>:<port>

  • tcpdump -ni any port 4500

  • Nc -vu destination-IP 4500

  • Traceroute -U -p 4500 -s local and remote IP

Verify Network Connection for Download

  • tcpdump port 443



Perftest gives a report on the available bandwidth for the HCX tunnels. Hitting tab should give you a list of available commands.

In order to run "perftest"

  1. from HCM manager, run ccli by calling "ccli" as "root" user

  2. inside ccli context, list deployed appliances by command "list"

  3. from the list shown, select the appliance to run test. Each appliance has a "Id" field. select appliance by command "go <Id>".

  4. run perftest by calling "perftest site"

  5. run perftest with multiple flows with parameter "-P <num_of_flows>"

perftest all allows all tests to be run.

perftest uplink will test the links. use the -T flag to constrain by length of test, i.e. -T 1 will check one minute in each direction.


Health Check Commands

hc probe -d

Select an appliance and run a unit health check to check the state of the resources, services and tunnels.

go 2 (example)

hc unit all -d


HCX Connectivity Troubleshooting

Check the connectivity status from the HCX Interconnect:

  1. Check the Security associations are up: IPsec status

  2. Confirm the routing is appropriate: IP route

  3. Get the remote side IP: IP tunnel

  4. Try pinging the remote side (only if ICMP is allowed): ping <remote side IP>

  5. Confirm traffic on UDP 4500 is sent and received: tcpdump -s0 -n -i vNic_# udp

# tcpdump -c 10 -ni vNic_0 udp and port 4500

(note the vNic_# might be vNic_0 or another vNic, you can get the info via "ip route" and use that in the tcpdump command above)

[admin@HCX-MGR] go 0
Switched to node 0.
[admin@HCX-MGR] ssh
Welcome to HCX Central CLI
[root@<appliance0>] ip tun            
<status of gre tunnel>
[root@<appliance0>] tcpdump -c20 -i vNic_0 -n -e host <ip address of remote side>                                                                        

1. Start the capture

You can just run tcpdump command to start the capture. As you can see below, by default traffic on the eth0 interface is being captured. You will have to terminate the capture with CTRL + C once you have taken enough captures.

root@hcx-mgr:/home# tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
12:40:07.641494 ARP, Request who-has (dc:a6:32:04:d2:56 (oui Unknown)) tell, length 46
12:40:07.641560 ARP, Reply is-at dc:a6:32:04:d2:56 (oui Unknown), length 28
12:40:10.159445 IP > UDP, length 101
12:40:11.487595 IP > UDP, length 21
12:40:11.488865 IP > UDP, length 21
12:40:11.604242 IP > UDP, length 518
12:40:11.891715 IP > UDP, length 504
12:40:12.025215 IP > UDP, length 502
12:40:12.280478 IP > UDP, length 438
12:40:12.725948 IP > UDP, length 447

2. Capture from a specific interface

You can pass -i flag to specify the interface. The below example shows how to capture the packets from the wlan0 interface. You can also use any argument to capture packets on all the interfaces tcpdump -i any

root@hcx-mgr:/home# tcpdump -i wlan0

💡 You can pass -n flag to instruct tcpdump not to convert IP addresses to names.

root@hcx-mgr:/home# tcpdump -i wlan0 -n

3. Capture only a specific number of packets

There are thousands of packets that traverse the interface at any given moment so, you might get overwhelmed by the number of packets on the screen. You can pass -c flag to capture only a specific number of packets. Let's say I only want to capture 10 packets.

root@hcx-mgr:/home# tcpdump -i wlan0 -n -c 10

4. Capture traffic based on specific IP addresses

You can pass src or dst arguments to specify the IP addresses. For example, let's say you only want to capture the traffic destined to HCX IX appliance in target (

root@hcx-mgr:/home# tcpdump -i wlan0 -n dst

If you want to specify both source and destination IPs then ensure to pass the and keyword, for example tcpdump -i wlan0 -n src and dst

5. Capture traffic based on ports

Using the previous example, you can pass port argument to specify ports and protocols. If you capture only ICMP traffic:

root@hcx-mgr:/home# tcpdump -i wlan0 -n icmp

You can also capture only UDP based traffic as shown below

root@hcx-mgr:/home# tcpdump -i wlan0 -n udp

You can also go wild and capture traffic based on packet flags. For example, TCP-SYN, TCP-ACK or ICMP echo-request or echo-reply.

root@hcx-mgr:/home# tcpdump -i wlan0 -n 'icmp[icmptype] = icmp-echoreply'

6. View and export the captures

You can pass -w flag to save the captures as a PCAP file. Use -s 0 to capture full-sized packets.

root@hcx-mgr:/home# tcpdump -i wlan0 -s 0 -n udp -w /home/pi/udp_capture.pcap

You can use -r flag to view the file locally.

root@hcx-mgr:/home# tcpdump -r /home/hcx/udp_capture.pcap 
You can also export the capture file (using FTP/SCP etc) and later view it on Wireshark.
72 views0 comments

Recent Posts

See All
bottom of page