If logging is enabled for firewall rules, you can look at the firewall packet logs to troubleshoot issues.

The log file is /var/log/dfwpktlogs.log for both ESXi and KVM hosts.

Table 1. Firewall Log File Variables
Variable Possible Values
Filter hash A number that can be used to get the filter name and other information.
AF Value INET, INET6
Reason
  • match: Packet matches a rule.
  • bad-offset: Datapath internal error while getting packet.
  • fragment: The non-first fragments after they are assembled to the first fragment.
  • short: Packet too short (for example, not even complete to include an IP header, or TCP/UDP header).
  • normalize: Malformed packets that do not have a correct header or a payload.
  • memory: Datapath out of memory.
  • bad-timestamp: Incorrect TCP timestamp.
  • proto-cksum: Bad protocol checksum.
  • state-mismatch: TCP packets that do not pass the TCP state machine check.
  • state-insert: Duplicate connection is found.
  • state-limit: Reached the maximum number of states that a datapath can track.
  • SpoofGuard: Packet dropped by SpoofGuard.
  • TERM: A connection is terminated.
Action
  • PASS: Accept the packet.
  • DROP: Drop the packet.
  • NAT: SNAT rule.
  • NONAT: Matched the SNAT rule, but cannot translate the address.
  • RDR: DNAT rule.
  • NORDR: Matched the DNAT rule, but cannot translate the address.
  • PUNT: Send the packet to a service VM running on the same hypervisor of the current VM.
  • REDIRECT: Send the packet to network service running out of the hypervisor of the current VM.
  • COPY: Accept the packet and make a copy to a service VM running on the same hypervisor of the current VM.
  • REJECT: Reject the packet.
Rule set and rule ID rule set/rule ID
Direction IN, OUT
Packet length length
Protocol TCP, UDP, ICMP, or PROTO (protocol number)

For TCP connections, the actual reason that a connection is terminated is indicated after the keyword TCP.

If TERM is the reason for a TCP session, then an extra explanation appears in the PROTO row. The possible reasons for terminating a TCP connection include: RST (TCP RST packet), FIN (TCP FIN packet), and TIMEOUT (idle for too long)

In the example above, it is RST. So it means that there is a RST packet in the connection that must be reset.

For non-TCP connections (UDP, ICMP or other protocols), the reason for terminating a connection is only TIMEOUT.

Source IP address and port IP address/port
Destination IP address and port IP address/port
TCP flags S (SYN), SA (SYN-ACK), A (ACK), P (PUSH), U (URGENT), F (FIN), R (RESET)
Number of packets Number of packets.

22/14 - in packets / out packets

Number of bytes Number of bytes.

7684/1070 - in bytes/ out bytes

The following is a regular log sample for distributed firewall rules: 
2018-07-03T19:44:09.749Z b6507827 INET match PASS mainrs/1024 IN 52 TCP 192.168.4.3/49627->192.168.4.4/49153 SEW

2018-07-03T19:46:02.338Z 7396c504 INET match DROP mainrs/1024 OUT 52 TCP 192.168.4.3/49676->192.168.4.4/135 SEW

2018-07-06T18:15:49.647Z 028cd586 INET match DROP mainrs/1027 IN 36 PROTO 2 0.0.0.0->224.0.0.1

2018-07-06T18:19:54.764Z 028cd586 INET6 match DROP mainrs/1027 OUT 143 UDP fe80:0:0:0:68c2:8472:2364:9be/546->ff02:0:0:0:0:0:1:2/547
The elements of a DFW log file format include the following, separated by a space:
  • timestamp:
  • last eight digits of the VIF ID of the interface
  • INET type (v4 or v6)
  • reason (match)
  • action (PASS, DROP, REJECT)
  • rule set name/ rule ID
  • packet direction (IN/OUT)
  • packet size
  • protocol (TCP, UDP, or PROTO #)
  • SVM direction for netx rule hit
  • source IP address/source port>destination IP address/destination port
  • TCP flags (SEW)
For passed TCP packets there is a termination log when the session has ended: 
2018-07-03T19:44:30.585Z 7396c504 INET TERM mainrs/1024 OUT TCP RST 192.168.4.3/49627->192.168.4.4/49153 20/16 1718/76308
The elements of a TCP termination log include the following, separated by a space:
  • timestamp:
  • last 8 digits of the VIF ID of the interface
  • INET type (v4 or v6)
  • action (TERM)
  • ruleset name/ rule ID
  • packet direction (IN/OUT)
  • protocol (TCP, UDP, or PROTO #)
  • TCP RST flag
  • SVM direction for netx rule hit
  • source IP address/source port>destination IP address/destination port
  • IN packet count/OUT packet count (all accumulated)
  • IN packet size/OUT packet size
The following is a sample of FQDN log file for distributed firewall rules: 
2019-01-15T00:34:45.903Z 7c607b29 INET match PASS 1031 OUT 48 TCP 10.172.178.226/32808->23.72.199.234/80 S www.sway.com(034fe78d-5857-0680-81e4-d8da6b28d1b4)
The elements of an FQDN log include the following, separated by a space:
  • timestamp:
  • last eight digits of the VIF ID of the interface
  • INET type (v4 or v6)
  • reason (match)
  • action (PASS, DROP, REJECT)
  • ruleset name/ rule ID
  • packet direction (IN/OUT)
  • packet size
  • protocol (TCP, UDP, or PROTO #) - for TCP connections, the actual reason that a connection is terminated is indicated after the following IP address
  • source IP address/source port>destination IP address/destination port
  • TCP flags - S (SYN), SA (SYN-ACK), A (ACK), P (PUSH), U (URGENT), F (FIN), R (RESET
  • domain name/UUID where UUID is the binary internal representation of the domain name
The following is a sample of Layer 7 log file for distributed firewall rules: 
2019-01-15T00:35:07.221Z 82f365ae INET match REJECT 1034 OUT 48 TCP 10.172.179.6/49818->23.214.173.202/80 S APP_HTTP

2019-01-15T00:34:46.486Z 7c607b29 INET match PASS 1030 OUT 48 UDP 10.172.178.226/42035->10.172.40.1/53 APP_DNS
The elements of a Layer 7 log include the following, separated by a space:
  • timestamp:
  • last eight digits of the VIF ID of the interface
  • INET type (v4 or v6)
  • reason (match)
  • action (PASS, DROP, REJECT)
  • ruleset name/ rule ID
  • packet direction (IN/OUT)
  • packet size
  • protocol (TCP, UDP, or PROTO #) - for TCP connections, the actual reason that a connection is terminated is indicated after the following IP address
  • source IP address/source port>destination IP address/destination port
  • TCP flags - S (SYN), SA (SYN-ACK), A (ACK), P (PUSH), U (URGENT), F (FIN), R (RESET
  • APP_XXX is the discovered application