Performing Remote Diagnostics Tests

ARP Table Dump

Run this test to view the contents of the ARP table. The output is limited to display 1000 ARP entries.

Clear ARP Cache

Run this test to clear the ARP cache entries for the specified interface.

DNS Test

Run this test to perform a DNS lookup of the specified domain name.

DNS/DHCP Service Restart

Run this test to restart the DNS/DHCP service. This can serve as a troubleshooting step if DHCP or DNS requests are failing for clients.

DSL Status

The DSL diagnostic test is available only for 610 devices. In the 4.3 release, testing is also available for the 620, 640, and 680 devices. Run this test to show the DSL status, which includes information such as Mode (Standard or DSL), Profile, xDSL Mode, etc. as shown in the image below.

Flush Firewall Sessions

Run this test to reset established sessions from the firewall. Running this test on an Edge not only flushes the firewall sessions, but actively send a TCP RST for the TCP-based sessions.

Flush Flows

Run this test to flush the flow table, causing user traffic to be re-classified. Use source and destination IP address filters to flush specific flows.

Flush NAT

Run this test to flush the NAT table.

Gateway

Run this test by choosing whether cloud traffic should or should not use the Gateway Service.

Note: This does not affect the routing of VPN traffic.

GPON Status

Run this test on any selected 6x0 Edge device to view the GPON SFP status, including Vendor MAC, Host Link Status, Link Rate, TX and RX power, and Optical Status.

HA Info

Run this test to view basic and interface information of active and standby Edges when HA is enabled.

IPv6 Clear ND Cache

Run this test to clear the cache from the ND for the selected Interface.

IPv6 ND Table Dump

Run this test to view the IPv6 address details of Neighbor Discovery (ND) table.

IPv6 RA Table Dump

Run this test to view the details of the IPv6 RA table.

IPv6 Route Table Dump

Run this test to view the contents of the IPv6 Route Table.

Interface Status

Run this test to view the MAC address and connection status of physical interfaces.

LTE Modem Information

Run this test on a selected Edge that has an integrated LTE module, such as 510-LTE or 610-LTE, to collect diagnostic details such as Modem information, Connection information, Location information, Signal information, and Status information for the internal LTE modem.

LTE SIM Switchover

For 610-LTE devices only, run this test to switch active SIMs. Both SIMs must be inserted to run this test. The test will take approximately four to five minutes.

After the test is successful, you can check the status of the current active interface in the SD-WAN Orchestrator under the Monitor -> Edges - > Overview tab.

List Active Firewall Sessions

Run this test to view the current state of the active firewall sessions (up to a maximum of 1000 sessions). You can limit the number of sessions returned by using filters: source and destination IP address, source and destination port, and Segment.

Note: You cannot see sessions that were denied as they are not active sessions. To troubleshoot those sessions you will need to check the firewall logs.

The Remote Diagnostics output displays the following information: Segment name, Source IP, Source Port, Destination IP, Destination Port, Protocol, Application, Firewall Policy, current TCP state of any flows, Bytes Received/Sent, and Duration. There are 11 distinct TCP states as defined in RFC 793:

LISTEN - represents waiting for a connection request from any remote TCP and port. (This state is not shown in a Remote Diagnostic output).
SYN-SENT - represents waiting for a matching connection request after having sent a connection request.
SYN-RECEIVED - represents waiting for a confirming connection request acknowledgment after having both received and sent a connection request.
ESTABLISHED - represents an open connection, data received can be delivered to the user. The normal state for the data transfer phase of the connection.
FIN-WAIT-1 - represents waiting for a connection termination request from the remote TCP, or an acknowledgment of the connection termination request previously sent.
FIN-WAIT-2 - represents waiting for a connection termination request from the remote TCP.
CLOSE-WAIT - represents waiting for a connection termination request from the local user.
CLOSING - represents waiting for a connection termination request acknowledgment from the remote TCP.
LAST-ACK - represents waiting for an acknowledgment of the connection termination request previously sent to the remote TCP (which includes an acknowledgment of its connection termination request).
TIME-WAIT - represents waiting for enough time to pass to be sure the remote TCP received the acknowledgment of its connection termination request.
CLOSED - represents no connection state at all.

List Active Flows

Run this test to list active flows in the system. Use source and destination IP address filters to view the exact flows you want to see. This output is limited to a maximum of 1000 flows.

List Clients

Run this test to view the complete list of clients.

List Paths

Run this test to view the list of active paths between local WAN links and each peer.

MIBs for Edge

Run this test to dump Edge MIBs.

NAT Table Dump

Run this test to view the contents of the NAT Table. Use the destination IP address filter to view the exact entries you want to see. This output is limited to a maximum of 1000 entries.

NTP Dump

Run this test to view the current date and time on Edge and NTP information.

Ping IPv6 Test

Run a ping test to the specified IPv6 destination.

Ping Test

Run a ping test to the specified IPv4 destination.

Reset USB Modem

Run this test on a selected Edge interface to reset an unworking USB modem connected to the given interface. Note that not all USB modems support this type of remote reset.

Route Table Dump

Run this test to view the contents of the Route Table.

The Lost Reason column specifies the code for the reason why a route loses the routing preference calculation logic to the next preferred route, on both Edges and Gateways.

Note: An unresolved route, learnt over multi-hop BGP, might point to an intermediate interface. For more information, see Multi-hop BGP Routes.

The following table lists the reason codes for an Edge and the corresponding descriptions:


Reason Code	Description
PR_UNREACHABLE	In case of overlay routes, the remote peer, which is either Gateway or Edge, is not reachable.
IF_DOWN	Egress Interface is down.
INVALID_IFIDX	Egress Interface if-index for this route is invalid.
SLA_STATE_DOWN	State given by IP SLA tracking is down.
HA_STANDBY	When the local Edge is a Standby, all routes synced from the active are marked as reachable for operational convenience.
LOCAL_MGMT	Management routes are always reachable.
LOOPBACK	Loopback IP address is always reachable.
SELF_ROUTE	Self IP routes are always reachable.
RECUR_UNRES	Recursive routes are marked as reachable so that recursive resolution can be done for operational convenience.
VPN_VIA_NAT	vpnViaNat routes are always reachable.
SLA_STATE_UP	State given by IP SLA tracking is up.
IF_RESOLVED	Egress interface is up and resolved.
PR_REACHABLE	In case of overlay routes, the remote peer, which is either Gateway or Edge, is reachable.
LR_NO_ELECTION	Best route.
LR_NP_SWAN_VS_VELO	Predecessor is selected because it is a non-preferred static WAN route (route configured with preferred flag set to false) when compared to the current route which is a via velocloud route.
LR_NP_SWAN_VS_DEFRT	Predecessor is selected because it is a non-preferred static WAN route when compared to the current route which is default route.
LR_NP_ROUTE_TYPE	Predecessor is selected because its route type is better when compared to the current route. Also, one of the routes being compared is a non-preferred route in this case.
LR_BGP_LOCAL_PREF	Both routes are learnt using BGP. The predecessor is selected because it has a higher local preference than the current route.
LR_BGP_ASPATH_LEN	Both routes are learnt using BGP. Predecessor is selected because it has a lower AS path value than the current route.
LR_BGP_METRIC	Both routes are learnt using BGP. Predecessor is selected because it has a lower metric value than the current route.
LR_EXT_OSPF_INTER	Predecessor is selected because it is a route learnt from OSPF with an inter or intra area metric when compared to the current route which is learnt from BGP.
LR_EXT_BGP_RT	Predecessor is selected because it is a route learnt from BGP when compared to the current route which is a route learn from OSPF with metric type OE1 or OE2.
LR_EXT_METRIC_TYPE	Both routes are OSPF routes. The predecessor is selected because it has a better metric type than the current route. Order of preference for OSPF metric types: OSPF_TYPE_INTRA, OSPF_TYPE_INTER, OSPF_TYPE_OE1, OSPF_TYPE_OE2.
LR_EXT_METRIC_VAL	Both routes are OSPF routes. The predecessor is selected because it has a lesser metric than the current route.
LR_EXT_NH_IP	Both routes are OSPF ECMP routes. The current route is lost to the predecessor since it was learnt later.
LR_PG_BGP_ORDER	Both are remote BGP routes with same BGP parameters. The current route is selected because it is a Partner Gateway (PG) route and has a lesser "order" value when compared to the current route.
LR_NON_PG_BGP_ORDER	Both are remote BGP routes with same BGP parameters. The current route is selected because it is a non-PG route and has a lesser "order" value when compared to the current route.
LR_EXT_ORDER	Both are remote OSPF routes with same metric. The predecessor is selected because it has a lesser order value than the current route.
LR_PREFERENCE	Both are either BGP or OSPF routes. The predecessor is selected because it has a lesser preference value than the current route.
LR_DCE_NSD_STATIC_PREF DCE - Data center, NSD - Non-SDWAN site	Both are local NSD static routes. The predecessor is selected because it is a preferred route (preferred flag set to true) when compared to the current which is non-preferred.
LR_DCE_NSD_STATIC_METRIC	Both are NSD static routes. The predecessor is selected because it has a lesser metric value than the current route.
LR_DCE_NON_REMOTE	Both are NSD static routes. The predecessor is selected because it is a local route (non-remote) and the current route is a remote route.
LR_DCE_NSD_STATIC_REMOTE_ORDER	Both are remote NSD static routes. The predecessor is selected because it has a lesser order value when compared to the current route.
LR_DCE_DC_DIRECT	Both are NSD static routes. The predecessor is selected because its DC_DIRECT flag is set and the current route does not have this flag set. This is the route with "n - nonVelocloud" flag set in the debug.py --routes output. These are routes learnt from an NVS from Edge.
LR_DCE_LOGICAL_ID	Both are NSD static routes. The predecessor is selected because it has a better logical ID than the current route.
LR_NETMASK	The predecessor is selected because it has a higher netmask than the current. This will not hit since the netmask is different, it is a separate network/route entry of its own.
LR_NETADDR	The predecessor is selected because it has a higher network address than the current. This will not hit since the network address is different, it is a separate network/rouute entry of its own.
LR_CONN_FLAG	The predecessor is selected because it is a connected route and the current route is not a connected route.
LR_SELF_FLAG	The predecessor is selected because it is a self route and the current route is not a self route.
LR_SLAN_FLAG	The predecessor is selected because it is a static LAN route and the current route is not a static LAN route.
LR_SWAN_FLAG	The predecessor is selected because it is a static WAN route and the current route is not a static WAN route.
LR_NSD_STATIC_LOCAL	The predecessor is selected because it is a local NSD static route and the current route is an NSD BGP route.
LR_NSD_BGP_VS_NON_PREF_STATIC	The predecessor is selected because it is a NDS BGP route and the current route is a local NSD static non-preferred route.
LR_NSD_STATIC_PREF_VS_NSD_STATIC	The predecessor is selected because it is an NSD static preferred route and the current route is not an NSD static route.
LR_CONN_STATIC_VS_NSD_BGP	The predecessor is selected because it is a remote connected/static route and the current route is an NSD BGP route.
LR_OPG_SECURE_STATIC	The predecessor is selected because it is a PG secure static route and the current is not.
LR_ROUTED_VS_VELO	The predecessor is selected because it is a route learnt from routing protocols when compared the current route which is a "v - ViaVeloCloud" route.
LR_INTF_DEF_VS_ROUTED	The predecessor is selected because it is an interface default cloud route when compared to the cuurrent route which is a route learnt using routing protocols (local or remote).
LR_ROUTE_TYPE	The predecessor is selected because it has a better route than the current.
LR_E2DC_REMOTE	The predecessor is selected because it is a, Edge2DC route and it is a local route and the current route is a remote route.
LR_CONNECTED_LAN	Both are connected routes. The predecessor is selected because it is a connected LAN route and the current route is not a connected LAN route.
LR_VELO_REMOTE_FLAG	Both are cloud routes. The predecessor is selected becauuse it is a remote route when compared to the remote cloud route when compared to the current which is a local cloud route.
LR_VELO_EdgeD_ROUTED	Both are cloud routes. The predecessor is selected because it is a route learnt via routing protocol and the current route is not learnt via routing protocol.
LR_VELO_PG_ROUTE	Both are cloud routes. The predecessor is selected because it is a PG route and the current route is not a PG route.
LR_VIA_VELO_ROUTE	Both are cloud routes. The predecessor is selected because it is a via velocloud route and the current is not a via-velocloud route.
LR_REMOTE_NON_ROUTED	Both are remote (overlay) routes. The predecessor is selected because it is a route not learnt via routing protocol (static/connected) and the current route is a route learnt via routing protocol.
LR_REMOTE_DCE_FLAG	Both are remote (overlay) routes. The predecessor is selected because it is a data center Edge route ("D - DCE" is set in the debug.py --routes output) and the current is not a data center Edge route.
LR_METRIC	The predecessor is selected because it has a lesser metric than the current route.
LR_ORDER	The predecessor is selected because it has a lesser order than the current route.
LR_LOGICAL_ID	The predecessor is selected because it has a better logical ID than the current route.
LR_EXT_BGP_VIA_PRIMGW	Both are BGP routes. The predecessor is selected because it is an NSD BGP route learnt from the primary NSD VCG. The current route might have been learnt from the redundant NDS VCG.

The following table lists the reason codes for a Gateway and the corresponding descriptions:


Reason Code	Description
LR_NO_ELECTION	Best route.
LR_NVS_STATIC_PREF	The predecessor is selected because it is an NVS static route and the current route is not.
LR_EXT_BGP_VS_OSPF	Predecessor is selected because it is a BGP route and the current route is an OSPF route with metric type OE1/OE2.
LR_EXT_BGP_ROUTE	Both are cloud routes. The predecessor is selected because it is a BGP learnt cloud route and the current route is not (it is static).
LR_CLOUD_ROUTE_VS_ANY	The predecessor is selected because it is an Edge2Edge or Edge2Datacenter route and the current route is a cloud static route. Edge2Edge/Edge2Datacenter > Cloud static.
LR_BGP_LOCAL_PREF	Both are either Edge2Edge or Edge2Datacenter routes learnt via BGP. The predecessor is selected because it has a greater local preference value than that of the current route.
LR_BGP_ASPATH_LEN	Both are either Edge2Edge or Edge2Datacenter routes learnt via BGP. The predecessor is selected because it has a lesser AS path value than that of the current route.
LR_BGP_METRIC	Both are either Edge2Edge or Edge2Datacenter routes learnt via BGP. The predecessor is selected because it has a lesser metric value than that of the current route.
LR_DCE_NSD_STATIC_PREF	Both are Edge2Datacenter routes. Predecessor is selected because it is an NSD static route and the current route is not.
LR_DCE_NSD_STATIC_METRIC	Both are Edge2Datacenter static routes. Predecessor is selected because it has lesser metric value than that of the current route.
LR_DCE_NSD_STATIC_GW_NON_REMOTE	Both are Edge2Datacenter static routes. Predecessor is selected because it is a local route and the current is a remote route.
LR_DCE_LOGICAL_ID	Both are Edge2Datacenter static routes. Predecessor is selected because it has better logical ID than that of the current route.
LR_E2DC_METRIC	Both are Edge2Datacenter routes. The predecessor is selected because its metric is lesser than that of the current route.
LR_DC_IPADDR	Both are Edge2Datacenter routes. The predecessor is selected because its datacenter IP address is lesser than that of the current route.
LR_E2DC_NETADDR	Both are Edge2Datacenter routes. The predecessor is selected because its network address is lesser than the current.
LR_E2E_PREFERENCE	Both are Edge2Edge routes. The predecessor is selected because its preference value is lesser than the current route.
LR_E2E_METRIC	Both are Edge2Edge routes. The predecessor is selected because its metric value is lesser than the current route.
LR_E2E_LOGICAL_ID	Both are Edge2Edge routes. The predecessor is selected because it has better logical ID than the current route.
LR_E2E_NETADDR	Both are Edge2Edge routes. The predecessor is selected because its network address is lesser than the current.
LR_OPG_SECURE_STATIC	The predecessor is selected because it is a PG secure static route and the current route is not a PG secure static.
LR_ROUTE_TYPE	The predecessor is selected because it has a better route type than the current route.
LR_NETMASK	The predecessor is selected because it has a higher netmask than the current.
LR_METRIC	The predecessor is selected because it has a lesser metric value than the current route.
LR_PREFERENCE	Both are routes learnt from routing protocols. The predecessor is selected because it has a lesser preference value than the current route.
LR_NETADDR	The predecessor is selected because its network address is lesser than that of the current route.
LR_LOGICAL_ID	The predecessor is selected because its logical ID is better than the current route.

Source Interface Dump

Run this test to view the list of source interfaces used by various services for a segment.

System Information

Run this test to view system information such as system load, recent WAN stability statistics, monitoring services. WAN stability statistics include the number of times individual VPN tunnels and WAN links lost connectivity for at least 700 milliseconds. The tunnel disconnects do not include the count of direct IPsec connections.

Traceroute

Run a traceroute via the Gateway or directly out any of the WAN interfaces to the destination specified.

Troubleshoot BFD - Show BFD/BFDv6 Peer Status

Run this test to show all the status of BFD peers with IPv4 or IPv6 address.

Troubleshoot BFD - Show BFD/BFDv6 Peer counters

Run this test to view all the counters of BFD peers with IPv4 or IPv6 address.

Troubleshoot BFD - Show BFD Setting

Run this test to view BFD setting and neighbor status.

Troubleshoot BFDv6 - Show BFDv6 Setting

Run this test to view BFDv6 setting and neighbor status.

Multi-hop BGP Routes

Over Multi-hop BGP, the system might learn routes that require recursive lookup. These routes have a next-hop IP which is not in a connected subnet, and do not have a valid exit interface. In this case, the routes must have the next-hop IP resolved using another route in the routing table that has an exit interface. When there is traffic for a destination that needs these routes to be looked up, routes requiring recursive lookup will get resolved to a connected Next Hop IP address and interface. Until the recursive resolution happens, the recursive routes point to an intermediate interface.

You can view the unresolved routes pointing to intermediate interface in the following Remote Diagnostics tests:

Troubleshoot BGP - List BGP Redistributed Routes

Run this test to view routes redistributed to BGP neighbors.

Note: An unresolved route, learnt over multi-hop BGP, might point to an intermediate interface. For more information, see Multi-hop BGP Routes.

Troubleshoot BGP - List BGP Routes

Run this test to view the BGP routes from neighbors. You can enter IPv4 or IPv6 prefix to view specific BGP routes or leave the prefix empty to view all the BGP routes.

Note: An unresolved route, learnt over multi-hop BGP, might point to an intermediate interface, as shown in the above image. For more information, see Multi-hop BGP Routes.

Troubleshoot BGP - List Routes per Prefix

Run this test to view all the Overlay and Underlay routes for a specific IPv4 or IPv6 prefix and the related details.

Note: An unresolved route, learnt over multi-hop BGP, might point to an intermediate interface. For more information, see Multi-hop BGP Routes.

Troubleshoot BGP - Show BGP Neighbor Advertised Routes

Run this test to view the BGP routes advertised to a neighbor.

Troubleshoot BGP - Show BGP Neighbor Learned Routes

Run this test to view all the accepted BGP routes learned from a neighbor after filters.

Troubleshoot BGP - Show BGP Neighbor Received Routes

Run this test to view all the BGP routes learned from a neighbor before filters.

Troubleshoot BGP - Show BGP Neighbor details

Run this test to view the details of BGP neighbor.

Troubleshoot BGP - Show BGP Routes per Prefix

Run this test to view all the BGP routes and their attributes for the specified prefix.

Troubleshoot BGP - Show BGP Summary

Run this test to view the existing BGP neighbor and received routes.

Troubleshoot BGP - Show BGP Table

Run this test to view the BGP table.

Troubleshoot BGPv6 - Show BGPv6 Neighbor Advertised Routes

Run this test to view the BGPv6 routes advertised to a neighbor.

Troubleshoot BGPv6 - Show BGPv6 Neighbor Learned Routes

Run this test to view all the accepted BGPv6 routes learned from a neighbor after filters.

Troubleshoot BGPv6 - Show BGPv6 Neighbor Received Routes

Run this test to view all the BGPv6 routes received from a neighbor before filters.

Troubleshoot BGPv6 - Show BGPv6 Neighbor details

Run this test to view the details of BGPv6 neighbor.

Troubleshoot BGPv6 - Show BGPv6 Routes per Prefix

Run this test to view all the BGPv6 routes for the prefix and their attributes.

Troubleshoot BGPv6 - Show BGPv6 Summary

Run this test to view the existing BGPv6 neighbor and received routes.

Troubleshoot BGPv6 - Show BGPv6 Table

Run this test to view the details of BGPv6 table.

Troubleshoot OSPF - List OSPF Redistributed Routes

Run this test to view all the routes redistributed to OSPF neighbor.

Troubleshoot OSPF - List OSPF Routes

Run this test to view the OSPF routes from neighbors for the specified Prefix. Displays all the OSPF routes from the neighbors if the Prefix is not specified.

Troubleshoot OSPF - Show OSPF Database

Run this test to view the OSPF link state database summary.

Troubleshoot OSPF - Show OSPF Database for E1 Self-Originate Routes

Run this test to view the E1 LSA's self-originated routes that are advertised to OSPF router by the Edge.

Troubleshoot OSPF - Show OSPF Neighbors

Run this test to view all the OSPF neighbors and associated information.

Troubleshoot OSPF - Show OSPF Route Table

Run this test to view the existing OSPF route table.

Troubleshoot OSPF - Show OSPF Setting

Run this test to view the OSPF setting and neighbor status.

USB Port Status

Run this test to view the status of USB ports on an Edge.

VPN Test

Select a segment from the drop-down menu and click Run to test VPN connectivity to each peer.

When the VPN test is run, the Edge selects the Source and Destination IP and initiates the tunnel request. The selected Source and Destination IP should meet the following criteria:

It should be a connected route IP
It should be reachable and the routes should be advertised

When the Edge cannot select a valid IP as the Source IP to initiate the tunnel request, the VPN Test will fail with the following error.

Branch-to-Branch vpn is disabled. Please enable it before running the test

WAN Link Bandwidth Test

Run the bandwidth test on a specified WAN link. This test has the benefit of being non-disruptive in multi-link environments. Only the link under test is blocked for user traffic. This means that you can re-run the test on a specific link and the other link(s) will continue to serve user traffic.

As the bandwidth test is run when the tunnel reconnects after a period of instability, there have been occasions in the field where the link has recovered enough for tunnel connectivity, but not enough to accurately measure the bandwidth of the WAN link. To address these scenarios, if the bandwidth test fails or measures a significantly reduced value, the last known “good” measurement will be used and a re-test of the link will be scheduled for 30 minutes after the tunnel is established to ensure a proper measurement.

Note: For WAN link over 900Mbps, it is recommended that the user define the bandwidth of the WAN link.