The VMware Partner Gateway provides different configuration options. A worksheet should be prepared before the installation of the Gateway.

Worksheet

SD-WAN Gateway
  • Version
  • OVA/QCOW2 file location
  • Activation Key
  • SD-WAN Orchestrator (IP ADDRESS/vco-fqdn-hostname)
  • Hostname
Hypervisor Address/Cluster name
Storage Root volume datastore (>40GB recommended)
Note: It is recommended that on a Partner Gateway Host, the free disk space in the /tmp/partition directory is at least twice the size of memory (RAM).
CPU Allocation CPU Allocation for KVM/VMware.
Installation Selections DPDK—This is optional and enabled by default for higher throughput. If you choose to disable DPDK, contact VMware Customer Support.
OAM Network ( Optional See Custom Configurations)
  • DHCP
  • OAM IPv4 Address
  • OAM IPv4 Netmask
  • DNS server - primary
  • DNS server - secondary
  • Static Routes
ETH0 – Internet Facing Network
  • IPv4 Address
  • IPv4 Netmask
  • IPv4 Default gateway
  • DNS server - primary
  • DNS server - secondary
Handoff (ETH1) - Network
  • MGMT VRF IPv4 Address
  • MGMT VRF IPv4 Netmask
  • MGMT VRF IPv4 Default gateway
  • DNS server - primary
  • DNS server - secondary
  • Handoff ( QinQ (0x8100), QinQ (0x9100), none, 802.1Q, 802.1ad)
  • C-TAG
  • S-TAG
Console access
  • Console_Password
  • SSH:
    • Enabled (yes/no)
    • SSH public key
NTP ( Optional see Custom Configuration Section)
  • Public NTP:
    • server 0.ubuntu.pool.ntp.org
    • server 1.ubuntu.pool.ntp.org
    • server 2.ubuntu.pool.ntp.org
    • server 3.ubuntu.pool.ntp.org
  • Internal NTP server - 1
  • Internal NTP server - 2

SD-WAN Gateway Section

Most of the SD-WAN Gateway section is self-explanatory.

SD-WAN Gateway
  • Version - Should be same or lower than SD-WAN Orchestrator
  • OVA/QCOW2 file location - Plan ahead the file location and disk allocation
  • Activation Key
  • SD-WAN Orchestrator (IP ADDRESS/vco-fqdn-hostname)
  • Hostname - Valid Linux Hostname “RFC 1123”

Creating a Gateway and Getting the Activation Key

  1. Go to Operator > Gateway Pool and create a new SD-WAN Gateway pool. For running SD-WAN Gateway in the Service Provider network, check the Allow Partner Gateway checkbox. This will enable the option to include the partner gateway in this gateway pool.

    vcg-new-pool

  2. Go to Operator > Gateway and create a new gateway and assign it to the pool. The IP address of the gateway entered here must match the public IP address of the gateway. If unsure, you can run curl ipinfo.io/ip from the SD-WAN Gateway which will return the public IP of the SD-WAN Gateway.

    vcg-create-new-gateway

  3. Make a note of the activation key and add it to the worksheet.

    vcg-gateway-activation-key

Enable Partner Gateway Mode

  1. Go to Operator > Gateways and select the SD-WAN Gateway. Check the Partner Gateway checkbox to enable the Partner Gateway.

    vcg-properties-partner-gateway-checkbox

    There are additional parameters that can be configured. The most common are the following:

    Advertise 0.0.0.0/0 with no encrypt

    vcg-advanced-handoff-static-routes

This option will enable the Partner Gateway to advertise a path to Cloud traffic for the SAAS Application. Since the Encrypt Flag is off, it will be up to the customer configuration on the business policy to use this path or not.

The second recommend option is to advertise the SD-WAN Orchestrator IP as a /32 with encrypt.

vcg-advanced-handoff-static-routes-encrypt-checkbox

This will force the traffic that is sent from the Edge to the SD-WAN Orchestrator to take the Gateway Path. This is recommended since it introduces predictability to the behavior that the SD-WAN Edge takes to reach the SD-WAN Orchestrator.

Networking

Important: The following procedure and screenshots focus on the most common deployment, which is the 2-ARM installation for the Gateway. The addition of an OAM network is considered in the section titled, OAM Interface and Static Routes.

vcg-partner-gateway-pe-image

The diagram above is a representation of the SD-WAN Gateway in a 2-ARM deployment. In this example, we assume eth0 is the interface facing the public network (Internet) and eth1 is the interface facing the internal network (handoff or VRF interface).

Note: A Management VRF is created on the SD-WAN Gateway and is used to send a periodic ARP refresh to the default gateway IP to check that the handoff interface is physically up and speed ups the failover time. It is recommended that a dedicated VRF is set up on the PE router for this purpose. Optionally, the same management VRF can also be used by the PE router to send an IP SLA probe to the SD-WAN Gateway to check for SD-WAN Gateway status ( SD-WAN Gateway has a stateful ICMP responder that will respond to ping only when its service is up).If a dedicated Management VRF is not set up, then you can use one of the customer VRFs as a Management VRF, although this is not recommended.

For the Internet Facing network, you only need the basic network configuration.

ETH0 – Internet Facing Network
  • IPv4_Address
  • IPv4_Netmask
  • IPv4_Default_gateway
  • DNS_server_primary
  • DNS_server_secondary

For the Handoff interface, you must know which type of handoff you want to configure and the Handoff configuration for the Management VRF.

ETH1 – HANDOFF Network
  • MGMT_IPv4_Address
  • MGMT_IPv4_Netmask
  • MGMT_IPv4_Default gateway
  • DNS_Server_Primary
  • DNS_Server_Secondary
  • Handoff (QinQ (0x8100), QinQ (0x9100), none, 802.1Q, 802.1ad)
  • C_TAG_FOR_MGMT_VRF
  • S_TAG_FOR_MGMT_VRF

Console Access

Console access
  • Console_Password
  • SSH:
    • Enabled (yes/no)
    • SSH public key

In order to access the Gateway, a console password and/or an SSH public key must be created.

Cloud-Init Creation

The configuration options for the gateway that we defined in the worksheet are used in the cloud-init configuration. The cloud-init config is composed of two main configuration files, the metadata file and the user-data file. The meta-data contains the network configuration for the Gateway, and the user-data contains the Gateway Software configuration. This file provides information that identifies the instance of the SD-WAN Gateway being installed.

Below are the templates for both Meta_data and User_data files.

Fill the templates with the information in the worksheet. All #_VARIABLE_# need to be replaced, also check any #ACTION#

Important: The template assumes you are using static configuration for the interfaces. It also assumes that you are either using SR-IOV for all interfaces or none. For more information, see OAM - SR-IOV with vmxnet3 or SR-IOV with VIRTIO.
meta-data file:
instance-id: #_Hostname_#
local-hostname: #_Hostname_#
 
network-interfaces: |
   auto eth0
      iface eth0 inet static
         address #_IPv4_Address_# 
         mac_address #_mac_Address_#       
         netmask #_IPv4_Netmask_# 
         gateway #_IPv4_Gateway_# 
         dns-nameservers 
            #_DNS_server_primary_# 
            #_DNS_server_secondary_# 
   auto eth1
      iface eth1 inet static
         metric '13'
         address #_MGMT_IPv4_Address_# 
         mac_address #_MGMT_mac_Address_# 
         netmask #_MGMT_IPv4_Netmask_# 
         gateway #_MGMT_IPv4_Gateway_# 
         dns-nameservers 
            #_DNS_server_primary_# 
            #_DNS_server_secondary_# 
 
user-data file:
#cloud-config
hostname: #_Hostname_#
password: #_Console_Password_# 
chpasswd: 
  expire: false

ssh_authorized_keys: 
  - #_SSH_public_Key_#
ssh_pwauth: true

velocloud: 
  vcg: 
    activation_code: #_Activation_Key_#
    vco: #_VCO_#

runcmd: 
  - "echo \"[]\" > /opt/vc/etc/vc_blocked_subnets.json"
  - "sed -iorig \"s/wan=\\\".*/wan=\\\"eth0 eth1\\\"/\" /etc/config/gatewayd-tunnel"
  - /var/lib/cloud/scripts/per-boot/config_gateway
  - "sleep 10"
  - "/opt/vc/bin/vc_procmon restart"

write_files: 
  - 
    content: |
        #!/usr/bin/python
        import json
        ### EDIT GATEWAYD ###
        with open("/etc/config/gatewayd", "r") as jsonFile:
          data = json.load(jsonFile)
        data["global"]["vcmp.interfaces"] = ["eth0"]
        data["global"]["wan"] = ["eth1"]
        # NOTE FOR HAND OFF IT CAN BE "QinQ (0x8100)" "QinQ (0x9100)" "none" "802.1Q" "802.1ad"
        data["vrf_vlan"]["tag_info"][0]["mode"] = "#_Handoff_" 
        data["vrf_vlan"]["tag_info"][0]["interface"] = "eth1"
        data["vrf_vlan"]["tag_info"][0]["c_tag"] = "#_C_TAG_FOR_MGMT_VRF_#" 
        data["vrf_vlan"]["tag_info"][0]["s_tag"] = "#_S_TAG_FOR_MGMT_VRF_" 
        with open("/etc/config/gatewayd", "w") as jsonFile:
          jsonFile.write(json.dumps(data,sort_keys=True,indent=4, separators=(",", ": ")))
        ### EDIT DPDK ###
        with open("/opt/vc/etc/dpdk.json", "r") as jsonFile:
          data = json.load(jsonFile)
        #SET 0 or 1 for enabled or DISABLED example data["dpdk_enabled"] = 0
        data["dpdk_enabled"] = #_DKDP_ENABLED_(1)_OR_DISABLED_(0)_# 
        with open("/opt/vc/etc/dpdk.json", "w") as jsonFile:
          jsonFile.write(json.dumps(data,sort_keys=True,indent=4, separators=(",", ": ")))
    path: /var/lib/cloud/scripts/per-boot/config_gateway
    permissions: "0755"

final_message: "==== Cloud-init completed ===="

power_state: 
  condition: true
  delay: "+1"
  message: "Bye Bye"
  mode: reboot
  timeout: 30
Important:
  • VMware recommends to have a proper fully qualified domain name (FQDN) configured for all production Orchestrators so proper TLS certificates may be issued for them.
  • If activation using the Orchestrator’s IP address is the only option, use the following example which instructs the Edge to bypass TLS verification.
    commands.getoutput("/opt/vc/bin/ activate.py -s myvco.example.com -i #_activation_key_#")
  • This configuration is not recommended for production use and we highly encourage you to reactivate against the Orchestrator’s hostname at the soonest possible.
Note: Always validate user-data and metadata, using http://www.yamllint.com/. The metadata should also be a valid network configuration under the network-interface ( /etc/network/interfaces) section, once the cloud-init completes. Sometimes when working with the Windows/Mac copy paste feature, there is an issue of introducing Smart Quotes which can corrupt the files. Run the following command to make sure you are smart quote free.
sed s/[”“]/'"'/g /tmp/user-data > /tmp/user-data_new

Create ISO File

Once you have completed your files, they need to be packaged into an ISO image. This ISO image is used as a virtual configuration CD with the virtual machine. This ISO image, called vcg01-cidata.iso, is created with the following command on a Linux system:

genisoimage -output vcg01-cidata.iso -volid cidata -joliet -rock user-data meta-data

If you are on a MAC OSX, use the command below instead:

mkisofs -output vcg01-cidata.iso -volid cidata -joliet -rock {user-data,meta-data}

This iso file which we will call #CLOUD_INIT_ISO_FILE# is going to be used in both OVA and VMware installations.