OAM - SR-IOV with vmxnet3 or SR-IOV with VIRTIO

It is possible in some installations to mix and match and provide different interface types for the Gateway. This generally happens if you have an OAM without SR-IOV. This custom configuration requires additional steps since this causes the interfaces to come up out of order.

1. Record the MAC address of each interface.

   VMWare

   After creating the machine, go to Edit Settings and copy the Mac address.

   

   KVM

   After defining the VM, run the following command:

   

2. Edit the user-data and lock the mac address to the interface order as follows:

   Userdata

   #cloud-config
    hostname: #_Hostname_#
    password: #_Console_Password_#
    chpasswd: {expire: False}
    ssh_pwauth: True
    ssh_authorized_keys:
    - #_SSH_public_Key_#
    runcmd:
    - 'echo "[]" > /opt/vc/etc/vc_blocked_subnets.json'
    - 'sed -iorig "s/wan=\".*/wan=\"eth0 eth1\"/" /etc/config/gatewayd-tunnel'
    - '/var/lib/cloud/scripts/per-boot/config_gateway'
    - 'sleep 10'
    - '/opt/vc/bin/vc_procmon restart'
    write_files:
     - path: "/etc/udev/rules.d/70-persistent-net.rules" 
     permissions: '0644' 
     content: | 
     SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="#_ETH0_MAC_ADDRESS_#", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"  
     SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="#_ETH1_MAC_ADDRESS_#", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"  
     # NOTE ETH2 IS OAM IF NO OAM PRESENT THEM REMOVE 
     SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="#_ETH2_MAC_ADDRESS_#", ATTR{type}=="1", KERNEL=="eth*", NAME="eth2"  
    - path: "/var/lib/cloud/scripts/per-boot/config_gateway"
    permissions: "0777"
    content: |
    #!/usr/bin/python
    import json
    import commands
    is_activated = commands.getoutput("/opt/vc/bin/is_activated.py")
    if "True" in str(is_activated):
    print "Gateway already activated"
    exit
    commands.getoutput("/opt/vc/bin/activate.py -s #_VCO_# #_Activation_Key_# ")
     
    ### EDIT GATEWAYD ###
    with open("/etc/config/gatewayd", "r") as jsonFile:
    data = json.load(jsonFile)
    data["global"]["vcmp.interfaces"] = ["eth0"]
    data["global"]["wan"] = ["eth1"]
    # NOTE FOR HAND OFF IT CAN BE "QinQ (0x8100)" "QinQ (0x9100)" "none" "802.1Q" "802.1ad"
    data["vrf_vlan"]["tag_info"][0]["mode"] = "#_Handoff_"
    data["vrf_vlan"]["tag_info"][0]["interface"] = "eth1"
    data["vrf_vlan"]["tag_info"][0]["c_tag"] = "#_C_TAG_FOR_MGMT_VRF_#"
    data["vrf_vlan"]["tag_info"][0]["s_tag"] = "#_S_TAG_FOR_MGMT_VRF_"
    with open("/etc/config/gatewayd", "w") as jsonFile:
    jsonFile.write(json.dumps(data,sort_keys=True,indent=4, separators=(",", ": ")))
     
    ### EDIT DPDK ###
    with open("/opt/vc/etc/dpdk.json", "r") as jsonFile:
    data = json.load(jsonFile)
    #SET 0 or 1 for enabled or DISABLED example data["dpdk_enabled"] = 0
    data["dpdk_enabled"] = #_DKDP_ENABLED_OR_DISABLED_#
    with open("/opt/vc/etc/dpdk.json", "w") as jsonFile:
    jsonFile.write(json.dumps(data,sort_keys=True,indent=4, separators=(",", ": ")))
    final_message: "==== Cloud-init completed ===="
    power_state:
    delay: "+1"
    mode: reboot
    message: Bye Bye
    timeout: 30
    condition: True