You can configure BGP dynamic routing for the provider and tenant networks in your environment.

After you enable BGP, the logical subnets created by your tenants are advertised outside of your environment without requiring source NAT or floating IP addresses. You must first create a VXLAN external network that you later use as internal interface for your gateway edges.

Procedure

  1. Log in to the OpenStack Management Server as viouser.
  2. Log in to the controller node as viouser.
  3. Switch to the root user and load the cloud administrator credentials file. 
    sudo su -
source ~/cloudadmin.rc
  4. Create an IPv4 address scope for future tenant subnets and the subnet of your external VXLAN network. 
    neutron address-scope-create name 4
  5. Create a subnet pool for the external network. 
    neutron subnetpool-create external-pool --pool-prefix network-address --default-prefixlen prefix-bits --address-scope scope-name --shared
    Option Description
    external-pool Enter a name for the subnet pool.
    --pool-prefix Enter the network address of the subnet pool in CIDR format (for example, 192.0.2.0/24). Subnets will be allocated from this network.
    --default-prefixlen Enter the network prefix length (in bits) to use for new subnets that are created without specifying a prefix length.
    --address-scope Enter the name of the IPv4 address scope that you created in Step 4.
  6. Create a subnet pool for tenant networks. 
    neutron subnetpool-create tenant-pool --pool-prefix network-address --default-prefixlen prefix-bits --address-scope scope-name --shared
    Note: OpenStack will advertise this subnet pool to the physical fabric. Specify a prefix that is not currently in use.
    Option Description
    tenant-pool Enter a name for the subnet pool.
    --pool-prefix Enter the network address of the subnet pool in CIDR format (for example, 192.51.100.0/24). Subnets will be allocated from this network.
    --default-prefixlen Enter the network prefix length (in bits) to use for new subnets that are created without specifying a prefix length.
    --address-scope Enter the name of the IPv4 address scope that you created in Step 4.
  7. Create a VXLAN-based external network. 
    neutron net-create network-name --provider:network_type vxlan --router:external

    This command creates a new logical switch in NSX Data Center for vSphere.

  8. Create a subnet on the external network.
    The subnet must have DHCP disabled and no gateway. 
    neutron subnet-create external-network external-subnet-address --name external-subnet --allocation-pool start=subnet-ip1,end=subnet-ip2 --subnetpool provider-subnet-pool --no-gateway --disable-dhcp
    Option Description
    external-network Enter the name of the VXLAN-based external network that you created in Step 7.
    external-subnet-address Enter the network address for the subnet in CIDR format (for example, 192.51.100.0/28).
    --name Enter a name for the subnet.
    --allocation-pool Enter the first and last IP addresses of the range that you want to allocate from this subnet.
    --subnetpool Enter the subnet pool that you created in Step 5 for the external network.
  9. Create BGP edge nodes. 
    sudo -u neutron nsxadmin -r bgp-gw-edge -o create --property name=edge-name --property local-as=local-as-number --property external-iface=portgroup-moid:mgmt-network-ip --property internal-iface=physical-net-id:external-network-ip
    Option Description
    name Enter a name for the BGP edge node.
    local-as Enter the local AS number for the edge node. The edges and physical routers cannot be in the same AS.
    external-iface Enter the managed object identifier (MOID) of the port group associated with the VLAN that connects the edge nodes to the physical routers. After the colon, enter the IP address of the edge node on the management network.
    internal-iface

    Enter the virtual wire identifier of the VXLAN-based external network. After the colon, enter the IP address of the edge node on the physical network.

    To find the virtual wire identifier, run the openstack network show external-network-name command and locate the value of the provider:physical_network parameter.

  10. Enable BGP advertisement on the edge nodes. 
    sudo -u neutron nsxadmin -r routing-redistribution-rule -o create --property gw-edge-ids=edge1-id,edge2-id --property learner-protocol=bgp --property learn-from=connected,bgp --property action=permit

    For the gw-edge-ids parameter, use the edge identifier (for example, edge-4) instead of the name. You can run the sudo -u neutron nsxadmin -r bgp-gw-edge -o view command to display the identifier of each BGP edge node.

  11. Establish a BGP neighbor relationship between the edge nodes. 
    sudo -u neutron nsxadmin -r bgp-neighbour -o create --property gw-edge-ids=edge1-id,edge2-id --property ip-address=physical-router1-ip --property remote-as=remote-as-number --property password=bgp-password
    Option Description
    gw-edge-ids Enter the edge identifier of each node, separated by a comma.
    ip-address Enter the IP address on the physical router.
    remote-as Enter the AS number of the physical routers connected to the edge nodes.
    password Enter the BGP password.
  12. Configure your physical routers.
    1. Ensure that the AS of the physical routers is the remote AS of the edge nodes.
    2. Configure the edge nodes as BGP neighbors.
    3. Set each router to advertise itself as a dynamic gateway.
  13. Create and configure the BGP speaker.
    1. Create the BGP speaker. 
      neutron bgp-speaker-create --local-as local_as_value name_bgp_speaker
    2. Create BGP peers. 
      neutron bgp-peer-create --peer-ip internal_interface_network_GW-EDGE1 --remote-as 65001 --password BGP_password --auth-type md5 name_GW-EDGE1 --esg-id edge-ID_GW-EDGE1

neutron bgp-peer-create --peer-ip internal_interface_network_GW-EDGE2 --remote-as 65001 --password BGP_password --auth-type md5 name_GW-EDGE2 --esg-id edge-ID_GW-EDGE2
    3. Add the BGP peer to the BGP speaker. 
      neutron bgp-speaker-peer-add name_bgp_speaker name_GW-EDGE1 

neutron bgp-speaker-peer-add name_bgp_speaker name_GW-EDGE2
    4. Associate the speaker with the VXLAN network. 
      neutron bgp-speaker-network-add name_bgp_speaker external_VXLAN_network_name
  14. (Optional) Create BGP routers for tenants.
    Tenant users can create their BGP routers. The tenant user must be admin to configure a router without SNAT.
    1. Create two logical switches for a tenant and subnet pools for them. 
      neutron net-create name_Tenant1_LS1

neutron subnet-create --name name_network_Tenant1-LS1 name_Tenant1_LS1 --subnetpool selfservice

neutron net-create name_Tenant1_LS2

neutron subnet-create --name name_network_Tenant1-LS2 name_Tenant1_LS2 --subnetpool selfservice
    2. Create a router with BGP configuration.
      BGP works with all OpenStack Logical Routers form factors : shared , distributed , and exclusive. 
      neutron router-create name_Tenant1-LR --router_type=exclusive

neutron router-interface-add name_Tenant1-LR name_network_Tenant1-LS1

neutron router-interface-add name_Tenant1-LR name_network_Tenant1-LS2

neutron router-gateway-set name_Tenant1-LR --disable-snat external_VXLAN_network_name

Results

BGP dynamic routing is now configured on the provider side and tenants can also use it.