This section explains how to configure dedicated interfaces for HSM and sideband (ASM) communication on a new SE.
Configuring Dedicated Interfaces for both HSM and ASM (Sideband) Communication on a new SE
The dedicated HSM and sideband interfaces on SEs use the following configuration parameters:
avi.hsm-ip.SE
avi.hsm-static-routes.SE
avi.hsm-vnic-id.SE
For new SEs, these parameters can be provided in the day-zero YAML file.
YAML Parameters for HSM
YAML Parameter |
Description |
Format |
Example |
|---|---|---|---|
avi.hsm-ip.SE |
IP address of the dedicated HSM vNIC on the SE (this is NOT the IP address of the HSM device) |
IP-address/subnet-mask |
avi.hsm-ip.SE: 10.160.103.227/24 |
avi.hsm-static-routes.SE |
These are comma-separated, static routes to reach HSM devices. Even /32 routes can be provided. Note: If there is a single static route, provide the same and ensure the square brackets are matched. Also, if HSM devices are in the same subnet as the dedicated interfaces, provide the gateway as the default gateway for the subnet. |
[ hsm network1/mask1 via gateway1, hsm network2/mask2 via gateway2 ] OR [ hsm network1/mask1 via gateway1 ] |
avi.hsm-static-routes.SE:[ 10.128.1.0/24 via 10.160.103.1, 10.128.2.0/24 via 10.160.103.2] |
avi.hsm-vnic-id.SE |
ID of the dedicated HSM vNIC and is typically 3 on CSP (vNIC0 is management interface, vNIC1 is data-in interface, and vNIC2 is data-out interface) |
numeric vNIC ID |
avi.hsm-vnic-id.SE: '3' |
YAML Parameters for ASM
YAML Parameter |
Description |
Format |
Example |
|---|---|---|---|
avi.asm-ip.SE |
IP address of the dedicated ASM vNIC on the SE (this is NOT the IP address of the ASM) |
IP-address/subnet-mask |
avi.asm-ip.SE: 10.160.103.227/24 |
avi.hsm-static-routes.SE |
These are comma-separated, static routes to reach the sideband ASM vips. Even /32 routes can be provided. The gateway will be the self IP of the ASM device. Note: If there is a single static route, provide the same and ensure the square brackets are matched. Also, if the ASM virtual service IPs are in the same subnet as the dedicated interfaces, provide the gateway as the default gateway for the subnet. |
[asm-vip-network1/mask1 via gateway1, asm-vip-network2/mask2 via gateway2] or [asm-vip-network1/mask1 via gateway1] |
avi.asm-static-routes.SE: [169.254.1.0/24 via 10.160.102.1, 169.254.2.0/24 via 10.160.102.2] |
avi.asm-vnic-id.SE |
ID of the dedicated ASM vNIC and is typically 3 on CSP (vNIC0 is management interface, vNIC1 is data-in interface, and vNIC2 is data-out interface) |
numeric vNIC ID |
avi.asm-vnic-id.SE: '3' |
Instructions
A sample Service Engine YAML file for the day-zero configuration on Cisco CSP looks as follows:
bash# cat avi_meta_data_dedicated_asm_hsm_SE.yml avi.mgmt-ip.SE: "10.128.2.18" avi.mgmt-mask.SE: "255.255.255.0" avi.default-gw.SE: "10.128.2.1" AVICNTRL: "10.10.22.50" AVICNTRL_AUTHTOKEN: “febab55d-995a-4523-8492-f798520d4515” avi.hsm-ip.SE: 10.160.103.227/24 avi.hsm-static-routes.SE:[ 10.128.1.0/24 via 10.160.103.1, 10.128.2.0/24 via 10.160.103.2] avi.hsm-vnic-id.SE: '3' avi.asm-vnic-id.SE: ‘4' avi.asm-static-routes.SE: [169.254.1.0/24 via 10.160.102.1, 169.254.2.0/24 via 10.160.102.2] avi.asm-ip.SE: 10.160.102.227/24
Once the SE is created with this day-zero configuration and appropriate virtual NIC interfaces are added to the SE service instance in CSP, verify that the dedicated vNIC configuration is applied successfully and the HSM devices and ASM virtual service IPs are reachable via the dedicated interfaces. In this sample configuration, the interface eth3 is configured as the dedicated HSM interface with IP 10.160.103.227/24 and the interface eth4 is configured as the sideband ASM interface with IP 10.160.102.227/24.
The SE requires the following 5 interfaces for this configuration:
vNIC0: Management interface
vNIC1: Data in interface
vNIC2: Data out interface
vNIC3: Dedicated HSM interface
vNIC4: Dedicated sideband interface
To verify configuration of both the dedicated interfaces, ssh to the SE IP, run the IP route command, and perform a ping test.
bash# ssh [email protected] bash# ifconfig eth3 eth3 Link encap:Ethernet HWaddr 02:6a:80:02:11:05 inet addr:10.160.103.227 Bcast:10.160.103.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:4454601 errors:0 dropped:1987 overruns:0 frame:0 TX packets:4510346 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:672683711 (672.6 MB) TX bytes:875329395 (875.3 MB)
bash# ip route default via 10.10.2.1 dev eth0 10.10.1.0/24 via 10.160.103.1 dev eth3 10.10.2.0/24 via 10.160.103.2 dev eth3 10.10.2.0/24 dev eth0 proto kernel scope link src 10.128.2.27 10.160.103.0/24 dev eth3 proto kernel scope link src 10.160.103.227 bash# ping -I eth3 <HSM-IP> ping -I eth3 10.10.1.51 PING 10.10.1.51 (10.128.1.51) from 10.160.103.227 eth3: 56(84) bytes of data. 64 bytes from 10.10.1.51: icmp_seq=1 ttl=62 time=0.229 ms
Configuring Dedicated Interfaces for both HSM and ASM (Sideband) Communication on an Existing NSX Advanced Load Balancer Service Engine
The dedicated sideband interfaces on the SEs use the following configuration parameters:
avi.asm-ip.SE
avi.hsm-static-routes.SE
avi.asm-vnic-id.SE
For existing SEs, these parameters can be populated in the /etc/ovf_config file.
All parameters in this file are comma-separated and the file format is slightly different from the YML file used for spinning up new SEs. However, the parameters and their respective formats are exactly the same as they are for the new SEs.
YAML Parameters
YAML Parameter |
Description |
Format |
Example |
|---|---|---|---|
avi.asm-ip.SE |
IP address of the dedicated ASM vNIC on the SE (this is NOT the IP address of the ASM) |
IP-address/subnet-mask |
avi.asm-ip.SE: 10.160.103.227/24 |
avi.hsm-static-routes.SE |
These are comma-separated, static routes to reach the sideband ASM virtual service IPs. Even /32 routes can be provided. The gateway will be the self IP of the ASM device. Note: If there is a single static route, provide the same and ensure the square brackets are matched. Also, if the ASM virtual service IPs are in the same subnet as the dedicated interfaces, provide the gateway as the default gateway for the subnet. |
[asm-vip-network1/mask1 via gateway1, asm-vip-network2/mask2 via gateway2] or [asm-vip-network1/mask1 via gateway1] |
avi.asm-static-routes.SE: [169.254.1.0/24 via 10.160.102.1, 169.254.2.0/24 via 10.160.102.2] |
avi.asm-vnic-id.SE |
ID of the dedicated ASM vNIC and is typically 3 on CSP (vNIC0 is management interface, vNIC1 is data-in interface, and vNIC2 is data-out interface) |
numeric vNIC ID |
avi.asm-vnic-id.SE: '3' |
Instructions for Configuring the Cisco CSP
The following are the steps to add a dedicated ASM vNIC on an existing SE CSP service. In this example, vNIC 3, which is actually the fourth NIC on the CSP service is used.
Navigate to to power off the SE service on Cisco CSP.
To add a new vNIC to the SE with desired parameters, navigate to , click on Add vNIC and provide VLAN ID, VLAN type, VLAN tagged, network Name, Model etc., and click on the Submit button.
Navigate to and select Power On to power on the SE service on Cisco CSP.
Instructions for Configuring the NSX Advanced Load Balancer Service Engine
Perform the following steps on the SE using bash shell.
SSH to the SE IP and perform the following steps:
ssh admin@<SE-MGMT-IP> bash# bash# sudo su bash# /opt/avi/scripts/stop_se.sh bash# mv /var/run/avi/ovf_properties.saved /home/admin
Do not copy this file; move instead. Edit it to provide the three comma-separated ASM-dedicated NIC related parameters. The file looks as follows:
bash# cat /home/admin/ovf_properties.saved AVICNTRL: 10.128.2.18, AVICNTRL_AUTHTOKEN: 1403771c- fc59-4d76-89b2-b3c35682b342, avi.default-gw.SE: 10.128.2.1, avi.asm-ip.SE: 10.160.102.227/24, avi.asm-static-routes.SE: [169.254.1.0/24 via 10.160.102.1, 169.254.2.0/24 via 10.160.102.2], avi.asm-vnic-id.SE: '3', avi.mgmt-ip.SE: 10.128.2.27, ovf_source: CSP, uuid: FCE9B12D-A1B0-4EF3-B922-BDC2A5F8AA11} bash# cp /home/admin/ovf_properties.saved /etc/ovf_config bash# /opt/avi/scripts/start_se.sh
Verify that the dedicated vNIC information is applied correctly and the ASM virtual service IPs are reachable via this interface. In this case, the interface eth3 is the dedicated ASM interface and it is configured with IP 10.160.102.227/24.
bash# ssh admin@<SE-MGMT-IP>
bash# ifconfig eth3
eth3 Link encap:Ethernet HWaddr 02:6a:80:02:11:05
inet addr:10.160.102.227 Bcast:10.160.102.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:4454601 errors:0 dropped:1987 overruns:0 frame:0
TX packets:4510346 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:672683711 (672.6 MB) TX bytes:875329395 (875.3 MB)
bash# ip route
default via 10.128.2.1 dev eth0
10.128.2.0/24 dev eth0 proto kernel scope link src 10.128.2.27
10.160.102.0/24 dev eth4 proto kernel scope link src 10.160.102.227
169.254.1.0/24 via 10.160.102.1 dev eth3
169.254.2.0/24 via 10.160.102.2 dev eth3
bash# ping -I eth3 <ASM-VIP>
ping -I eth3 169.254.1.10
PING 169.254.1.10 (169.254.1.10) from 10.160.102.227 eth3: 56(84) bytes of data.
64 bytes from 169.254.1.10: icmp_seq=1 ttl=62 time=0.229 ms
