NSX Advanced Load Balancer Controller cluster instances are frequently required to be deployed from a common initial single Controller configuration that differs from the factory-default settings. This can be accomplished by creating a JSON file with the required object definitions and then using it to deploy subsequent controllers, which, as leaders, can then add followers to become clusters. You will have a set of identically initialized Controller clusters, ready to be individualized as needed.
Create setup.json
In most cases, these objects can be created by referring to the NSX Advanced Load Balancer REST API section.
The following example updates the system configuration by adding 8.8.8.8 to the DNS configuration:
{
"SystemConfiguration": [
{
"dns_configuration": {
"search_domain": "",
"server_list": [
{
"type": "V4",
"addr": "8.8.8.8"
}
]
}
}
]
}
In the case of complex objects such as the SSLKeyAndCertificate object, the JSON file can be created by running a diff command against two configuration files. In a typical deployment, generating setup.json on a test Controller environment is recommended. This generated file can then be used as a template for actual deployments. An NSX Advanced Load Balancer Controller configuration snapshot can be taken using the export CLI command.
> export configuration file before.cfg Please enter the passphrase to encrypt configuration: Downloaded the attachment to before.cfg Completed writing the export configuration to before.cfg
Configure objects using the UI or CLI, as required.
> export configuration file after.cfg Please enter the passphrase to encrypt configuration: Downloaded the attachment to after.cfg Completed writing the export configuration to after.cfg
Beyond this, configuration diff can be taken using a Python script.NSX Advanced Load Balancer has written expressly to customize initial configuration of another Controller.
/opt/avi/scripts/diff_config.py -f before.cfg -t after.cfg > setup.json
User passwords can be encrypted using the following code when creating setup.json with the User object.
/opt/avi/scripts/avi_passwd_tool.py --password admin --salt fF6ngAb3pvPgpbkdf2_sha256$100000$fF6ngAb3pvPg$ijkEue1M9fR/qsLVgzvPe7N0VvOxIjDiJVmK9NIx+0Q=$6$fF6ngAb3pvPg$CqAKtNRZtgXtJchrPmoxUgdLFM7rFGmta1tWb7sobQI4iSZAY2QuAOBNtboVGrmDYPMCvqXXH6lARr9RedCJT.
Deployment using setup.json
It is recommended to take a configuration backup before deploying the Controller using the setup.json file created by the Python script. Use the following command to create an encrypted backup of the existing configuration.
/opt/avi/python/bin/portal/manage.py export_configuration --file ~/setup-old.json --passphrase secret
For a Mesos/Bare-Metal Deployment
Copy setup.json to the persistent directory in the host mounted as /vol in the Controller container. If you are using the avi_baremetal script, the default location is /opt/avi/controller/data on the host. When deploying the Controller as a container, setup.json can be passed as an additional argument to the avi_baremetal_setup.py script. For example:
./avi_baremetal_setup.py -c -cc 4 -cm 12 -i 10.10.22.108 -m 10.10.22.108 --setup-json /root/configs/avi-setup.json
For a Controller Deployment as a Virtual Machine
Wait until the Controller comes up. Place the config file on the Controller as /var/lib/avi/etc/setup.json (note the filename). Upon reboot or fresh-start, the NSX Advanced Load Balancer Controller will self-configure using the provided setup.json file.
reboot
For an OpenStack Deployment
UserData config size is limited to 48 Kb if the size of setup.json is within allowable limits.
># cfgdrv userdata >nova boot --config-drive true --image avicontroller --key-name mykey --flavor 4 --user-data /root/my-avi-config.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 avicontrollers ># metasvc userdata >nova boot --config-drive false --image avicontroller --key-name mykey --flavor 4 --user-data /root/my-avi-config.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 avicontrollers
If the setup.json size is bigger then than the allowable limit, setup.json can be uploaded and referred to in the deployment phase.
User data can refer to the file using the “url” or “file” tag. Following is an example of my-avi-config-url.json with URL.
{
"META": {
"init_config": {
"url": "https://s3-us-west-2.amazonaws.com/avi-controller-configs/linuxserver-awsipam-setup.json"
}
}
}
Following is an example of my-avi-config-url.json with filepath.
{
"META": {
"init_config": {
"file": "/vol/linuxserver-awsipam-setup.json"
}
}
}
Following is an example of deployment.
># cfgdrv userdata indirection >nova boot --config-drive true --image avicontroller --key-name mykey --flavor 4 --user-data /root/my-avi-config-url.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 --min-count=3 --max-count=3 avicontrollers ># metasvc userdata indirection nova boot --config-drive false --image avicontroller --key-name mykey --flavor 4 --user-data /root/my-avi-config-url.json --nic net-id=7402bf4f-240f-4172-99c1-90000ea45f86 --min-count=3 --max-count=3 avicontrollers
For an AWS Deployment
UserData config size is limited to 16Kb. If the size of setup.json is within allowable limits, cut-paste the my-avi-config.json into the user-data section during launch from AWS Web Console.
# metasvc userdata ec2-run-instances ami-b7ea27d7 -f /root/my-avi-config.json -t c4.2xlarge -s subnet-62f1b707 -g sg-642d8d02
If the setup.json size is bigger than the allowable limit, cut-paste the my-avi-config-url.json into the user-data section during launch from AWS Web Console.
# metasvc userdata indirection ec2-run-instances ami-b7ea27d7 -f /root/my-avi-config-url.json -t c4.2xlarge -s subnet-62f1b707 -g sg-642d8d02
my-avi-config-url.json follows similar formats as discussed in the OpenStack section. Following is a sample my-avi-config-url.json file for the S3 bucket.
{
"META": {
"init_config": {
"s3": "avi-controller-configs/linuxserver-awsipam-setup.json"
}
}
}
For uploading setup.json on the S3 bucket:
Public : use the url style or s3 style
Private through RBAC on VM: use the s3 style. The VM role must have
s3:GetObjectaction allowed to be able to s3-get the object using IAM.Private through RBAC on S3-bucket: use the s3 style. The VM role must have AWS access. The S3 bucket must have permissions for the account or user or VM role to download the object.
Example Bucket Policy:
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:iam::139284885014:role/BM-AviController-Role",
"arn:aws:iam::139284885014:root"
]
},
"Action": "s3:*",
"Resource": "arn:aws:s3:::avi-controller-configs/*"
}
]
}
For an Azure Deployment
There are two ways to provide initial configuration in an Azure environment:
Using Azure CLI
Using ARM (Azure Resource Manager) template
Using Azure CLI
If the NSX Advanced Load Balancer Controller is deployed using Azure CLI, the JSON file can be provided during the deployment.
az vm create --resource-group rahulr-jenkins-resource-group --location centralus --image avi-networks:avi-vantage-adc:avi-vantage-adc-byol:17.2.7 --name Avi-Test-Controller --size Standard_F8s --subnet /subscriptions/<subscription_id>/resourceGroups/<resource_group_name>/providers/Microsoft.Network/virtualNetworks/<vnet_name>/subnets/rahulr-subnet --public-ip-address "" --nsg "" --custom-data ./initial_config.json
Using Azure ARM Template
If the NSX Advanced Load Balancer Controller is deployed using the ARM template, the JSON data is provided as the Custom Data on the Custom deployment page of the Azure portal. Navigate to .
Use the required JSON template in the Custom Data field. For reference, the below JSON template is for adding 8.8.8.8 to DNS configuration. Copy the JSON configuration mentioned below, and add it to the Custom Data field in the ARM template.
{
"SystemConfiguration": [
{
"dns_configuration": {
"search_domain": "",
"server_list": [
{
"type": "V4",
"addr": "8.8.8.8"
}
]
}
}
]
}
