This topic tells you how to prepare your network for IPsec for VMware Tanzu, create an IPsec runtime config, and add IPsec to your deployment.

To install IPsec, you must do the following:

  1. Configure Network Security
  2. Create the IPsec Manifest
  3. Download and Deploy IPsec
  4. Verify Your IPsec Installation

Prerequisites

Before you install IPsec, you must have the following:

  • Stemcell v1.93 or later for deployments that use the Ubuntu Jammy Stemcell.

  • Google Cloud Platform (GCP), vSphere, Azure, Amazon Web Services (AWS), or OpenStack as your IaaS.

  • In your IaaS, external load balancers that are not Network Load Balancers (NLB). IPsec does not work with NLBs. For more information, see IPsec Load Balancer Issues.

  • Tanzu Operations Manager operator administration rights

  • BOSH deployed through Tanzu Operations Manager v1.8 or later.

  • The Maximum Transmission Unit (MTU) for your IaaS set in the VMware Tanzu Application Service for VMs tile. You can find the Applications Network Maximum Transmission Unit (MTU) (in bytes) setting by navigating to Tanzu Operations Manager > TAS for VMs > Networking.

VMware recommends the following MTU values:

IaaS Recommended MTU Value
GCP 1354
Azure 1438
AWS Default value
vSphere Default value
OpenStack Follow the recommendations of your Neutron ML2 plugin provider, or empirically test the correct MTU for your environment.

Best Practices

  • IPsec might affect the functionality of other service tiles. As a result, VMware recommends deploying TAS for VMs and each service tile to different isolated subnets. Alternatively, you can minimally deploy all service tiles to a single isolated subnet, apart from the TAS for VMs subnet. Some service tiles do not support IPsec and must be placed in a non-IPsec subnet.
  • For IPsec on Linux VMs, VMware recommends any Ubuntu stemcells for vSphere, OpenStack, and HVM stemcells for AWS. These stemcells are available on Broadcom Support. If you use PV stemcells obtained from bosh.io, see the Packet Loss section of the Troubleshooting IPsec for VMware Tanzu topic to adjust MTU values.

Configure Network Security

To configure network security, do one of the following procedures for your IaaS:

Configure Network Security for Google Cloud Platform

To configure your Google Cloud Platform (GCP) environment for IPsec, do the following:

  1. Navigate to the Networking section of the GCP Console.
  2. Click Firewall rules.
  3. Click Create Firewall Rule.
  4. For Name, enter ipsec.
  5. For Network, select the network where Tanzu Operations Manager is deployed. For example, opsmgr.
  6. For Source filter, select Allow from any source (0.0.0.0/0).
  7. For Allowed protocols and ports, enter udp:500; ah; esp.
  8. Click Create.
  9. Adjust the MTU value to 1354 by performing the procedure in the Packet Loss section of the Troubleshooting IPsec for VMware Tanzu topic.

Configure Network Security for vSphere

Note IPsec is not compatible with VMware NSX-T Container Plug-in for VMware Tanzu Application Service for VMs.

To configure your vSphere environment for IPsec, confirm that your network allows the protocols listed in the table below:

Protocol Name Protocol Number Port(s)
AH 51 Any
ESP 50 Any
UDP 17 500

Configure Network Security for Azure

To configure your Azure environment for IPsec, do the following:

  1. Confirm that your network allows the protocols listed in the table below.

    Protocol Name Protocol Number Port(s)
    AH 51 Any
    ESP 50 Any
    UDP 17 500

  2. Adjust the MTU value to 1438. For instructions, see Explanation: Packet Loss.

Configure Network Security for AWS

To configure your AWS environment for IPsec, do the following:

  1. Go to EC2 Dashboard > Security Groups.

  2. Select the Security Group with the description TAS for VMs VMs Security Group and click Edit.

  3. Create the following Inbound Rules.

    Type Protocol Name Protocol Number Port Range Source
    Custom Protocol AH 51 All 10.0.0.0/16
    Custom Protocol ESP 50 All 10.0.0.0/16
    Custom UDP Rule UDP 17 500 10.0.0.0/16

    Note The default TAS for VMs VMs Security Group is typically specified with a subnet of 10.0.0.0/16. If your TAS for VMs subnet is deployed to a different CIDR block, adjust the source as needed.

Configure Network Security for OpenStack

The following network configuration is optimized for Mirantis OpenStack, but other OpenStack distributions have a similar workflow.

To configure your Mirantis OpenStack environment for IPsec, perform the following steps:

  1. Go to Project / Access & Security.

  2. Select the security group and click Manage Rules.

  3. Create the following Ingress and Egress Rules. Adjust the source CIDR as needed for your environment.

    Protocol Name Protocol Number Port Range Source
    ESP 50 Any 0.0.0.0/0
    AH 51 Any 0.0.0.0/0
    UDP 17 500 0.0.0.0/0

Create the IPsec Manifest

To add IPsec to VMs in your deployment, you must create a runtime config file named ipsec-addon.yml. The ipsec-addon.yml configures IPsec add-on properties for Linux VMs.

To create the IPsec manifest, add IPsec to Linux VMs in your deployment:

  1. Create an IPsec runtime config file called ipsec-addon.yml and add the following YAML:

    releases:
- name: ipsec
  version: 1.x.x
addons:
- name: ipsec-addon
  jobs:
  - name: ipsec
    release: ipsec
    properties:
      ipsec:
        optional: false
        ipsec_subnets:
        - 10.0.1.1/20
        no_ipsec_subnets:
        - 10.0.1.10/32  # bosh director
        - 10.0.1.4/32 # ops managers
        instance_certificate: |
          -----BEGIN CERTIFICATE-----
          EXAMPLExCERTIFICATExEXAMPLExCERTIFICATExEXAMPLExCERTIFICATE
          ...
          -----END CERTIFICATE-----
        instance_private_key: |
          -----BEGIN EXAMPLE RSA PRIVATE KEY-----
          EXAMPLExRSAxPRIVATExKEYxDATAxEXAMPLExRSAxPRIVATExKEYxDATA
          ...
          -----END EXAMPLE RSA PRIVATE KEY-----
        ca_certificates:
          - |
            -----BEGIN CERTIFICATE-----
            EXAMPLExCERTIFICATExEXAMPLExCERTIFICATExEXAMPLExCERTIFICATE
            ...
            -----END CERTIFICATE-----
          - |
            -----BEGIN CERTIFICATE-----
            EXAMPLExCERTIFICATExEXAMPLExCERTIFICATExEXAMPLExCERTIFICATE
            ...
            -----END CERTIFICATE-----
        prestart_timeout: 30
        stop_timeout: 30
        esp_proposals: aes128gcm16!
        ike_proposals: aes128-sha256-modp2048!
        log_level: 1
        ike_version: ike
        optional_warn_interval: 1
        force_udp_encapsulation: false
        instance_certificate_info_period: 30
        instance_certificate_warn_period: 14
        instance_certificate_error_period: 7
        instance_certificate_critical_period: 3
  include:
    stemcell:
    - os: ubuntu-xenial
    - os: ubuntu-jammy

  2. Replace the values listed in the template above as follows:

    Property Instructions
    version Specify the version number of your IPsec download from Broadcom Support.
    optional This value makes IPsec enforcement optional. To add IPsec to an existing deployment, set this flag to true. After IPsec has been successfully installed, set this flag back to false and redeploy.

    CautionCommunication between existing components fails if you try to add IPsec to an existing deployment without setting optional to true.

    ipsec_subnets List the subnets that you want to be encrypted. You can include the entire deployment to encrypt all traffic or a portion of the network. Encrypt any network that handles business-sensitive data.
    no_ipsec_subnets List the IP address of your BOSH Director and Tanzu Operations Manager VM, along with any other IP addresses in your deployment to which you want to communicate without encryption. VMware recommends that you list the subnets that are used for TAS for VMs managed services. Subnets for TAS for VMs managed services that do not support IPsec, such as Tanzu Operations Manager or service tiles, must be listed under no_ipsec_subnets and installed in a separate network.

    Note If you have an external load balancer such as F5, add it to the no_ipsec_subnets property. If you want to include it in the ipsec_subnet, you must configure it manually.

    Caution IPs that are not in ipsec\_subnets or no\_ipsec\_subnets have no default behavior and cannot communicate with other internal VMs. You must specify internal IPs in ipsec_subnets or no_ipsec_subnets.

    Caution In GCP, if you use the default router for DNS instead of the Google public DNS at 8.8.8.8, you must add the IP address of the default router in your subnet to no_ipsec_subnets. For example, 10.0.0.1/32.

    instance_certificate Copy in the signed certificate to be used by all your instance VMs. You must use one of the CAs in the ca_certificates property to sign this certificate.

    VMware recommends that you use a self-signed certificate. For more information, see Generate a Self-Signed Certificate below.
    instance_private_key Copy in the private key that corresponds to the instance_certificate above. This key must not use a pass phrase.

    ca_certificates Copy in CA certificates for the instance VM to trust during the validation process. In most cases, you only need the CA certificate used to sign the instance certificate. During CA credential rotation, you need two CA certificates.

    IPsec v1.8.12 and later supports the CA certificate chain. Concatenate the contents of the root and the intermediate certificates as one of the list items in ca_certificates, with the root CA at the top: 
    ca_certificates:
     - |
       -----BEGIN CERTIFICATE-----
       ... <ROOT>
       -----END CERTIFICATE-----
       -----BEGIN CERTIFICATE-----
       ... <INTERMEDIATE 1 ISSUED BY
       THE ROOT CERT ABOVE>
       -----END CERTIFICATE-----
       -----BEGIN CERTIFICATE-----
       ... <INTERMEDIATE 2 ISSUED BY
       THE INTERMEDIATE 1 ABOVE
       ... AND SIGNS THE INSTANCE CERT>
       -----END CERTIFICATE-----
     - |
       -----BEGIN CERTIFICATE-----
       ... <SECOND ROOT>
       -----END CERTIFICATE-----

    Important The root and the intermediate certificates cannot have the same subjectName, also called the common name and set with CN=. The root must be the first certificate of the chain.

    prestart_timeout Use this to modify the 30-second default prestart timeout value. This value limits the number of seconds allowed for IPsec to start before failing the attempt.
    stop_timeout Use this to modify the 30-second default post stop timeout value. When IPsec stops, communication between VMs is not encrypted and might cause errors. By default, IPsec waits 30 seconds for all other running jobs to stop before stopping itself.
    log_level You can specify the IKE daemon numerical log level, ranging from -1 to 4. For more information, see Logger Configuration in the strongSwan documentation.
    optional_warn_interval The interval in hours of warning when optional property is set to true. It prints the warning message {Date} - IPsec is set to "Optional" in the file /var/vcap/sys/log/ipsec/ipsec.stdout.log for Linux.
    force_udp_encapsulation Available on Linux-only deployments. If set to true it forces UDP encapsulation for ESP packets.

    Caution Setting this property to true in mixed deployments causes the deployment to fail. If you do not have a Linux-only deployment, you must set force_udp_encapsulation to false.

    instance_certificate
    _info_period    		 If the instance certificate expires during the set period in days, the IPsec release writes an [INFO] message in the logs.
    instance_certificate
    _warn_period    		 If the instance certificate expires during the set period in days, the IPsec release writes a [WARN] message in the logs.
    instance_certificate
    _error_period    		 If the instance certificate expires during the set period in days, the IPsec release writes an [ERROR] message in the logs.
    instance_certificate
    _critical_period    		 If the instance certificate expires during the set period in days, the IPsec release writes a [CRITICAL] message in the logs.

(Optional) Configure Custom Deployment Proposals

The manifest templates previously shown configure a default proposal for the ipsec-addon.yml.

If you want to use different proposals, modify the ipsec-addon.yml using the following table:

  1. Select the encryption type from the first row.

    Encryption Type Linux (ipsec-addon)
    ike_proposals esp_proposals
    128 Bit Encryption (default) 
    aes128-sha256-modp2048!
    		 
    aes128gcm16!
    256 Bit Encryption 
    aes256-sha256-modp2048!
    		 
    aes256gcm16!

  2. Copy the properties from that row into ipsec-addon.yml accordingly. See the previous ipsec-addon.yml file example.

    Property Instructions
    ike_proposals You can modify the IKE (Main Mode) encryption and integrity algorithms, and the Diffie-Hellman group. The default, aes128-sha256-modp2048!, is 128 bit AES-CBC for encryption, SHA2_256_128 HMAC for integrity, and Group 14 for Diffie-Hellman.
    esp_proposals You can modify the ESP (Quick Mode) encryption and integrity algorithms. The default, aes128gcm16!, is 128 bit AES-GCM with 128 bit ICV for both encryption and integrity.

Download and Deploy IPsec

To download the IPsec binary, add your IPsec runtime config to your BOSH manifest, and deploy IPsec, complete the following steps:

Breaking change If you are using TAS for VMs v1.12 or later, you must use named runtime configs. If you have not already split your runtime config into multiple named files, do so before upgrading IPsec for VMware Tanzu. For general information about named runtime config files, see Generic Configs in the BOSH documentation.

  1. Download the IPsec software binary from the Broadcom Support to your local machine.

  2. Copy the software binary to your Tanzu Operations Manager instance.

    $ scp -i PATH-TO-PRIVATE-KEY ipsec-release.tar.gz ubuntu@YOUR-OPS-MANAGER-VM-IP:

  3. Copy the IPsec runtime config file to your Tanzu Operations Manager instance.

    $ scp -i PATH-TO-PRIVATE-KEY ipsec-addon.yml ubuntu@YOUR-OPS-MANAGER-VM-IP:

  4. SSH into Tanzu Operations Manager.

    $ ssh -i PATH-TO-PRIVATE-KEY ubuntu@YOUR-OPS-MANAGER-VM-IP

  5. On the Tanzu Operations Manager VM, navigate to the software binary location in your working directory.

    $ cd PATH-TO-BINARY

  6. Log in to the BOSH Director.

    1. On the Tanzu Operations Manager VM, create an alias in the BOSH CLI for your BOSH Director IP address. For example: 
      $ bosh alias-env my-env -e 10.0.0.3
    2. Log in to the BOSH Director, specifying the newly created alias. For example: 
      $ bosh -e my-env log-in

  7. Upload your release, specifying the path to the tarballed IPsec binary, by running the following command:

    $ bosh -e my-env upload-release PATH-TO-BINARY/BINARY-NAME.tar

  8. List the releases by running the following command, and confirm that the IPsec binary file appears:

    $ bosh -e my-env releases

  9. Download your current runtime config and save as bosh-manifest.yml by running the following command:

    $ bosh -e my-env runtime-config > bosh-manifest.yml

  10. Update your runtime configuration to include IPsec.

    $ bosh -e my-env update-runtime-config --name=ipsec PATH/bosh-manifest.yml

  11. Verify your runtime configuration changes match what you specified in the IPsec manifest file.

     $ bosh -e my-env runtime-config --name=ipsec

    For example:

      $ bosh -e my-env runtime-config --name=ipsec
  Acting as user 'admin' on 'micro'

  releases:
   - {name: ipsec, version: 1.9.9}

  addons:
   name: ipsec-addon
    jobs:
    - name: ipsec
      release: ipsec
      ...

  12. If you have already deployed TAS for VMs or are adding IPsec to an existing deployment:

    1. Set the optional flag to true.
    2. Go to your Installation Dashboard in Tanzu Operations Manager.
    3. If you are using Tanzu Operations Manager v2.3 or later, click Review Pending Changes. For more information about this Tanzu Operations Manager page, see Reviewing your pending product changes in Tanzu Operations Manager.
    4. Click Apply Changes
    5. Wait for the installation to complete.
    6. Set the optional flag to false.
    7. Update the runtime config. 
      $ bosh -e my-env update-runtime-config --name=ipsec PATH/bosh-manifest.yml

  13. Go to your Installation Dashboard.

  14. If you are using Tanzu Operations Manager v2.3 or later, click Review Pending Changes.

  15. Click Apply Changes.

  16. If the TAS for VMs tile is not yet installed:

    1. Go to your Installation Dashboard in Tanzu Operations Manager.
    2. If you are using Tanzu Operations Manager v2.3 or later, click Review Pending Changes.
    3. Click Apply Changes
    4. Deploy TAS for VMs by following the installation instructions for your IaaS. For more information, see Installing Tanzu Operations Manager overview.

  17. The bosh-manifest.yml and ipsec-addon.yml files contain sensitive information. When the deployment process is completed, remove any unneeded copies of these files from the local workstation. VMware recommends that you use encryption or logical access controls to secure any archival copies of manifest files you want to retain.

Verify Your IPsec Installation

After you install IPsec and deploy TAS for VMs, complete the following steps to verify your IPsec installation:

  1. List the job VMs in your deployment by running the following command:

    bosh -e BOSH-ENVIRONMENT vms

  2. Open an SSH connection into the VM, using the job name and index of any VM found above, by running the following command:

    bosh -e BOSH-ENVIRONMENT -d DEPLOYMENT-NAME ssh JOB-NAME/INDEX

    Note The exact VM does not matter, because installing IPsec loads IPsec on all VMs deployed by Tanzu Operations Manager.

  3. Enter the root environment with root privileges by running:

    sudo su -

  4. Confirm that your ipsec job is listed as a bosh job by running:

    monit summary

    For example:

    $ monit summary
The Monit daemon 5.2.5 uptime: 18h 32m
...
Process 'ipsec'                     running
System 'system_localhost'           running

  5. Confirm that IPsec is running by using the following commands to get uptime and connections information:

    • For IPsec v1.9.40 and later, run these commands:

      /var/vcap/packages/strongswan/sbin/swanctl --stats

      /var/vcap/packages/strongswan/sbin/swanctl --list-conns

      For example:

      $ /var/vcap/packages/strongswan/sbin/swanctl --stats
plugin 'openssl' failed to load: libcrypto.so.1.0.2: cannot open shared object file: No such file or directory
uptime: 79 minutes, since Feb 17 16:25:20 2023
worker threads: 16 total, 11 idle, working: 4/0/1/0
job queues: 0/0/0/0
jobs scheduled: 12
IKE_SAs: 4 total, 0 half-open
mallinfo: sbrk 2494464, mmap 0, used 797184, free 1697280
loaded plugins: charon aes sha1 sha2 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pem gmp xcbc cmac hmac attr kernel-netlink socket-default vici openssl

$ /var/vcap/packages/strongswan/sbin/swanctl --list-conns
plugin 'openssl' failed to load: libcrypto.so.1.0.2: cannot open shared object file: No such file or directory
ipsec: IKEv2, no reauthentication, rekeying every 14400s, dpd delay 10s
  local:  10.0.4.8/32
  remote: 10.0.0.0/8
  local public key authentication:
    id: CN=ipsec_addon_instance_cert_1
    certs: CN=ipsec_addon_instance_cert_1
  remote public key authentication:
  ipsec: TRANSPORT, rekeying every 3600s, dpd action is restart
    local:  10.0.4.8/32
    remote: 10.0.0.0/8
no-ipsec: IKEv1/2, no reauthentication, rekeying every 14400s
  local:  10.0.4.8/32
  remote: 10.0.8.0/24
  remote: 10.0.0.5/32
  remote: 10.0.0.2/32
  remote: 10.0.12.0/24
  local unspecified authentication:
  remote unspecified authentication:
  no-ipsec: PASS, no rekeying
    local:  10.0.4.8/32
    remote: 10.0.8.0/24 10.0.0.5/32 10.0.0.2/32 10.0.12.0/24
$

    • For IPsec v1.9.39 and earlier, run:

      PATH-TO-IPSEC/ipsec statusall

      For example:

      $ /var/vcap/packages/strongswan-5.6.3/sbin/ipsec statusall
Status of IKE charon daemon (strongSwan 5.6.3, Linux 3.19.0-56-generic, x86_64):
  uptime: 18 hours, since Mar 16 23:58:50 2016
  malloc: sbrk 2314240, mmap 0, used 1182400, free 1131840
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 206
  loaded plugins: charon aes sha1 sha2 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs8 pkcs12 pem gmp xcbc cmac hmac attr kernel-netlink socket-default stroke
Listening IP addresses:
  10.10.5.66
Connections:
ipsec-10.10.4.0/24:  %any...%any  IKEv1/2
ipsec-10.10.4.0/24:   local:  [CN=test-cert-1-ca-1] uses public key authentication
ipsec-10.10.4.0/24:    cert:  "CN=test-cert-1-ca-1"
ipsec-10.10.4.0/24:   remote: uses public key authentication
ipsec-10.10.9.0/24:   child:  10.10.5.66/32 === 10.10.9.0/24 TRANSPORT
no-ipsec-10.10.4.1/32:  %any...%any  IKEv1/2
no-ipsec-10.10.4.1/32:   local:  uses public key authentication
no-ipsec-10.10.4.1/32:   remote: uses public key authentication
no-ipsec-10.10.4.1/32:   child:  dynamic === 10.10.4.1/32 PASS
Shunted Connections:
no-ipsec-10.10.4.1/32:  dynamic === 10.10.4.1/32 PASS
no-ipsec-10.10.5.1/32:  dynamic === 10.10.5.1/32 PASS
no-ipsec-10.10.6.1/32:  dynamic === 10.10.6.1/32 PASS
Routed Connections:
ipsec-10.10.9.0/24{6}:  ROUTED, TRANSPORT, reqid 6
ipsec-10.10.9.0/24{6}:   10.10.5.66/32 === 10.10.9.0/24
ipsec-10.10.8.0/24{5}:  ROUTED, TRANSPORT, reqid 5
ipsec-10.10.4.0/24{1}:   10.10.5.66/32 === 10.10.4.0/24
Security Associations (45 up, 0 connecting):
ipsec-10.10.4.0/24[459]: ESTABLISHED 13 seconds ago, 10.10.5.66[CN=test-cert-1-ca-1]...10.10.4.38[CN=test-cert-1-ca-1]
ipsec-10.10.4.0/24{1527}:   10.10.5.66/32 === 10.10.4.38/32
...

Generate a Self-Signed Certificate with OpenSSL

Follow these steps to generate a self-signed certificate for your IPsec manifest:

  1. Download the openssl-create-ipsec-certs.sh bash script from the pivotal-cf/docs-addon-ipsec public repository on GitHub.
  2. Go to the directory where you downloaded the script: 
    $ cd ~/workspace
  3. Change the permissions of the script: 
    $ chmod u+x openssl-create-ipsec-certs.sh
  4. Run the script: 
    $ ./openssl-create-ipsec-certs.sh
  5. This generates four files in a new certs directory where the script is run:
    • pcf-ipsec-ca-cert.pem — this value can be used as the CA Cert in the ca_certificates manifest field.
    • pcf-ipsec-ca-key.pem — the key used to sign the generated CA Cert.
    • pcf-ipsec-peer-key.pem — this value can be used as the instance private key in the instance_private_key manifest field.
    • pcf-ipsec-peer-cert.pem — this value can be used as the instance certificate in the instance_certificate manifest field.
  6. Because this certificate expires in 365 days, set a calendar reminder to rotate the certificate within the year. For instructions on changing certificates, see Rotating IPsec Certificates.
check-circle-line exclamation-circle-line close-line
Scroll to top icon