This document introduces approaches to sync data onto airgap appliance.

Data flow operations

Airgap appliance supports sync, export, import, and rsync operations.

Login Airgap Appliance console

After OVA deployment, log in to the appliance via SSH. To execute the agctl operations, switch to root user.
Note: You will not be able to execute agctl command by “sudo” due to known ansible issues.

Sync

Sync operation will fetch data from remote repositories on the internet which include TCA images, helm charts, TKG images, and Photon packages.

Standard sync operation

  1. Use vim to edit user-inputs.yml.
  2. Modify parameters in products and build_sync.
    • products-versions: Define the specific release versions of TCA. This will sync images, Helm charts, and related TKG images of the TCA releases mentioned. For more details on the usability of this field, refer to the file /usr/local/airgap/scripts/vars/setup-user-inputs.yml.
    • Build_sync: Modify the build number to sync. It is the current build number of the TCA appliance.
      Note: Photon repos to be fetched irrespective of TCA release defined in product versions. 
      vi /usr/local/airgap/scripts/vars/user-inputs.yml
      products:
  - name: "tca"
    versions: "2.3.0-3.1.0"
 #  - name: "photon"
 #    baseline: "2.3.0"

 # TCA build. Format is [version]-[build number], enabled since TCA 3.0.0 release
build_sync: "3.1.0-12345678"

      Save the file and exit vim editor.

  3. Execute sync command, agctl sync.

    After initiating the sync operation, the process will be in a pending state and waiting for the user to input the harbor admin password to validate. Once done, the process will continue to start sync operation. Monitor all related logs in /usr/local/airgap/logs/ folder to get more information on the progress of image sync.

    Once the operation is completed, the ansible will end. The field failed=0 indicates that all processes and data sync operations are completed.

    TASK [Gathering Facts] *********************************************************
ok: [localhost]

TASK [Clear system wide harbor password] ***************************************
changed: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=297  changed=117  unreachable=0    failed=0    skipped=52   rescued=0    ignored=1

Sync images from local bom file(s) only

Parameter local_only enables syncing images from local bom file(s) only and can be used to sync images of a specific TCA build. Enabling this field will only sync the TCA images and skip the TKG image sync.
  1. Use vim to edit user-inputs.yml.
  2. Modify parameters in products and build_sync.
    • products-versions: Define the specific release of TCA images to sync that will sync TCA images, helm charts, and related TKG images.
    • Build_sync: Modify the build number to sync. It is the TCA build number which is deployed in the system to be used to create kubernetes clusters.
      Note: Photon repos are to be fetched irrespective of TCA release defined in product versions. 
      vi /usr/local/airgap/scripts/vars/user-inputs.yml
      products:
  - name: "tca"
    versions: "2.3.0-3.1.0"
 #  - name: "photon"
 #    baseline: "2.3.0"

 # TCA build. Format is [version]-[build number], enabled since TCA 3.0.0 release
build_sync: "3.1.0-12345678"
  3. Set parameter local_only to True.
    # local_only is a parameter that only syncs images in local bom. When True then sync images from local bom only
local_only: True
  4. Save the file and exit editor. Execute agctl sync. Location of log files is the same as standard sync operation.

Keep local BOM files when syncing images

Parameter keep_bom enables users to retain BOM files not to be deleted before run sync and export. The default value of the parameter is False, the existing BOM files will be deleted before run sync and export. Set the value to True and the files will not be deleted.
  1. Use vim to edit user-inputs.yml.
  2. Modify parameters in products and build_sync.
    • products-versions: Define the specific release of TCA images to sync that will sync TCA images, helm charts, and related TKG images.
    • Build_sync: Modify the build number to sync. It is the TCA build number which is deployed in the system to be used to create kubernetes clusters.
      Note: Photon repos are to be fetched irrespective of TCA release defined in product versions. 
      vi /usr/local/airgap/scripts/vars/user-inputs.yml
      products:
  - name: "tca"
    versions: "2.3.0-3.1.0"
 #  - name: "photon"
 #    baseline: "2.3.0"

 # TCA build. Format is [version]-[build number], enabled since TCA 3.0.0 release
build_sync: "3.1.0-12345678"
  3. Set value to keep_bom.
    # keep_bom is a parameter if you keep existing bom files before run sync and export operations.
 # The default value is False, which is to delete all existing bom files before run operations.
keep_bom: True

Set customized retry_times in user-inputs.yml

Users can define retry times when failing to download an image. The default value of the parameter is 3. Following are the steps to set customized retry times:

  1. Use vim to edit user-inputs.yml.
  2. Modify parameters in products and build_sync.
    • products-versions: Define the specific release of TCA images to sync that will sync TCA images, helm charts, and related TKG images.
    • Build_sync: Modify the build number to sync. It is the TCA build number which is deployed in the system to be used to create kubernetes clusters.
      Note: Photon repos are to be fetched irrespective of TCA release defined in product versions. 
      vi /usr/local/airgap/scripts/vars/user-inputs.yml
      products:
  - name: "tca"
    versions: "2.3.0-3.1.0"
 #  - name: "photon"
 #    baseline: "2.3.0"

 # TCA build. Format is [version]-[build number], enabled since TCA 3.0.0 release
build_sync: "3.1.0-12345678"
  3. Set value to retry_times.
  4. Save the file, exit editor, and then execute agctl sync. Location of log files is the same as standard sync operation.

Export

Export operation fetches data from remote repositories to local as sync operation. Instead of pushing images to local harbor, export downloads images into tar packages, packaged in a bundle for import.

Export full size bundle
  1. Use vim to edit user-inputs.yml.
  2. Share the same parameters with standard sync operation. Modify parameters in products and build_sync. When photon-baseline parameters are commented out, it will export a full size bundle.
    • products-versions: Define the specific release of TCA images to sync that will sync TCA images, helm charts, and related TKG images.
    • Build_sync: Modify the build number to sync. It is the TCA build number which is deployed in the system to be used to create kubernetes clusters.
      Note: Photon repos are to be fetched irrespective of TCA release defined in product versions.
  3. Save the file and exit editor. Kick-off command agctl export.

The process will be in a pending state. Input the harbor admin password to validate. Once validation passes, the process will continue to start export operation. All the related logs are available in /usr/local/airgap/logs/ folder and can be used to get more information on the progress of export.

Once the export is complete, ansible will end the playbook with failed=0 in summary log.

The exported bundle can be found in /photon-reps/export-bundle/.

The bundle can also be downloaded via web download tools from https://%3Cairgap-server-FQDN%3E/export-bundle/.

Export incremental bundle

For environments with synced/imported data in TCA 2.3 or 3.0, an incremental bundle can be exported for import. The overall steps are the same as exporting a full-size bundle, the only difference is to uncomment the photon baseline parameters in user-inputs.yml. 
products:
  - name: "tca"
    versions: "2.3.0-3.1.0"
  - name: "photon"
    baseline: "2.3.0"

When uncommenting the baseline parameter, only the photon packages created after the baseline will be synced to local. Compared to the full-size sync, this will save time and disk space.

Save the file and exit editor, run command agctl export. Logs can be found in the folder, /usr/local/airgap/logs/.

Export adhoc RAN BOM images

In addition to the export data bundle, RAN BOM bundle can also be exported.

  1. Use vim to edit user-inputs.yml.
  2. Modify parameters:
    • ran_bom_images: Set to True when exporting the RAN BOM bundle. Default value is False.
    • Ranbom_images_list: Comma separated list, default value is ranbom-tcp.3.0.0-image to export TCP 3.0 images.
  3. Save the file and exit editor, run command, agctl export. The process is the same as exporting a full-size bundle. Exported bundle will be generated as airgap-ranbom-export-bundle*tar.gz in /photon-reps/export-bundle/ran-bom folder.

Import

Import operation is designed to extract exported tar.gz bundle and upload all data onto the airgap server. Before executing the import operation, copy the exported bundle, .tar.gz file, into /photon-reps folder of the target airgap server.

Import Data bundle

User inputs yaml modification is not needed to import data bundle.
  1. Check availability of data bundle under /photon-reps folder.
    ls -lt /photon-reps
  2. Execute command, agctl import.
  3. Check logs under /usr/local/airgap/logs for more information on import operation.
Import RAN BOM bundle
  1. Copy the bundle to /photon-reps folder on target airgap server.
  2. Use vim to edit user-inputs.yml.
  3. Set ran_bom_images to True.
    # 8. Option for sync ran bom image
 # Since TCA 3.0 release support adhoc exports and imports of RAN BOM images.
 # Enable this to true for RAN BOM import to airgap harbor.
ran_bom_images: True
  4. Save file and exit editor, then kick-off command agctl import.
  5. Review the logs under /usr/local/airgap/logs/ folder to get more information.

Data replication from remote airgap server

Data replication can perform synchronization between airgap servers that is helpful in migrating data from legacy airgap servers to new airgap appliance.
  1. Use vim to edit user-inputs.yml.
  2. Modify the parameters under section 6. Options for remote sync.
    • remote_server_fqdn: Remote server’s FQDN, IP address applicable.
    • endpoint_name: Unique name for the remote registry.
    • username: Remote harbor’s username, default value is admin.
    • secret: Remote harbor’s secret.
    • remote_server_cert_file: Certificate file contains remote server’s harbor CA cert.
    • reg_des: Description of new endpoint.
    • policy_des: Description of new policy.
    • policy_name: Name of the new policy.
    • cron: Sync scheduler, by default, start sync per 30 mins.
  3. Save the file and exit editor.
  4. Create a remote airgap server certificate file which is specified in user-inputs.yml.
  5. Copy content in remote server’s CA certificate. 
    /etc/docker/certs.d/<airgap-server-FQDN>/ca.crt
  6. Paste the certificate content in the created airgap server certificate file in the target server.
  7. Execute the command, agctl rsync.
  8. Track all related logs in /usr/local/airgap/logs/ folder. You can use that to get more information on the progress of the remote sync.
  9. Verify that a new registry and a policy were created in the harbor UI, along with successful replication.