Restore the Credential Escrow Gateway on another location with minimal steps through a unique disaster recovery procedure. When enabling Disaster Recovery in Escrow Gateway, we use an Active Passive setup with a common network file store. At any time, both active and passive servers cannot be turned on and be mounted to the NFS mountpoint (passive node must be in an off state). Other configurations lead to a loss of requests and possibly a corruption of data in the shared store. Currently disaster recovery is only supported from Credential Escrow Gateway 1.2.0.

Server type

OS

Version

IP

DNS

Mount points

NFS Server

Ubuntu

18.04.4 LTS (Bionic Beaver)

172.16.84.234

/home/eg/redis/data

/home/eg/composeconfig

EG-Active Server

Photon

EUC Credential Escrow Gateway 1.2.0

172.16.70.52

https://beta1eg.ssdevrd.com/

/opt/vmware/docker/ceg/redis/data

/opt/vmware/docker/ceg/compose-config

EG- Passive Server

Photon

EUC Credential Escrow Gateway 1.2.0

172.16.70.128

https://beta2eg.ssdevrd.com/

/opt/vmware/docker/ceg/redis/data

/opt/vmware/docker/ceg/compose-config

Workspace ONE UEM environment

Windows

2008

172.16.99.159

https://egmma2007.ssdevrd.com/

N/A

Active Server Setup

NFS Server Setup

  1. Create mount directories with appropriate permissions (Redis and other services running on EG requires read and write permission for user 1001 to read/write to the mounted folders).
    mkdir -p /home/eg/redis/data
chown -R 1001:1001/home/eg/redis/data
chmod -R 755/home/eg/redis/data

mkdir -p /home/eg/composeconfig
chown -R 1001:1001/home/eg/composeconfig
chmod -R 755/home/eg/composeconfig
  2. Copy the initial configuration information from EG's active server to NFS server. Only copy once for a given NFS server. The following example uses SCP for the copy.
    scp -r /opt/vmware/docker/ceg/compose-config/* admin@172.16.84.234:/home/eg/composeconfig
  3. Update /etc/exports
    vi /etc/exports

    Add the following lines to the end of the file by specifying the IP of Active EG server

    /home/eg/redis/data  172.16.70.52(rw,sync,no_subtree_check)
/home/eg/composeconfig  172.16.70.52(rw,sync,no_subtree_check)
  4. Restart NFS to apply changes
    service nfs-kernel-server restart

Credential Escrow Gateway Active Server Setup:

  1. Install nfs utils.
    Tdnf install nfs-utils
  2. Stop docker services.
    docker stack rm ceg
  3. Mount the file system to the NFS server by specifying the IP of the NFS server.
    mount -t nfs 172.16.84.234:/home/eg/redis/data /opt/vmware/docker/ceg/redis/data
mount -t nfs 172.16.84.234:/home/eg/composeconfig /opt/vmware/docker/ceg/compose-config
  4. Optional: Configure Encryption*. Refer to the escrow gateway configuration guide. With encryption enabled, the encryption configuration must be done on application startup and only once in the lifetime of a given EG setup. Configure both active and passive nodes to use the same encryption configuration which happens automatically when this setup guide is followed.
  5. Start docker services.
    docker stack deploy -c /opt/vmware/docker/ceg/docker-compose.yml ceg
  6. Configure the Workspace ONE UEM instance to point to the Active Server by calling the Escrow Gateway Configuration API with the url of Active EG server.
    curl --location --request PUT 'https://egmma2007.ssdevrd.com/api/system/groups/96429a7f-6f42-4a17-a451-d487633d2336/escrow-gateway-settings'\
--header 'Content-Type: application/json'\
--header 'aw-tenant-code: XfGiwT8DxsMOopVdtJztHKc8b4DjiSknHF4cpdCQ9EU='\
--header 'Authorization: Basic YWRtaW5pc=='\
--data-raw '{ 
"gateway_url": "https://beta1eg.ssdevrd.com",
 "client_cert_thumbprint": <thumbprint>
}'

Fail-over Setup

NFS Server Setup

  1. Update /etc/exports.
    vi /etc/exports

    Change the IP to point to Passive EG server.

    /home/eg/redis/data  172.16.70.128(rw,sync,no_subtree_check)
/home/eg/composeconfig  172.16.70.128(rw,sync,no_subtree_check)
  2. Restart nfs to apply changes.
    service nfs-kernel-server restart

Credential Escrow Gateway Passive Server Setup:

  1. Install nfs utils.
    tdnf install nfs-utils
  2. Stop docker services.
    docker stack rm ceg
  3. Mount the file system to the NFS server by specifying the IP of the NFS server.
    mount -t nfs 172.16.84.234:/home/eg/redis/data /opt/vmware/docker/ceg/redis/data
mount -t nfs 172.16.84.234:/home/eg/composeconfig /opt/vmware/docker/ceg/compose-config
  4. Start docker services.
    docker stack deploy -c /opt/vmware/docker/ceg/docker-compose.yml ceg
  5. Configure the Workspace ONE UEM instance to point to the Passive Server by calling the Escrow Gateway Configuration API by specifying the url of Passive EG server.
    curl --location --request PUT 'https://egmma2007.ssdevrd.com/api/system/groups/96429a7f-6f42-4a17-a451-d487633d2336/escrow-gateway-settings' \
--header 'Content-Type: application/json' \
--header 'aw-tenant-code: XfGiwT8DxsMOopVdtJztHKc8b4DjiSknHF4cpdCQ9EU=' \
--header 'Authorization: Basic YWRtaW5pc==' \
--data-raw '{ 
"gateway_url": "https://beta2eg.ssdevrd.com",
"client_cert_thumbprint": <thumbprint>
}'

Escrow Gateway Outbound Proxy Configuration

Credential Escrow Gateway (CEG) makes outbound API calls to Workspace ONE UEM to deliver encrypted payloads for S/MIME certificates or to inform UEM about the availability of S/MIME certificates. These outbound calls are made over HTTPS directly to UEM SaaS endpoint. When you have a proxy server that is in the call path from EG to SaaS UEM API, you need to use the following configuration to enable using proxy server in EG.

The following steps need to be done from a terminal off the EG VA.

  1. Create an env file with this path /opt/vmware/docker/ceg/compose-config/proxy_settings.env. The content of this file looks like the following. Be sure to modify the values according to your environment.
    # Proxy configuration for containers
HTTP_PROXY=http://127.0.0.1:3001
HTTPS_PROXY=https://127.0.0.1:3001
NO_PROXY=*.example.com
  2. Modify /opt/vmware/docker/ceg/docker-compose.yml by inserting one new line for the new env file ONLY to this section
    completed-profile-consumer: image: 'vmware.uem.escrowgateway.completedprofileconsumer:latest' env_file: - /opt/vmware/docker/ceg/compose-config/redis.settings.env - /opt/vmware/docker/ceg/compose-config/statsd.settings.env - /opt/vmware/docker/ceg/compose-config/proxy_settings.env networks: - cegnet
    Note: Only the completed profile consumer needs this change.
  3. Stop and restart the CEG service.
    docker stack rm ceg
docker stack deploy -c /opt/vmware/docker/ceg/docker-compose.yml ceg