ENS notifications are applicable only for emails in the Inbox folder. The notifications do not work for Calendar events, sub folders, and so on. This topic describes the steps to troubleshoot the ENS2 notification issues for emails in the Inbox folder.

Public Key Request from the ENS

The Boxer application requests the public key from the ENS. The public key is used to encrypt the user credentials. When the ENS processes the request, the ENS sends the public key and creates a user record in the database against the user ID. In the following sample, the ENS logs for the GetPublicKeyRequest, the Boxer application sends the SHA256 hash of the email address as the user ID.

2019/10/18 05:54:05.395 WIN-HTCPEDXIUVF 7b21cd56-4c45-4a7c-88d9-a7f225cea3b9    [0000000-0000000]   (5)     Debug   MailNotificationService.Controllers.EnsController.GetPublicKey  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Processing Get Public key request for Userid[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] 
2019/10/18 05:54:05.457 WIN-HTCPEDXIUVF 7b21cd56-4c45-4a7c-88d9-a7f225cea3b9    [0000000-0000000]   (5)     Debug   MailNotificationService.BusinessImpl.GetPublicKeyBusiness.ProcessGetPublicKeyRequestAsync   User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Key generated for user id [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]   
2019/10/18 05:54:05.457 WIN-HTCPEDXIUVF 7b21cd56-4c45-4a7c-88d9-a7f225cea3b9    [0000000-0000000]   (5)     Debug   MailNotificationService.Controllers.EnsController.GetPublicKey  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Get Public Key request processed. HttpStatusCode:[OK] ResponseCode:[UpdateSuccess] 

The possible error types and solutions that you might see during a GetPublicKeyRequest is listed as follows:

Error: Unauthorized Request

If you see the following error when you send a GetPublicKeyRequest, then ensure that the provided API token is correct. Verify if the API token is the same at the following instances:

  • API token in the ENS logs - API token : [12341*********fasdf]
  • The Boxer application configuration in the UEM console. See, the Workspace ONE Boxer Admin Guide for more information on the Boxer application configuration values.
  • API token in the Boxer application logs - Verify the API token in the Boxer application logs.

Error: Unable to add a NULL value into the PublicKey column

Note: This section is applicable for an on-premises installation only.

When the available RSA keys in the database are exhausted, you might see the following error. This issue is automatically fixed when the RSAKey tracker service triggers and generates new keys again.

2019/10/18 12:20:04.121 WIN-HTCPEDXIUVF b4a42dc8-6896-4243-9a4c-8ed476ae94ab    [0000000-0000000]   (5)     Debug   MailNotificationService.Controllers.EnsController.GetPublicKey  User Id [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Processing Get Public key request for Userid[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] 
2019/10/18 12:20:04.136 WIN-HTCPEDXIUVF b4a42dc8-6896-4243-9a4c-8ed476ae94ab    [0000000-0000000]   (5)     Debug   MailNotificationService.BusinessImpl.GetPublicKeyBusiness.ProcessGetPublicKeyRequestAsync   User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Error: 515, Severity: 16, State: 2, Message: "Cannot insert the value NULL into column 'PublicKey', table 'onpremensdev.dbo.UserInfo'; column does not allow nulls. INSERT fails.", Procedure: "UserInfo_Save", Line: 39   
2019/10/18 12:20:04.136 WIN-HTCPEDXIUVF b4a42dc8-6896-4243-9a4c-8ed476ae94ab    [0000000-0000000]   (5)     Debug   MailNotificationService.Controllers.EnsController.GetPublicKey  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Get Public Key request processed. HttpStatusCode:[InternalServerError] ResponseCode:[UpdateFail]
Note: The RSAKey tracker trigger interval time is 120 minutes. If the number of keys available in the database during the tracker trigger time is less than 250, then the RSAKey tracker starts generating a new batch of RSA keys. By default, the RSAKey tracker generates 500 new keys at a time.

Ensure that the following values are present in the RSAKey tracker configuration file:

<add key="numberOfKeysToBeInserted" value="500"/>
<add key="wakeUpIntervalInMins" value="120"/>
<add key="keysThreshold" value="250"/>

Error: ENS service communication

When communicating with the ENS service, if you see the following error in the Boxer application logs, then ensure that your device has proper connectivity.

2019-11-04T12:23:12Z  E [710337] [ENS] An error occurred when communicating with the ENS service: Error Domain=NSURLErrorDomain Code=-1009 "The Internet connection appears to be offline." UserInfo={NSUnderlyingError=0x281fead90 {Error Domain=kCFErrorDomainCFNetwork Code=-1009 "The Internet connection appears to be offline." UserInfo={NSErrorFailingURLStringKey=https://ens-staging.getboxer.com/api/ens/getpublickey, NSErrorFailingURLKey=https://ens-staging.getboxer.com/api/ens/getpublickey, _kCFStreamErrorCodeKey=50, _kCFStreamErrorDomainKey=1, NSLocalizedDescription=The Internet connection appears to be offline.}}, NSErrorFailingURLStringKey=https://ens-staging.getboxer.com/api/ens/getpublickey, NSErrorFailingURLKey=https://ens-staging.getboxer.com/api/ens/getpublickey, _kCFStreamErrorDomainKey=1, _kCFStreamErrorCodeKey=50, NSLocalizedDescription=The Internet connection appears to be offline.} at URL: https://ens-staging.getboxer.com/api/ens/getpublickey. Data: . Response Code: 0
2019-11-04T12:23:12Z  E [710337] [ENS] Error registering new account: vmwUser4@awRed.onmicrosoft.com
Error:Error Domain=com.alamofire.serialization.response.error.response Code=-1 "invalid public key" UserInfo={NSLocalizedDescription=invalid public key}
2019-11-04T12:23:12Z  E [703318] [ENS] Error registering device for push notification
Error:Error Domain=com.alamofire.serialization.response.error.response Code=-1 "invalid public key" UserInfo={NSLocalizedDescription=invalid public key}
2019-11-04T12:23:12Z  E [726177] - Unexpected error: {
    BXLocalizedContextMessageErrorKey = "Could not update settings for the push notification service";
    BXLocalizedTitleErrorKey = "Could not update settings for the push notification service";
    NSLocalizedDescription = "Could not update settings for the push notification service. ";
    NSLocalizedFailureReason = "Failed to update push notification settings. Please contact your administrator.";
} context: 1

Register Device Request

The Boxer application sends a Register request to the ENS, a push subscription to the EWS, and a subscribe for notification. If the GetPublicKey request is successful, then the Boxer application sends a register request to the ENS with the necessary information required to register a device for notification.

Scenario 1: - If the EWS URL is not configured in the console, then the ENS tries autodiscovery to obtain the EWS URL to subscribe the user.

Scenario 2: - If the EWS URL is configured in the console, then the ENS uses the same EWS URL to subscribe the user.

When the subscription is successful, the ENS receives the [UserSubscribed] message with the subscription ID as mentioned in the following code snippet.

2019/11/05 08:18:49.674 A3  726c4072-5144-4450-848b-821f65174b89    [0000000-0000000]   (23)    Info    MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  User [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] subscribed with subscriptionId [JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAJ6RYazaIoUCfX7KheUsQYUQnw9rIYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4=]
2019/11/05 08:18:49.767 A3  726c4072-5144-4450-848b-821f65174b89    [0000000-0000000]   (28)    Debug   MailNotificationService.Controllers.EnsController.RegisterDeviceV2  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Register device request processed. HttpStatusCode:[OK] ResponseCode:[UserSubscribed]

In the Android Boxer logs, you must see the following log entries to confirm a successful registration:

--------------------------------------------------------------
    ENS SETTINGS
--------------------------------------------------------------
ENS_LINK_ADDRESS = https://ens.getboxer.com/api/ens
ENS_API_TOKEN = 17413**********************88c08
POLICY_ACCOUNT_NOTIFY_PUSH = true
EWS_URL =
ENS_STATE = (8 -> Registered)
 
--------------------------------------------------------------
    HEALTH STATUS
--------------------------------------------------------------
App version health status: Green, Current app version: 5.11.0.4, New version: 5.10.0
Sync Health Status: Green, Sync durations in seconds: [0.522, 0.49, 0.416, 0.379, 0.424, 0.368, 0.465, 0.496, 0.565, 1.344], Sync results [OK, OK, OK, OK, OK, OK, OK, OK, OK, OK]
Ens health status: Green , Ens state: Registered
Overall health status: Green
Ens registration for account (id=8) is successful!

For the iOS Boxer logs, you must see the following log entries to confirm a successful registration:

For normal subscription
2019-11-11T09:31:41Z  I [12347] [ENS] Successfully registered account.
Note: For iOS Boxer logs, open the Boxer application, navigate to the Boxer Settings, click the VMware Secure Email, and ensure the Use Push Service switch is enabled to confirm a successful ENS registration.

The possible errors and solutions that you might see when you are unable to locate the autodiscover services are listed as follows:

Error: Unable to Locate the Autodiscover Services

If you see the following error, then ensure to enable autodiscovery, check the availability and connectivity of the autodiscovery server using the EWSEditor and the MS remote connectivity analyzer.

2019/11/06 07:01:56.207 A3  d252be19-1c5d-4e30-9155-a0ae3a529679    [0000000-0000000]   (94)    Warn    MailNotificationService.BusinessImpl.SubscriptionBusiness.SubscribeV2Async  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Exception while auto discovery occured for userId [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], Exception Message [The Autodiscover service couldn't be located.] , Exception [Microsoft.Exchange.WebServices.Data.AutodiscoverLocalException: The Autodiscover service couldn't be located.
   at Microsoft.Exchange.WebServices.Autodiscover.AutodiscoverService.InternalGetLegacyUserSettings[TSettings](String emailAddress, List`1 redirectionEmailAddresses, Int32& currentHop)
   at Microsoft.Exchange.WebServices.Autodiscover.AutodiscoverService.GetLegacyUserSettings[TSettings](String emailAddress)
   at Microsoft.Exchange.WebServices.Autodiscover.AutodiscoverService.InternalGetLegacyUserSettings(String emailAddress, List`1 requestedSettings)
   at Microsoft.Exchange.WebServices.Data.ExchangeService.GetAutodiscoverUrl(String emailAddress, ExchangeVersion requestedServerVersion, AutodiscoverRedirectionUrlValidationCallback validateRedirectionUrlCallback)
   at Microsoft.Exchange.WebServices.Data.ExchangeService.AutodiscoverUrl(String emailAddress, AutodiscoverRedirectionUrlValidationCallback validateRedirectionUrlCallback)
   at MailNotificationService.BusinessImpl.ExchangeServiceBusiness.<GetExchangeServiceViaAutoDiscovery>d__10.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MailNotificationService.BusinessImpl.ExchangeServiceBusiness.<GetExchangeServiceAsync>d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at MailNotificationService.BusinessImpl.SubscriptionBusiness.<SubscribeV2Async>d__7.MoveNext()], Inner Exception [], Autodiscover url used [The Autodiscover service couldn't be located.]
2019/11/06 07:01:56.207 A3  d252be19-1c5d-4e30-9155-a0ae3a529679    [0000000-0000000]   (94)    Debug   MailNotificationService.Controllers.EnsController.RegisterDeviceV2  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Register device request processed. HttpStatusCode:[Conflict] ResponseCode:[SubscribeAgain]

Error: The remote server returned an error (403) Forbidden

If this error occurs during a subscription, then ensure to enter the proper EWS URL in the Boxer application KVP values of the UEM console. The EWSUrl used to subscribe must have the complete endpoint specified.

Example of a correct EWSUrl - [https://mail-mem13.xyz.com/EWS/exchange.asmx]

Example of an incorrect EWSUrl - [https://mail-xyz.com/]

To check the EWS URL availability and connectivity, check the EWSEditor and the MS remote connectivity analyzer.

2019/11/06 07:09:54.064 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.Controllers.EnsController.RegisterDeviceV2  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Processing register device request for Userid[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]
2019/11/06 07:09:54.080 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.RegisterDeviceBusiness.ProcessRegisterDeviceRequestAsyncV2 User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Exchange version sent by boxer [2] 
2019/11/06 07:09:54.080 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeServiceBusiness.GetExchangeServiceAsync    User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Using client ewsurl, mailServerUrlMatched : False, deletedEWSUrl: False
2019/11/06 07:09:54.080 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  EWSUrl used to subscribe: [https://mail-mem13.ssdevrd.com/]
2019/11/06 07:09:54.080 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  User subscribing with [Basic Auth] 
2019/11/06 07:09:54.173 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Warn    MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotifications   User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Service request exception occured  for userId [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], Inner exception message [The remote server returned an error: (403) Forbidden.] Going for a retry,   
2019/11/06 07:09:54.173 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  EWSUrl used to subscribe: [https://mail-mem13.ssdevrd.com/]
2019/11/06 07:09:54.173 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  User subscribing with [Basic Auth] 
2019/11/06 07:09:54.205 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Warn    MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotifications   User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Service request exception occured  for userId [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], Inner exception message [The remote server returned an error: (403) Forbidden.] Going for a retry,   
2019/11/06 07:09:54.205 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  EWSUrl used to subscribe: [https://mail-mem13.ssdevrd.com/]
2019/11/06 07:09:54.205 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  User subscribing with [Basic Auth] 
2019/11/06 07:09:54.236 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Warn    MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotifications   User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Service request exception occured  for userId [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], Inner exception message [The remote server returned an error: (403) Forbidden.] Going for a retry,   
2019/11/06 07:09:54.236 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  EWSUrl used to subscribe: [https://mail-mem13.ssdevrd.com/]
2019/11/06 07:09:54.236 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  User subscribing with [Basic Auth] 
2019/11/06 07:09:54.251 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Warn    MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotifications   User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Service request exception occured  for userId [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], Inner exception message [The remote server returned an error: (403) Forbidden.] Going for a retry,   
2019/11/06 07:09:54.251 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Warn    MailNotificationService.BusinessImpl.SubscriptionBusiness.SubscribeV2Async  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Service request exception occured  for userId [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], Inner exception message [The remote server returned an error: (403) Forbidden.] Going for a retry,   
2019/11/06 07:09:54.251 A3  f43eb3d0-e173-49de-9b52-3acb8a1107c4    [0000000-0000000]   (98)    Debug   MailNotificationService.Controllers.EnsController.RegisterDeviceV2  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Register device request processed. HttpStatusCode:[Conflict] ResponseCode:[SubscribeAgain] 

Sample error logs of Boxer during registration:

2019-11-11T09:13:43Z  E [9326] [ENS] An error occurred when communicating with the ENS service: Error Domain=com.alamofire.error.serialization.response Code=-1011 "Request failed: conflict (409)" UserInfo={NSLocalizedDescription=Request failed: conflict (409), NSErrorFailingURLKey=https://a3.ssdevrd.com/mailnotificationservice/api/ens/registerdevicev2, com.alamofire.serialization.response.error.data={length = 135, bytes = 0x7b227265 73706f6e 7365436f 6465223a ... 4f6e5072 656d227d }, com.alamofire.serialization.response.error.response=<NSHTTPURLResponse: 0x282db1fa0> { URL: https://a3.ssdevrd.com/mailnotificationservice/api/ens/registerdevicev2 } { Status Code: 409, Headers {
    "Content-Length" =     (
        135
    );
    "Content-Type" =     (
        "application/json; charset=utf-8"
    );
    Date =     (
        "Mon, 11 Nov 2019 09:13:40 GMT"
    );
    Server =     (
        "Microsoft-IIS/8.5"
    );
    "X-Powered-By" =     (
        "ASP.NET"
    );
} }} at URL: https://a3.ssdevrd.com/mailnotificationservice/api/ens/registerdevicev2. Data: {"responseCode":14,"errorMessage":"The Autodiscover service couldn't be located.","version":"1.5.7235.6268","environmentType":"OnPrem"}. Response Code: 409
2019-11-11T09:13:43Z  E [9326] [ENS] registerAccountOnENS: Error updating settings or credentials
Error:Error Domain=com.alamofire.error.serialization.response Code=-1011 "Request failed: conflict (409)" UserInfo={NSLocalizedDescription=Request failed: conflict (409), NSErrorFailingURLKey=https://a3.ssdevrd.com/mailnotificationservice/api/ens/registerdevicev2, com.alamofire.serialization.response.error.data={length = 135, bytes = 0x7b227265 73706f6e 7365436f 6465223a ... 4f6e5072 656d227d }, com.alamofire.serialization.response.error.response=<NSHTTPURLResponse: 0x282db1fa0> { URL: https://a3.ssdevrd.com/mailnotificationservice/api/ens/registerdevicev2 } { Status Code: 409, Headers {
    "Content-Length" =     (
        135
    );
    "Content-Type" =     (
        "application/json; charset=utf-8"
    );
    Date =     (
        "Mon, 11 Nov 2019 09:13:40 GMT"
    );
    Server =     (
        "Microsoft-IIS/8.5"
    );
    "X-Powered-By" =     (
        "ASP.NET"
    );
} }}
2019-11-11T09:13:43Z  E [9365] - Unexpected error: {
    BXLocalizedContextMessageErrorKey = "Could not update settings for the push notification service";
    BXLocalizedTitleErrorKey = "Could not update settings for the push notification service";
    NSLocalizedDescription = "Could not update settings for the push notification service. ";
    NSLocalizedFailureReason = "Failed to update push notification settings. Please contact your administrator.";
}
In the sample error logs of Boxer, you can see the following message:
{"responseCode":14,"errorMessage":"The Autodiscover service couldn't be
        located...

In this case, ensure that the autodiscovery URL is reachable from the ENS and the autodiscovery URL is configured correctly using the EWSEditor tool or MS connectivity analyzer tool.

If you are using the EWSUrl, ensure that the EWSUrl key is configured in the console with a correct value for the EWSUrl of their respective Exchange environments. To verify the EWSUrl is correct, open a browser, enter the EWSUrl, and ensure that you are prompted to enter the credentials.

You can find the error message and response code for different reasons. Based on the error message, you can start troubleshooting the issue.

Error: 403 or 401 error message

EWS must be accessible to the ENS application to subscribe the user for notification. If the EWS is not configured correctly, then you might receive 403 or 401 error. In such cases, refer the following documents:

Check the type of authentication you have enabled in the EWS. Ensure that the authentication is in parity with what the customer is using for ActiveSync (Basic, OAuth, and CBA). The Boxer application sends the user credentials to the ENS and the ENS uses the same credentials and the same type of authentication to communicate with the EWS.

Note: If the ENS can access the Office 365 and the Active Directory Federation Services (ADFS), then ensure that either the ENS IPs are whitelisted on the ADFS or the affected user has no block claim on the ADFS.

If you are using Office 365 and you receive a 401 error from the EWS URL, the reason for the error might be because the client access rules or ADFS claims are configured. In such scenarios, refer the following documents.

In a scenario where the ENS on-premises Exchange with CBA is enabled, you might need to confirm that the client certificate is arriving at the Exchange endpoint. To troubleshoot any errors, see the #GUID-508681D8-67F6-4AF5-A3D8-3B8D742743B5 topic.

Force Register or Re-register on the Boxer application:

On iOS devices only, you can manually perform a force subscription, in the following cases:

  • If there are any changes to the keys in console, then you must approximately wait for 1 hour and check if the users are still receiving the notification. If the users are not receiving notifications, you can proceed to re-register the Boxer application with the ENS2 service.
  • If you do not see any register request in the ENS logs from the Boxer application, then assume that the Boxer application has failed to send the register request automatically. Therefore, the ENS tries to re-register the Boxer application with the ENS2 service forcefully.

To force register or re-register on the Boxer application, perform the following steps:

  1. Open the Boxer application and click Settings.
  2. Under the Accounts tab, select your ENS-specific account.
  3. Turn off the Use Push Service option.
  4. Navigate to the Boxer application Settings screen.
  5. Repeat Step 2 through Step 4 to turn on the Use Push Service option.

When you perform either of the steps mentioned, then you can see the force register request in the ENS logs.

To confirm the force subscription in the ENS logs, search for the ForceSubscription and you must be able to see the following value: ForceSubscription : [True].

Registration Status Events

If the registration is successful, then the Exchange sends a status event to the ENS periodically against each subscription ID, to confirm the subscription. The ENS then sends an acknowledgment for each of the subscription IDs back to the Exchange.

2019/11/05 08:57:31.413 A3  1eb9186b-9370-45de-a172-0e452586f398    [0000000-0000000]   (58)    Debug   MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Received [StatusEvent] for subscription: [JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAl4H5dKboFUm1kJ8ZNBKkJILRTBjMYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4=]
2019/11/05 08:57:31.413 A3  1eb9186b-9370-45de-a172-0e452586f398    [0000000-0000000]   (58)    Debug   MailNotificationService.BusinessImpl.PushNotificationBusiness.HandleExchangeEvents  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Status event received for user: [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]
If the ENS receives the status event for the old subscription ID, then the ENS responds to the Exchange with an unsubscribe response as shown in the following logs.
2019/11/05 08:49:20.123 A3  d2adec8a-73d7-48f2-ba14-abbd917844cd    [0000000-0000000]   (54)    Info    MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  This JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAJ6RYazaIoUCfX7KheUsQYUQnw9rIYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4= is old subscription for user 1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d, sending unsubscribe response  
2019/11/05 08:49:20.123 A3  d2adec8a-73d7-48f2-ba14-abbd917844cd    [0000000-0000000]   (54)    Debug   MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Sent Unsubscribe response to EWS successfully for subscriptionId: [JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAJ6RYazaIoUCfX7KheUsQYUQnw9rIYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4=]   
2019/11/05 08:49:20.123 A3  d2adec8a-73d7-48f2-ba14-abbd917844cd    [0000000-0000000]   (54)    Debug   MailNotificationService.BusinessImpl.PushNotificationBusiness.ProcessPushNotificationV2Async    User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  ProcessNotificationResponse.IsUnSubscribeResponse is true

For more information on the status frequency, see the StatusFrequency topic.

ENS must receive the status events from the Exchange immediately after a subscription is successful. If the ENS is not receiving the status events, then check the following troubleshooting methods to verify the communication between the Exchange server and the ENS.

Error: Status event not received

If you do not see any status events in the ENS logs after a successful subscription, then check the communication between the Exchange server and the ENS. Access the following URLs in the browser on the CAS or the mailbox servers to check the communication between the Exchange and the ENS.

  • For on-premises ENS deployments, use the https://{ENS URL}/MailNotificationService/api/ens/alive.
  • For cloud ENS deployments, use the https://{ENS URL}/api/ens/alive. For example, https://ens.getboxer.com/api/ens/alive. Select the ENS cloud URL based on your region.

You must be able to see the following result when you browse the specified URLs from the browser.

  • For on-premises ENS deployments, use the https://{ENS URL}/MailNotificationService/api/ens/pushnotificationlistener.
  • For cloud ENS deployments, use the https://{ENS URL}/api/ens/pushnotificationlistener. Select the ENS cloud URL based on your region.
Note: When browsing the URLs, if you see any SSL error, then proceed to import the ENS certificate in the MMC of the server.

New Mail Event and Fetch Mail

When a device is successfully registered and the communication between the ENS and the Exchange is working correctly, the Exchange starts sending new mail events to the ENS whenever a new mail is received on the subscribed user mailbox. If the payloads of the created events contain an unread count, then the ENS uses the unread count, else the ENS gets the unread count from the EWS.

2019/11/05 09:39:56.608 A3  9f08ed6d-0726-430c-8440-9c396443c7ca    [0000000-0000000]   (74)    Debug   MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Received [CreatedEvent] for subscription: [JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAl4H5dKboFUm1kJ8ZNBKkJILRTBjMYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4=]   
2019/11/05 09:39:56.639 A3  9f08ed6d-0726-430c-8440-9c396443c7ca    [0000000-0000000]   (74)    Debug   MailNotificationService.BusinessImpl.UnreadCountExchangeBusiness.GetUnReadCountV2   User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  EWSUrl used to get unread count: [https://outlook.office365.com/EWS/Exchange.asmx] 
2019/11/05 09:39:56.889 A3  9f08ed6d-0726-430c-8440-9c396443c7ca    [0000000-0000000]   (74)    Info    MailNotificationService.BusinessImpl.PushNotificationBusiness.HandleNewMailEvent    User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Received new mail event for user [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] with BADGE count [893]

Whenever the ENS receives a new mail event, the ENS fetches the mail information from the Exchange. The possible errors and solutions that you might see during a fetch mail request is listed as follows:

Error: Stuck in EWSUrl used to sync email: [https://outlook.office365.com/EWS/Exchange.asmx] steps

When a mail event is received from the Exchange, the ENS tries to fetch all the information from the mail. If you are unable to see any ENS logs such as the Fetched email, then check the respective EWS logs in the Exchange. You can obtain the corresponding EWS logs using the client request ID or the activity ID.

Fetch New Mail Request

Sample client request ID or the activity ID: 03ea7f36-f72f-4322-8413-0dcd81c4ac78

Note: You can get the client request ID or the activity ID in the third column of the ENS logs. Copy that ID and search for the client request ID or the activity ID in the EWS logs.

ENS sends a push notification request to the CNS or the SNS

When the new mail information is fetched from the Exchange, the ENS composes and sends a notification payload to the CNS (for on-premises) or the SNS (for cloud).

Sample of sending a notification payload to the CNS (for on-premises)

2019/11/05 09:48:42.675 A3  fedf9a1d-6cc8-4607-acad-ae006766292a    [0000000-0000000]   (82)    Info    MailNotificationService.BusinessImpl.NotificationsProcessor.AddNotificationToBatch  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  About to Post Notification for user : [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] and Device Id : [1]   
2019/11/05 09:48:42.690 A3  fedf9a1d-6cc8-4607-acad-ae006766292a    [0000000-0000000]   (82)    Info    MailNotificationService.BusinessImpl.NotificationsProcessor.AddNotificationToBatch  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  About to Post Notification for user : [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] and Device Id : [5]   
2019/11/05 09:48:47.699 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Info    MailNotificationService.BusinessImpl.CNSHelper.ComposeAPNSPushNotification  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Total unread count retrieved [894] for user [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] 
2019/11/05 09:48:47.699 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Debug   MailNotificationService.BusinessImpl.CNSHelper.ComposeAPNSPushNotification  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Sending to :: User : [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d], DeviceId : [1], DeviceLogId : [], Message :  messageId: [AAMkAGMxYjUzZDA0LTI5NDItNDUyNi1hZDMzLWIxMmRiNDgyYzIzZQBGAAAAAAAOx2petA5rS4RDQM8RjW1TBwDnjcIsAp4/S4beDDAIaXMhAAAAAAEMAADnjcIsAp4/S4beDDAIaXMhAAGszQatAAA=]
2019/11/05 09:48:47.699 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Info    MailNotificationService.BusinessImpl.CNSHelper.ComposeAPNSPushNotification  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Total unread count retrieved [894] for user [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] 
2019/11/05 09:48:47.699 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Debug   MailNotificationService.BusinessImpl.CNSHelper.ComposeAPNSPushNotification  User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Sending to :: User : [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d], DeviceId : [5], DeviceLogId : [61F9BB13-863C-444C-A300-4F888383ACDD-534-0000000CE599EDE0], Message :  messageId: [AAMkAGMxYjUzZDA0LTI5NDItNDUyNi1hZDMzLWIxMmRiNDgyYzIzZQBGAAAAAAAOx2petA5rS4RDQM8RjW1TBwDnjcIsAp4/S4beDDAIaXMhAAAAAAEMAADnjcIsAp4/S4beDDAIaXMhAAGszQatAAA=]   
2019/11/05 09:48:47.699 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Debug   MailNotificationService.BusinessImpl.CNSHelper.CreateWebRequest User Id:[no-user-id]    CNS Url : [https://cns.awmdm.com/nws/notify/apns]  
2019/11/05 09:48:47.699 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Debug   MailNotificationService.BusinessImpl.CertificateHelper.ComputeCmsSignature  User Id:[no-user-id]    Signing URL [/nws/notify/apns] with Cert [CN=AW Cloud Notification - aTest]
2019/11/05 09:48:48.558 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Debug   MailNotificationService.BusinessImpl.CNSHelper.ReadResponse User Id:[no-user-id]    Response {"status":"success","errorReason":null}   
2019/11/05 09:48:48.558 A3  7e45c693-511b-4c19-ae7c-305e5f8f9f0e    [0000000-0000000]   (8)     Info    MailNotificationService.BusinessImpl.CNSHelper.ReadResponse User Id:[no-user-id]    ResponseCode OK

Sample of sending a notification payload to the SNS (for cloud)

2019-09-06 12:11:51.5380|INFO|MailNotificationService.BusinessImpl.NotificationsProcessor.AddNotificationToBatch|b1d8e164-c3fb-4f67-baa6-002dd3719c4e|User Id:[35045e4062200ca81c92d5b03928a7e86383ef8e9436d512187a711a4b18e94f] About to Post Notification for user [35045e4062200ca81c92d5b03928a7e86383ef8e9436d512187a711a4b18e94f]
 2019-09-06 12:11:52.5537|INFO|MailNotificationService.BusinessImpl.AmazonSNSHelper.PostNotifications|67d3c6f0-a197-4af4-958c-260eeedbf567|User Id:[35045e4062200ca81c92d5b03928a7e86383ef8e9436d512187a711a4b18e94f] Sending notification via SNS
 2019-09-06 12:11:52.5692|INFO|MailNotificationService.BusinessImpl.AmazonSNSHelper.PushNotificationViaSNS|67d3c6f0-a197-4af4-958c-260eeedbf567|User Id:[35045e4062200ca81c92d5b03928a7e86383ef8e9436d512187a711a4b18e94f] Notification successfully sent via SNS for [424716]

To confirm if your Android device is receiving notifications from the ENS, enable the Boxer application passcode and restart the device after a successful registration. You might see a notification, that is, a banner containing the email address configured. On the banner notification if you cannot perform actions such as, Delete, Reply, and Read option then, the notification is a push notification that is sent from the ENS and not locally from the Boxer application itself. If the notification banner contains notification actions such as Delete, Reply, Read, and so on, then the notification is a local notification from the Boxer application and not a push notification from the ENS.

The possible errors and solutions that you might see during a push notification request is listed as follows:

Error: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

2019/11/06 09:03:48.218 A3  aa57f568-6871-42cc-8b8d-39c77a15af41    [0000000-0000000]   (40)    Error   MailNotificationService.BusinessImpl.CNSHelper.ReadResponse User Id:[no-user-id]    Failed To Post to CNS [https://cns.awmdm.com/nws/notify/apns] Error: [The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.] Response: []
Note: Ensure you have followed the steps as mentioned in the Configure CNS and Download Email Notification Service Configuration Files topic.

If the issue still persists, download the latest public CNS certificate from http://resources.workspaceone.com/view/2hjxzvgkxyf8n738hy7x/en and perform the following steps:

  1. Click the SSLPinningCertTool shortcut present in the ENS server or click <ENS_INSTALL_DIR>\Email Notification Service\Tools\SSLPinningCertTool\SSLPinningCertTool.exe.
  2. Click the Upload CNS Certificate button.
  3. Select the certificate to be uploaded and click Submit. If the following screen appears, then the certificate is successfully added.
    Note: After uploading the SSL pinning certificate on the ENS, the tool adds the public key of the certificate to the ENS configuration. When the ENS posts payload to the CNS, the certificate validation is done against the newly added certificate public key.
  4. If the following screen appears, the certificate is successfully added to the resubscription configuration file.
    Note: After uploading the SSL pinning certificate, the tool adds the public key of the certificate to the resubscription configuration file. For the resubscription mechanism, after payload (silent notification) to the CNS, the certificate validation is done against the newly added certificate public key.
  5. If the certificate is already present in both the configuration files, then you are prompted with the following message.
    Note: The upload pinning certificate occurs as follows:
    • The tool tries to upload the certificate to the ENS configuration file only if the provided certificate is not present in the ENS configuration file. If the given certificate is already present, then the tool does not prompt any message and continues to upload the same certificate to the resubscription configuration file.
    • The tool tries to add a certificate to the resubscription configuration file only if the provided certificate is not present in the resubscription configuration file. If the given certificate is present, then the tool does not prompt any message to the user.
  6. If the certificate is added to the resubscription configuration file, then navigate to Services and restart the AirWatch Resubscription Mechanism service.

Error: The remote server returned an error: (401) Unauthorized.

Sample error log:

2019/11/06 09:25:13.688 A3  6c041e00-c909-45ff-b340-283844376c06    [0000000-0000000]   (6)     Error   MailNotificationService.BusinessImpl.CNSHelper.ReadResponse User Id:[no-user-id]    Failed To Post to CNS [https://cns.awmdm.com/nws/notify/apns] Error: [The remote server returned an error: (401) Unauthorized.] Response: [{"code":2007,"message":"Unable to verify if the signer cert as trusted. The associated request id is 154e9542-b695-497b-9896-a8fd9cb13e84."}]

If you see a 401 error while posting a notification and the UEM console is on-premises, then navigate to System > Advanced > Secure Channel Certificate and select the Download CNS Secure Channel Certificate Installer. You can also open a Zendesk ticket with the SaasOps > CNS Upload Request category. To install the certificate on the CNS server, send a request to the VMware Support team.

Error: ENS has posted notification to CNS/SNS successfully, but we don't see any notification on the device.

This error occurs due to the APNS or the GCM token issue. To verify the APNS or the GCM tokens, perform the following steps:

  1. Log in to the Workspace ONE UEM console and navigate to the organization group where the device is enrolled.
  2. Navigate to the Devices > List View and select the device.
  3. Click the SEND > PUSH NOTIFICATION and select the application as Boxer from the drop-down.
  4. Enter the Message Body and click SEND. After you click SEND, you must be able to see the notification on the device if the APNS token is correct.

Unregistered ENS Logs

The Boxer application sends an unregister request to the ENS in the following scenarios:

  • When a device account is removed from the Boxer application
  • When a device is deleted from the Workspace ONE UEM console.
  • During an enterprise wipe from the Workspace ONE UEM console.
  • Toggle off the push notification button in the Boxer application settings.

Sample of unregistered ENS logs:

2019/11/06 10:33:23.976 A3  2bd0af6a-ba08-479e-a606-b1326281902c    [0000000-0000000]   (53)    Debug   MailNotificationService.Controllers.EnsController.Unregister    User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Processing Unregister request. UserId:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]   
2019/11/06 10:33:24.054 A3  2bd0af6a-ba08-479e-a606-b1326281902c    [0000000-0000000]   (55)    Debug   MailNotificationService.BusinessImpl.UnregisterBusiness.ProcessUnregisterRequestAsync   User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Device Unregistered for user:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]
2019/11/06 10:33:24.054 A3  2bd0af6a-ba08-479e-a606-b1326281902c    [0000000-0000000]   (55)    Debug   MailNotificationService.Controllers.EnsController.Unregister    User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Unregister request processed. HttpStatusCode:[OK] ResponseCode:[DeviceUnregistered]

When the ENS receives an unregister request, the ENS processes the request and sends an unsubscribe request to the Exchange and deletes the records from the database. The possible errors and solutions that you might see when you unregister is listed as follows:

Error: 401 error Unauthorized

The following logs are seen when the Boxer application sends an unregister request with a wrong API token. You can confirm the API token comparing the API token logged in the ENS logs and present in the Boxer application logs.

ENS logs: API token : [12341*********fasdf]

Boxer application logs: ensapitoken: 17413********************88c08

Sample of UnAuthorizedRequest log:

2019/11/06 10:38:20.413 KAVINASH-W03    cd790dc0-ca7e-4f3d-b468-3c5181c34063    [0000000-0000000]   (31)    Warn    MailNotificationService.BusinessImpl.ApiKeyRepository.ValidateAsync User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  ApiKey header present [True], Value Empty/Null: [False] API key dictionary has keys:[True] Key: [12341:fasdf]  
2019/11/06 10:38:20.424   cd790dc0-ca7e-4f3d-b468-3c5181c34063    [0000000-0000000]   (31)    Debug   MailNotificationService.Controllers.EnsController.Unregister    user Id [1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] API token : [12341*********fasdf]   
2019/11/06 10:38:20.444   cd790dc0-ca7e-4f3d-b468-3c5181c34063    [0000000-0000000]   (31)    Warn    MailNotificationService.Controllers.EnsController.Unregister    User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Error Code:'23' Error message: 'UnAuthorizedRequest'
Stack Trace:   at MailNotificationService.Controllers.EnsController.<Unregister>d__21.MoveNext() in C:\Stash\MailNotificationService\Controllers\EnsController.cs:line 926  

Badge Update for ENS Logs

Badge update is only supported for iOS devices. The badge notification starts displaying only after the badge receives the first notification from ENS. The badge count is not seen in Boxer immediately after the badge counter is configured and subscribed.

Sample of badge update ENS logs:

2019/11/11 12:27:55.416 A3  04f06dcb-a721-4a90-a2ff-2be8e007f533    [0000000-0000000]   (52)    Debug   MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Received [ModifiedEvent] for subscription: [EgBleGNoMjAxMy5tZW0xMy5vcmcQAAAAjoO0qTL7hk2FF7QvXOHC1BLv0sChZtcIEAAAACanmwmX5x5OpwfUW+dfdrQ=]  
2019/11/11 12:27:55.525 A3  04f06dcb-a721-4a90-a2ff-2be8e007f533    [0000000-0000000]   (52)    Info    MailNotificationService.BusinessImpl.PushNotificationBusiness.HandleMoveModifiedEventAsync  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  -BADGE UPDATE- [5422] previous BADGE count is [5422] Received modified event for user [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]   
2019/11/11 12:27:55.525 A3  04f06dcb-a721-4a90-a2ff-2be8e007f533    [0000000-0000000]   (52)    Info    MailNotificationService.BusinessImpl.NotificationsProcessor.AddNotificationToBatch  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  About to Post Notification for user : [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586] and Device Id : [9]   
2019/11/11 12:28:00.531 A3  d7893c08-3a06-46a1-a8a7-45361572b573    [0000000-0000000]   (16)    Info    MailNotificationService.BusinessImpl.CNSHelper.ComposeAPNSPushNotification  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Total unread count retrieved [5422] for user [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]
2019/11/11 12:28:00.531 A3  d7893c08-3a06-46a1-a8a7-45361572b573    [0000000-0000000]   (16)    Debug   MailNotificationService.BusinessImpl.CNSHelper.ComposeAPNSPushNotification  User Id:[20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586]  Sending to :: User : [20943ad3f74ef04b3a2394b968cb46cc498f54994bdec0b3520d965e35356586], DeviceId : [9], DeviceLogId : [], Message :  messageId: []
2019/11/11 12:28:00.531 A3  d7893c08-3a06-46a1-a8a7-45361572b573    [0000000-0000000]   (16)    Debug   MailNotificationService.BusinessImpl.CNSHelper.CreateWebRequest User Id:[no-user-id]    CNS Url : [https://cns.awmdm.com/nws/notify/apns]  
2019/11/11 12:28:00.531 A3  d7893c08-3a06-46a1-a8a7-45361572b573    [0000000-0000000]   (16)    Debug   MailNotificationService.BusinessImpl.CertificateHelper.ComputeCmsSignature  User Id:[no-user-id]    Signing URL [/nws/notify/apns] with Cert [CN=AW Cloud Notification - aTest]
2019/11/11 12:28:00.748 A3  d7893c08-3a06-46a1-a8a7-45361572b573    [0000000-0000000]   (16)    Debug   MailNotificationService.BusinessImpl.CNSHelper.ReadResponse User Id:[no-user-id]    Response {"status":"success","errorReason":null}   
2019/11/11 12:28:00.748 A3  d7893c08-3a06-46a1-a8a7-45361572b573    [0000000-0000000]   (16)    Info    MailNotificationService.BusinessImpl.CNSHelper.ReadResponse User Id:[no-user-id]    ResponseCode OK

Understanding ENS Logs

The ENS logs contain information about registration, subscriptions, notifications, and the CNS or the APNS delivery status. For the on-premises ENS, you can find the ENS2 logs files at: %ENS Installed Directory%\Logs\Email Notification Service. For example, the ENS2 log file can be at: C:\AirWatch\Logs\Email Notification Service. The name of the log file is ENS.log.

Sample ENS2 log file:

2019/11/05 09:39:56.608 A3  9f08ed6d-0726-430c-8440-9c396443c7ca    [0000000-0000000]   (74)    Debug   
MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync  
User Id:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d]  Received [CreatedEvent] for subscription: [JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAl4H5dKboFUm1kJ8ZNBKkJILRTBjMYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4=]   

The following table provides a description of a sample ENS2 log file.

Log Format Value
Date 2019/11/05 09:39:56.608. The date is mentioned in the UTC format.
machinename A3
ActivityId 9f08ed6d-0726-430c-8440-9c396443c7ca
threadid (74)
logLevel Debug
Logger MailNotificationService.BusinessImpl.ExchangeNotificationParser.ScanEventNotificationAsync
Message UserId:[1743604ea20cda831dc7aea285e7fdc011ca233caf0fa7d5d926916622dd182d] Received [CreatedEvent] for subscription: [JwBtbjJwcjE5bWIzMDA1Lm5hbXByZDE5LnByb2Qub3V0bG9vay5jb20QAAAAl4H5dKboFUm1kJ8ZNBKkJILRTBjMYdcIEAAAAAQ9tcFCKSZFrTOxLbSCwj4=]
Note:

In the logs, you can find the user name or email address in the alphanumeric format and not in the plain text format. For example, the user ID is mentioned as an alphanumeric string such as, 4e9dc715faba719b266fe90f866caf8e377c08984cd1fd005bac72c7eba4db02. This string is a hash value that is calculated from the email address.

You can use the SHA-256 hash calculator to translate any email address to a hash value. You can then use the hash value to search logs for any user.

To obtain the logs for the cloud ENS, use the ENS2 Self-Help website based on your region.

Troubleshooting the ENS2 SEG Errors

This section describes the troubleshooting steps you might have to perform due to communication errors between the ENS2 and Exchange with SEGv2 as the proxy.

The following steps describe the interaction between the ENS2 and Exchange with SEGv2 as the proxy.

  1. Boxer application requests a public key from the ENS.
  2. Boxer application encrypts the user credentials using the public key and sends a subscription request to the ENS.
  3. ENS requests a subscription to the Exchange server using the SEG URL which also contains the encrypted credentials. The ENS also sends a client certificate. If the client certificate is configured on the Boxer application profile, then the authentication received from the Boxer profile is sent. For certificate-based authentication (CBA), when a register device request is sent to the cloud ENS server, the ENS routes the request to the SEG with the certificate information. The SEG follows the same token retrieval process similar to the ActiveSync request.
  4. SEG forwards the subscription request to the Exchange to complete the subscription. The same authentication method configured in the Boxer application profile is used for subscription. The ENS server callback URL is used to subscribe.
  5. The Exchange server receives an email.
  6. The Exchange server notifies the ENS callback URL of the subscriber to inform that a new email has arrived, hence update the email client with the notification. The ENS fetches the details of the email from the SEG.
  7. The ENS server requests the CNS or SNS to send notification to the Boxer application or the device of the subscriber.
  8. The CNS or the SNS server contacts the Apple Push Notifications (APNs for iOS devices) or GCM or FCM (for Android devices).
  9. The APNS or GCM server pushes the email notification to the device.

Using the transaction ID received in the ews-transaction log, you can search the ews-proxy.log. For example, if the transaction ID is 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 then search for 544ef2b7.

When you see 200 in the ENS transaction log, you can confirm if the notifications are going through the CNS communication.

The Ews-transaction log sample.

Time, LogLevel, Thread Id, Message, HTTP-Method, Remote-Host, X-Forwarded-For, SEG TransactionId, Request-DeviceId, EnsDevices, EmailServerResponseStatus, SegResponseStatus, EmailRequestBodySize, EmailResponseBodySize, TimeTakenByKerberosService(ms), TimeTakenBySeg(ms), TimeTakenByEmailServer(ms), BeginningOfRequest
2018-12-03 17:20:15.696, DEBUG, (vert.x-eventloop-thread-0), Responding back to ENS,POST,192.168.2.34,null,544ef2b7-9ca3-4009-b116-8a9f6513f2c7,6C30D0304E7A4EE795494DEB0F465B72,"6C30D0304E7A4EE795494DEB0F465B72:200",200,200,1243,1147,2547,0,16,1543875613133
2018-12-03 17:20:18.274, DEBUG, (vert.x-eventloop-thread-0), Responding back to ENS,POST,192.168.2.34,null,d77ae46b-2d38-46b5-9548-3bcc25a1bf03,6C30D0304E7A4EE795494DEB0F465B72,"6C30D0304E7A4EE795494DEB0F465B72:200",200,200,673,1806,2547,0,16,1543875615711
2018-12-03 17:20:41.430, DEBUG, (vert.x-eventloop-thread-0), Responding back to ENS,POST,192.168.2.34,null,b639bdee-0cfa-42b5-82ea-0629ab1d586a,6C30D0304E7A4EE795494DEB0F465B72,"6C30D0304E7A4EE795494DEB0F465B72:200",200,200,1632,2464,2562,0,47,1543875638821
2018-12-03 17:21:16.462, DEBUG, (vert.x-eventloop-thread-1), Responding back to ENS,POST,192.168.2.34,null,ed0db6c1-9dd4-420a-83d3-e746cb17445c,82B15D853CC14CA3989020257158BFC1,"82B15D853CC14CA3989020257158BFC1:200",200,200,1632,3028,2563,0,47,1543875673852
2018-12-03 17:21:26.493, DEBUG, (vert.x-eventloop-thread-1), Responding back to ENS,POST,192.168.2.34,null,425fc495-4ae1-4c26-abc5-c30f34a376cf,82B15D853CC14CA3989020257158BFC1,"82B15D853CC14CA3989020257158BFC1:200",200,200,673,1815,2547,16,15,1543875683915
2018-12-03 17:22:46.649, DEBUG, (vert.x-eventloop-thread-1), Responding back to ENS,POST,192.168.2.34,null,ba0b13ad-b341-43e3-a4a9-d1a79c5330e0,82B15D853CC14CA3989020257158BFC1,"82B15D853CC14CA3989020257158BFC1:200",200,200,1632,3028,2547,15,32,1543875764055
2018-12-03 17:23:01.649, DEBUG, (vert.x-eventloop-thread-1), Responding back to ENS,POST,192.168.2.34,null,262cc2b2-8ae4-4ea7-b062-da2b2eb42a68,82B15D853CC14CA3989020257158BFC1,"82B15D853CC14CA3989020257158BFC1:200",200,200,673,1815,2547,0,15,1543875779087
2018-12-03 17:26:47.353, DEBUG, (vert.x-eventloop-thread-3), Responding back to ENS,POST,192.168.2.34,null,c7e5a6c9-b1b0-4739-9132-49470306882c,6C30D0304E7A4EE795494DEB0F465B72,"6C30D0304E7A4EE795494DEB0F465B72:200",200,200,673,1806,2547,0,94,1543876004712
2018-12-03 17:26:51.884, DEBUG, (vert.x-eventloop-thread-3), Responding back to ENS,POST,192.168.2.34,null,d5cf2470-d818-45f6-ab0e-dd68599d4aa8,6C30D0304E7A4EE795494DEB0F465B72,"6C30D0304E7A4EE795494DEB0F465B72:200",200,200,673,1806,2547,0,15,1543876009322
2018-12-03 22:06:55.421, DEBUG, (vert.x-eventloop-thread-2), Responding back to ENS,POST,192.168.2.34,null,93f7f097-bda5-417a-ac67-5667b4088c84,6C30D0304E7A4EE795494DEB0F465B72,"6C30D0304E7A4EE795494DEB0F465B72:200",200,200,673,1806,12000,16,234,1543892803171
2018-12-03 22:07:00.031, DEBUG, (vert.x-eventloop-thread-0), Responding back to ENS,POST,192.168.2.34,null,d10c08a4-49cd-4240-bcc0-ba9bb81f74f0,82B15D853CC14CA3989020257158BFC1,"82B15D853CC14CA3989020257158BFC1:200",200,200,673,1815,11969,0,188,1543892807874
2018-12-04 10:31:33.786, DEBUG, (vert.x-eventloop-thread-2), Responding back to ENS,POST,192.168.2.34,null,3844719b-73c6-4b77-91d8-8a7d8b9a97c0,82B15D853CC14CA3989020257158BFC1,"82B15D853CC14CA3989020257158BFC1:200",200,200,1632,3028,2563,15,516,1543937490692

The Ews-transaction log sample filtered using the 544ef2b7.

2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestReadHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Incoming EWS request, Path: /EWS/Exchange.asmx. Headers are
2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsHelper] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Collected ENS devices: [6C30D0304E7A4EE795494DEB0F465B72]
2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestReadHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Getting device policy for request device 6C30D0304E7A4EE795494DEB0F465B72
2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsComplianceCheckHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Device list: [6C30D0304E7A4EE795494DEB0F465B72]
2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsComplianceCheckHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Checking compliance for device 6C30D0304E7A4EE795494DEB0F465B72
2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsComplianceCheckHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Device 6C30D0304E7A4EE795494DEB0F465B72 is compliant
2018-12-03 17:20:13.133 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestProxyHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 KCD authentication is (true), upn is TUSER1@MILKYWAY.LOCAL.
2018-12-03 17:20:15.680 DEBUG (pool-7-thread-5) [c.a.s.e.h.EwsRequestProxyHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Successfully got kerberos token for UPN TUSER1@MILKYWAY.LOCAL - token length 2024
2018-12-03 17:20:15.680 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestProxyHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Proxying request to EWS
2018-12-03 17:20:15.680 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestProxyHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - EWS client request headers:
2018-12-03 17:20:15.696 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestProxyHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - EWS client response headers:
2018-12-03 17:20:15.696 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsHelper] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - Response headers from SEG to ENS:
X-AW-SEG-TRANSACTION-ID : 544ef2b7-9ca3-4009-b116-8a9f6513f2c7
2018-12-03 17:20:15.696 DEBUG (vert.x-eventloop-thread-0) [c.a.s.e.h.EwsRequestProxyHandler] - 544ef2b7-9ca3-4009-b116-8a9f6513f2c7 - EWS response status 200, length 1147

The possible errors and solutions you might see during an interaction between the ENS2 and Exchange with SEGv2 as the proxy is listed as follows:

Error: 404 / https://[segURL]/EWS/Exchange.asmx is not found

If you see this error in the ENS logs, then ensure you have enabled the EWS proxy in the SEG server. If you have not enabled the EWS proxy in the SEG server then perform the following steps.

  1. Navigate to the SEG > Config folder using the File explorer.
  2. Select the application.properties file and edit the file.
  3. Select the enable.boxer.ens.ews.proxy value and update the value to enable.boxer.ens.ews.proxy=true.
  4. Save the file.
  5. Restart the VMware AirWatch Secure Email Gateway service.

Sample of the application.properties file.

#######################################################################################################################################
############################# Start - HTTP endpoint path for SEG active-sync, syncML and REST API.   ##################################
#######################################################################################################################################
# SEG HTTP server context path. This should be same as the context path of Email/Exchange server as Device won't know
# if it's sending request to email server or SEG Proxy. This value generally don't change but we want to give
# the ability to the Admin to change it, if needed in some exceptional cases.
# Right now Vertx doesn't support "ignore-case" on path, and also doesn't allow mounting sub-routers on RegEx.
# For now we're trying to avoid using RegEx anyway - https://groups.google.com/forum/#!topic/vertx/ck95b4juj4A
activesync.context.paths=/Microsoft-Server-ActiveSync,/microsoft-server-activesync# Context path when SEG works as EWS proxy for ENS. EWS endpoint will be disabled by default.
enable.boxer.ens.ews.proxy=true
ews.proxy.context.paths=/EWS,/ews
 
# Flag used to remove unsupported www-authenticate header such as NTLM and Negotiate (in absense of certificate) from EWS response to ENS.
 
remove.unsupported.auth.for.ews=true

Error: 401 - Please check the authentication type enabled in exchange (EWS endpoint)

If you see this error in the ENS logs, then the SEGv2 does not support the NTLM authentication. If both the Basic and NTLM authentication mechanisms are enabled for the EWS endpoint, then the SEGv2 version prior to version 2.9.0.1 cannot prefer Basic authentication over the unsupported NTLM authentication.

This results in the ENS attempting the NTLM-based authentication for requests through the SEG, that eventually causes 401 error responses as observed in the ews-transaction.log. If the user is unable to disable the NTLM authentication mechanism for the EWS endpoint, and is using any lower version of the SEG, then setup the KCD authentication for the ENS-SEG integration to work correctly.

If you connect directly to the EWS endpoint on the SEGv2 proxy through the https://[segURL]/EWS/Exchange.asmx URL, you might receive a 400 error message unless you connect using a permitted device.

Error: The request was aborted: Could not create SSL/TLS secure channel

In the ENS logs, if you see the following error during the registration process, then the error might be due to a cipher mismatch.

2019-12-05 15:33:40.5081|DEBUG|MailNotificationService.BusinessImpl.ExchangeRetriesHandler.SubscribeForNotificationsAsync|3ed2219d-42f2-4a2a-b857-ab7639ad1858|User Id:[af03aa8bb3cae692442ec673b207fbe5666e0762bf3ca62cbaaa61c4208cd7bd] EWSUrl used to subscribe: [https://uag.testdomain.com/ews/exchange.asmx]
 2019-12-05 15:33:40.5550|WARN|MailNotificationService.BusinessImpl.SubscriptionBusiness.SubscribeV2Async|3ed2219d-42f2-4a2a-b857-ab7639ad1858|User Id:[af03aa8bb3cae692442ec673b207fbe5666e0762bf3ca62cbaaa61c4208cd7bd] Service request exception occured for userId [af03aa8bb3cae692442ec673b207fbe5666e0762bf3ca62cbaaa61c4208cd7bd], Inner exception message [The request was aborted: Could not create SSL/TLS secure channel.].

To fix the cipher mismatch error, perform the following steps:

  1. Run a TCP dump on the UAG or SEG. Check the reason for the handshake failure, using the following commands. See the Troubleshooting Firewall and Connection Issues section in the Deploying and Configuring VMware Unified Access Gateway guide.
    /etc/vmware/gss-support/install.sh 
    tcpdump -i any -n -v tcp port any -w /tmp/vmware/capture.pcap
  2. Open the TCP dump logs using the Wireshark or any supported application. Filter the logs based on the IP source and IP destination and check for the client hello request as shown in the following log.

    Use the tls.alert_message.level filter to search for the SSL error or alert in the Wireshark. Identify the source and destination IP, right click, and select Follow > Follow → TLS stream.

  3. Right click and open the Client Hello information.
  4. Click the Show packet > TLS 1.2 Record Layer > Handshake Protocol : Client Hello > Transport Layer Security > Cipher Suits. You can see a list of cipher suites that the client ENS is sending to initiate a secure communication as shown in the following image.
  5. Ensure that the UAG or the SEG server has enabled the ciphers listed in the Client hello Request.
    Note: To check for the enabled cipher suites in the UAG or the SEG server, you can use the SSL report. Enter your SEG or UAG URL and wait for the test to complete. When the test is complete, you might see the following result.

The following table lists all the response codes and messages in the SEG logs.

Response Code Message Description
204 No Content Indicates that the policy data is not loaded in the SEG to run the compliance check on the requesting devices.
403 Forbidden Indicates that none of the devices listed in the ENS request headers are compliant.
400 Bad Request Indicates that none of the devices listed in the ENS request header are found in the SEG device policy cache.
5xx Indicates the server errors.