The release notes cover the following topics:
What's New in this Release
With Smart Assurance 10.1.5 release, the following features, enhancements, and changes are introduced:
- M&R version is upgraded to 18.104.22.168 [build 34]. Following SolutionPacks are qualified with M&R 22.214.171.124 release:
- Cisco IPSLA
- Cisco Unified Communication Manager
- VMware vCenter
- Traffic Flows
- VMware Smarts
Note: M&R MIB-Browser module has been deprecated in 7.x and removed in 126.96.36.199.
Support for vSphere 7.0.1 and 7.0.2 in ESM and M&R [VMware vCenter SP]:
Discovery and Monitoring of vSphere 7.0.1 and 7.0.2 are qualified on ESM 10.1.5. Also vSphere 7.0.1 and 7.0.2 reporting through VMware vCenter SP has been qualified on M&R 188.8.131.52.
- vRealize Operations 8.4 Support:
- Certification of VSA Management Pack with vROps v8.4 (On-Prem and Cloud) to show physical infrastructure topology, metric, and event data in vROps
- Support for token based vROps 8.4 APIs.
- Leverage vROPs 8.4 Truststore manager in place of Java supplied Truststore manager.
- Kafka Adapter credential management.
- Updated dashboards using vROps 8.4 widgets.
Multiple vROps support.
Smarts is enhanced to support multiple vROps.
vRealize Operations Collector Enhancements:
- Ability to contextually visualize vROps alerts as VMware Telco Cloud Operations notification occurring on respective VMs and Hosts. In previous release, the host mapping was not available.
- VMware Telco Cloud Operations notifications now include a normalized VM name across vROps and VMware Telco Cloud Operations.
- VMware Telco Cloud Operations notification data now includes vROps information for easier troubleshooting
- [VM-ER-205]: Certification of Nokia SR-1 device variant in ASAM domain manager.
- [VM-ER-185]: Notifications coming from vROps (VMware vRealize Operations manager), now gets associated to Element Name and Element Class. Due to this mapping, following two uses cases are getting addressed as part of this ER:
- A Notification for Service (for example: MS SQL) Down, Smarts will now inform on which Server(Element Name) its Down.
- Creating unique events in Smarts, if MS SQL is down for two servers, in Smarts count of Notification will increase.
For security vulnerabilities addressed in Smarts, see Smart Security Update for Multiple Vulnerabilities.
- Perl upgraded to 5.32.0.
- Java is upgraded to 184.108.40.206.101, except for ASAM and SAM console.
- Java is upgraded to 220.127.116.112 32 bit SAM console (windows / linux).
- Zlib is upgraded to 1.2.11.
- Jackson-databind is upgraded to 2.12.3, for SAM domain manager.
- Jackson-databind is upgraded to 2.11.0 for OTM domain manager.
- RabbitMQ client (amqp-client) is upgraded to 5.12.0 for SAM and ESM.
- Tomcat is upgraded to 9.0.45 on SAM.
- Log4J is upgraded to 1.2.17
- SQlight3 is upgraded to 3.32.3, on SAM windows console.
For OSL (Open Source License) file (open_source_license_VMware_Smart_Assurance_GA.txt), navigate to <BASEDIR>/smarts/setup/osl.
In VSA 10.1.5 release, support for RHEL 8 operating system is added.
Note: SAM Console is not supported on Linux 8.x versions.
The VMware Smart Assurance SAM, IP, ESM, MPLS, and NPM Managers Support Matrix available from the VMware Support website provides the latest platform and interoperability information. For detailed information about platform support and interoperability, refer support matrix for your release.
- In VSA 10.1.5 release, some of the document(s) do not require modification. The older version document(s) are released as it is.
- The VeloCloud, vIMS, and, vEPC discovery and monitoring functionality is only available with VMware Telco Cloud Operations deployment. It is not available for standalone VSA/DCF.
- SMARTA-1196 / SR-21198225502
Missing typologies for Nexus 3k devices.
Resolution: A new flag
EnableLACPDataConnectionhas been added to
tpmgr-param.conffile and when enabled TRUE, the LACP based connections are avoided by SMARTS. This has been introduced to handle buggy devices that expose wrong neighbouring device details through LACP MIB.
- SMARTA-1203 / SR-21196119902
The status of all interfaces is coming as unknown on VirtualSwitchSystem.
Resolution: Reconfigure code logic has been modified to avoid deletion of
Interface-fault-MIB2instrumentation classes that belongs to
- SMARTA-876 / SR-19010078206
A possible code flaw with
topo-split-sync-AM.aslscript with respect to AddPending call.
topo-split-sync-AM.aslcode has been modified to add the device IP address to the ADDPending call.
- SMARTA-1063 / SR-20171774511
IsValid not initialised in
wireless-ext-sys-setting.aslleading to error in domain log.
Resolution: Modified the post processing script
ext-sys-setting.aslto address the following asl errors:
[November 12, 2020 3:15:24 PM GMT+00:00 +489ms] t@1875900160 ASL-W-ERROR_RULE_SOURCE-While executing rule set '/opt/InCharge/IP/smarts/rules/devstat/ext-sys-setting.asl' ASL-ERROR_ACTION-While executing action at: ASL-CALL_STACK_RULE- RuleName: INSTRUMENT_HSRPENDPOINT, Line: 439 ASL-CALL_STACK_RULE- RuleName: START, Line: 117 ASL-ERROR_INVOKE-While attempting to invoke operation 'getVirtualIP' of object '::' MR2-MR_WHILE_OPERATION_INVOKE-While invoking operation getVirtualIP in object '::' MR-OBJECT_NOT_FOUND_EX-Cannot find object '::' MR-CALL_ON_NULL_OBJECT-Tried to make a call on a NULL repository object; in file "/work/redcurrent/DMT-10.1.0.0/99/smarts/repos/servif/SM_RepositoryServer.c" at line 1131 ----------------- [November 12, 2020 3:06:11 PM GMT+00:00 +607ms] t@3467638528 ASL-W-ERROR_RULE_SOURCE-While executing rule set '/opt/InCharge/IP/smarts/rules/devstat/wireless-ext-sys-setting.asl' ASL-ERROR_ACTION-While executing action at: ASL-CALL_STACK_RULE- RuleName: START, Line: 101 ASL-ERROR_UNDEFINED_VARIABLE_ACCESS-Variable 'IsValid' has not been assigned a value; in file "/work/redcurrent/DMT-10.1.0.0/99/smarts/ga/parser/ASL_Binding.c" at line 338
- SMARTA-980 / SR-20156100609
Smarts 10.1, Interfaces (with Vlan description) getting deleted and discovered during re-discovery.
Resolution: Modified the discovery code logic, not to delete the interface and create back between the asl files:
- SMARTA-1127 / SR-20180127712
High memory utilization observed on all the IP domains on Smarts 10.1.2 version.
Resolution: Refined the code logic in file
ic-post-topo-management.asl, that will call manage/unmanage based on network adapters managedState instead of IsManaged thus avoiding unnecessary calls to manage/unmanage over the network adapters.
Note: It is best recommended to EXPLICITLYL_MANAGED/ECPLICITLY_UNMANAGE the network adapters in the custom code so that they are remain in that exclusive state thus the GA code does not override the status in the subsequent manage/unmanage method.
To explicitly manage an interface use manage(TRUE); passing TRUE as an argument.
To explicitly unmanage an interface call unmanage(TRUE); passing TRUE as an argument.
- SMARTA-279 / SR-19000344106
In Smarts NPM, the EIGRP - CPU usage hits 100% on Instrumentation perl script.
Resolution: Perl module has been upgraded to 5.32.0, where this high CPU issue has been resolved.
- SMARTA-1271 / SR-21215181604
Unable to discover Wireless access points for SystemObjectID = .18.104.22.168.22.214.171.124.2170.
Resolution: Updated the certification for sysoid : .126.96.36.199.188.8.131.52.2170 and added the WIFI driver to discover the WAP [Wireless Access Points] associated to this device.
- SMARTA-1075 / SR-20176438112
Smarts 10.1.0.1 BGP does not discover
Resolution: Added the class definition
StackSwitchSystemin NPM domain manager to map the same class instance coming from the underlying IP domain manager.
- SMARTA-977 / SR-20157674509
StackSwitchSystemClass missing from SAM 10.1.2.
Resolution: Class model
StackSwitchSystemhas been added to SAM domain manager along with the DXA entry to map the same class imported via the underlying IP domain manager.
Issue in Cisco ACI discovery.
Resolution: Fixed the code flow in ACI discovery, where discovery of multiple cluster discovery without restart (clean-topology) is handled.
- SMARTA-999 / SR-20162531710
Smarts 10.1.x and 9.6 WebConsole does not work with latest Java 1.8. build 26.
Resolution: Code fixed has been done on the support the latest version of Java 1.8.X.
- SMARTA-1197 / SR-21199241602
Oracle Linux version Support in the Smarts.
Resolution: All the Oracle linux kernel version [minor] under 7.x support has been added.
The known issues are grouped as follows.
- Known Issues in SAM
- Known Issues in ESM
- Known Issues in IP
- Known Issues in MPLS
- Known Issues in NPM
- Common Issues to all products
- Known Issues in Smart Assurance Management Pack
- Known Issues in ASAM
- Known Issues in ACM
Installing SAM console on the Linux platform is not creating applications "VMware Smarts Global Console" icon for quick launch. User needs to go to install directory and type sm_gui command only, the icon option is not available. This appears when previous version 9.5.1. or 9.6 console is already installed.
Default DCF password is not working for DCF SAM-OI communication. Notification will not be pulled from vROps to SAM-OI.
Workaround: Create a new user and password in DCF at
admin1:admin123at users file, and restart the Webservice-Gateway in DCF.
All Services are not installed post upgrade for SAM and SAM-Console.
Refer Services for the Service Assurance Manager section in VMware Smart Assurance Installation Guide to install services manually.
- SMARTA-1088 / SR-20168937511
Cisco ACI related switches are discovered via Cisco APIC are not showing real device name in the display name of its components, instead it shows "node-xxx" as the device name
Resolution: As Cisco APIC provides actual name for the leaf/spine switches, the DisplayName for components of the switches are extracted and populated from the DisplayName of the switches in the discovery post processor. The parameter 'CiscoACIComponentsRenameEnabled' (default value is TRUE) in tpmgr-param.conf controls the components renaming.
SAM 10.1.5 installation succeeds even though mandatory rpm package libnsl.x86_64 required for it is not installed in some of the RHEL 8.x machine.
Post installation server starts, but not able to attach to SAM GUI, and SAM server log shows following error:
[August 11, 2021 6:02:45 PM GMT+00:00 +568ms] t@1682732800 InCharge Framework ASL-W-ERROR_RULE_SOURCE-While executing rule set '/largedisk/InCharge/SAM/smarts/rules/import.asl' ASL-ERROR_ACTION-While executing action at: ASL-CALL_STACK_RULE- RuleName: CALL_PARAMS, Line: 59 ASL-CALL_STACK_RULE- RuleName: CALL_OPERATION, Line: 34 ASL-ERROR_INVOKE-While attempting to invoke operation 'loadModel' of object 'MR_Object::SM-System' CI-ELIBNOTLOAD-Library "sm_ics_xml_conf" could not be loaded CI-ESYSINFO-System information: . libnsl.so.1: cannot open shared object file: No such file or directory RTLD-SI_ELOADERR-Dynamic loader error [August 11, 2021 6:02:45 PM GMT+00:00 +569ms] t@1682732800 InCharge Framework ASL-W-ERROR_RULE_SOURCE-While executing rule set '/largedisk/InCharge/SAM/smarts/rules/import.asl' ASL-ERROR_ACTION-While executing action at: ASL-CALL_STACK_RULE- RuleName: CALL_PARAMS, Line: 59 ASL-CALL_STACK_RULE- RuleName: CALL_OPERATION, Line: 34 ASL-ERROR_INVOKE-While attempting to invoke operation 'loadModel' peof object 'MR_Object::SM-System' CI-ELIBNOTLOAD-Library "sm_map_xmlconfig" could not be loaded CI-ESYSINFO-System information: . libnsl.so.1: cannot open shared object file: No such file or directory RTLD-SI_ELOADERR-Dynamic loader error
Resolution: To resolve the issue, install the following missing rpm package:
yum install libnsl.x86_64
Incase of Console mode installation and upgrade, below irrelevant message is displayed in command prompt which does not have any functional impact:
cwd: C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\Windows cmd: "C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\Windows\resource\jre\bin\java.exe" --add-opens java.base/jdk.internal.loader=ALL-UNNAMED -Xms16777216 -Xmx50331648 -classpath "C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\InstallerData\IAClasses.zip;C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\InstallerData\Execute.zip;C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\Windows\InstallerData\Execute.zip;C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\InstallerData\Resource1.zip;C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\Windows\InstallerData\Resource1.zip;C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\InstallerData;C:\Users\Administrator\AppData\Local\Temp\2\I1559131575\Windows\InstallerData;" com.zerog.lax.LAX "C:/Users/Administrator/AppData/Local/Temp/2/I1559131575/Windows/setup-CONSOLE-10_0_0_0-win.lax" "C:/Users/Administrator/AppData/Local/Temp/2/lax827D.tmp" -i console
The following map error warning message appears for INCHARGE-SA and INCHARGE-OI in respective logs, while starting the server.
WARNING: register package 'map_error' version 468026d2 failed (0x276f4e00) [May 11, 2020 2:50:50 AM EDT +804ms] t@375936768 InCharge Framework FDXAM-*-DXA_TOPOCONC-TopologySync is running in CONCURENT mode with minimum delay=120 seconds. WARNING: register package 'Map_mm' version a1894c1c failed (0x276f4e00) [May 11, 2020 2:50:52 AM EDT +072ms] t@375936768 InCharge Framework
There is no functional impact due to this issue.
The following ASL error message appears for INCHARGE-SA and INCHARGE-SA-PRES server, while starting the servers.
[May 11, 2020 2:50:50 AM EDT +710ms] t@881858304 InCharge Framework ASL-W-ERROR_RULE_SOURCE-While executing rule set '/opt/INCHARGE10120/SAM/smarts/rules/import.asl' ASL-ERROR_ACTION-While executing action at: ASL-CALL_STACK_RULE- RuleName: INSERT, Line: 181 ASL-CALL_STACK_RULE- RuleName: OBJECT_PROPERTY, Line: 177 ASL-CALL_STACK_RULE- RuleName: OBJECT_PROPERTIES, Line: 171 ASL-CALL_STACK_RULE- RuleName: OBJECT_VALUE, Line: 138 ASL-ERROR_INSERT-While attempting to insert into property 'MapViews' of object 'Map_Containment::Map-Containment' MR-CALL_OBJ_PROP-Object: Map-Containment property: Map-Containment; in file "/work/redcurrent/DMT-10.1.2.0/3/smarts/repos/mr/obj.c" at line 2102 MR-KEY_DUPLICATION-violation of uniqueness requirement for a key [May 11, 2020 2:50:52 AM EDT +095ms] t@881858304 InCharge Framework
There is no functional impact due to this issue.
The Rabbit MQ service is not started in SAM, due to the following error:
[root@vmwbgc052 bin]# ./sm_rabbitmq --ignoreme ERROR: epmd error for host vmwbgc052: address (cannot connect to host/port) [root@vmwbgc052 bin]#
Workaround: If the RabitMQ process fails to start, you need to add the following mapping in the /etc/hosts file:
127.0.0.1 <FQDN of the host>
In some hosts, the pre-installed services such as smarts-tomcat and smarts-elasticsearch may fail to start.
Workaround: Restart the ic-serviced daemon process by running the following commands:
/etc/init.d/ic-serviced stop /etc/init.d/ic-serviced stop
Note: If the ic-business-dashboard service in SAM Console does not start, then restart the ic-serviced daemon process by running the earlier commands.
After SAM 10.1.2.0 upgrade, epmd service-related errors appear in the SAM upgrade log:
/opt/InCharge/SAM/smarts/toolbox/OTP/erts-9.2/bin/epmd (Text file busy) at com.zerog.util.expanders.ExpandToDiskZip.ag(Unknown Source) at com.zerog.util.expanders.ExpandToDiskPMZ.ab(Unknown Source)
SMARTS NCM adapter fails to connect to VSA domain managers when there are a higher number of domains registered with the broker. As broker.getDomainManagers() API returning all domain managers attached to the broker and due to the character variable size restriction at the NCM adapter code, there are failures in processing the domain manager list. The acceptable characters are limited to 256.
Workaround: Connect to the broker where there is less number of domain managers registered.
Map nodes are not getting displayed intermittently through dmctl command.
Workaround: The maps instance gets refreshed in dmctl session when the Console GUI is launched and accessed.
When all the EdgeNodes of NSX-T goes down which has Tier 0 Router due to ESX(Host)Down or VM going down then Compute VMs host Unresponsive will not be explained by the Host(ESX) / VM Down RCA alert.
Currently there is no workaround available however RCA alert will be shown only impact to Host unresponsive will be not available for NSX-T Compute VMs. In the impact list EdgeNode Down, LogicalRouter(T0) Down will be shown.
User needs to mandatorily discover ESX Servers for getting Virtual Machine Down event. Currently the Virtual Machine Down event is not generated if the corresponding ESX Servers are not discovered in IP Server. So, its recommended to discover Virtual Machines to get proper Root cause events.
When Kafka is not reachable, monitoring failure event appears in SAM after reconfiguration of ESM Server.
- If Kafka server goes down, a notification for VCD monitoring failure appears in SAM once ESM Server is reconfigured.
- When Kafka is brought back, the failure notification is cleared from SAM once ESM server is reconfigured.
Exception appears when selecting "Tiered application map" by right clicking on any map icon.
Mapping of ElementName attribute to the device for notification on AggregatePort is missing. ElementName appears as Aggregate Port instance name instead of Device name.
Workaround: Detach the INCHARGE-AM-PM from SA and underly it again. Once you re-attach INCHARGE-SA-PRES to the domain manager, you may lose all the active notifications.
The following error message appears in the ESM log while discovering VCD and its dependant components like NSX-T, vCenter, etc..
NV_MESSAGE-*-NV_GENERIC-MSG ERR : [Thread-8 DmtObjectFactory]:insert(TransportNodeInterface::10.107.146.14/10.107.146.69::ConnectedVia += Tunnel:): SVIF-E-EREMOTE-Remote error occurred. See exception chain for detail.; in file "/work/redcurrent/DMT-10.1.2.0/3/smarts/skclient/linux_rhAS50-x86-64/optimize/SmLocalInterfaceHandler.c" at line 3623' should be empty.
Objects deleted in IP are not getting deleted in vROps even after object deletion time interval.
To delete the objects manually:
- Go to Administration > Configuration > Inventory Explorer.
- Select the objects to be deleted and click delete button (Bulk deletion is also supported).
A set of reports are created by default when user installs the Smart Assurance Adapter pack. These reports are retained in the dashboard even after uninstalling the management pack. Ideally, these reports must be deleted as part of the management pack uninstallation process.
User can manually delete the reports from the dashboard.
When a new credentials added without any username and password while configuring an instance of the adapter and edit it later with correct username and password, the password field wont be updated resulting the connection failure.
Edit the credentials again and update the password filed again and save it.
Special characters are present after migration from Windows to Linux.
Patch files are getting migrated, when performing migration on ESM 10.0.0.1 and in IP 10.0.0.2 to 10.1.0.0 respectively.
This is applicable only if migration is triggered on the below two products and patch versions:
- ESM - 10.0.0.1 (ESM 10.0 patch 1)
- IP - 10.0.0.2 (IP 10.0 Patch 2)
When performing migration only on the above two versions, you need to follow the below procedure:
Step 1: Collect the backup from the old installation using sm_migrate utility.
-Run 'sm_perl sm_migrate.pl --old=<dir> --archive=<tar/zip> --sitemod=<local_directories>' from the old installation.
Step 2: Perform manual migration on the customized files.
-Copy the archive file from ‘step1’ to new host.
-Run 'sm_perl sm_migrate.pl --archive=<tar/zip> --new=<dir>' from the new installation.
-Once the sm_migrate utility backs up all the customizations under smarts directory (Eg: /opt/InCharge/IP/smarts/.migrate.bkp.10.0.0.2) and prompts for merging the customizations with below options, the user has to select “N” to not merge.
a. Press 'n' to skip FileMergeUtilty.
b. Press any other key to start FileMergeUtilty...[y]n
Step 3: You need to manually copy customizations from backup directory (Eg: /opt/InCharge/IP/smarts/.migrate.bkp.10.0.0.2) to respective local directory and rename the files to remove ‘.local’ extension.
Null pointer exceptions are observed in IP server log file after Cisco ACI Discovery.
ASL error present in INCHARGE-MPLS-TOPOLOGY after starting server from fresh install.
Post migration from Windows to Linux, EIGRP server is not coming up.
While starting the EIGRP server use option --ignore-restore-errors.
EIGRP classes are not populated in NPM EIGRP, due to ASL error.
During product upgrade, patch folder is not highlighted.
On RHEL 7.8 version, if you start any domain manager as a service, the domain gets registered to a broker using both v4 and v6 IP address space. Due to this issue domain manager v6 entry will go to DEAD state in brcontrol output and the communication between the servers is failing sometimes due to this issue.
Note: Issue also detected on some machines with RHEL 7.2 and 7.6
Workaround: To avoid a domain running in v6 mode, allow only v4, by setting the below flag in runcmd_env.sh file:
Restart the domain manager, after updating runcmd_env.sh file.
"-help" command installer displays following invalid options:
- -i [gui]
- -jvmxms <size>
- -jvmxmx <size>
- -add <feature_name_1> [<feature_name_2 ...]
- -remove <feature_name_1> [<feature_name_2 ...]
Smart Assurance products in silent mode are getting installed in root folder, when user disable or comment the user install directory (<Products>SUITE.installLocation=/opt/InCharge) in silent response file.
Few Log messages are tagged as ERROR in the collector log incorrectly.
The vCenter Management Pack does not have any API to detect floating IP. If a VNF (sprout or bono) is configured with floating IP and private IP, only the private IP is used to establish the relationship between VNFs (P-CSCF or I/S-CSCF) and VirtualMachines.
After successful installation, while viewing the content of management pack the number of dashboards displayed as 0 instead of 11. There is no functional impact due this, you can still view all the management packs in the dashboard area.
ASAM server is not running in service way, but the user able to run via the Server way. When the user installs any application with java 11 enabled on the VM, ic-serviced (sm_serviced) is also installed, which is compiled with java 11. And, when the user installs another application with java 8 installed (in this case ASAM), then the application crashes during the starting of service (ic-asam). This is because all the executables installed under ASAM bin directory are compiled with java 8 which in turn not compatible with ic-serviced, hence crashing only in the service way starting.
Workaround: User needs to stop ic-serviced and start it again from the ASAM bin directory and then again try to start the server in service way. Also, the ASAM must be installed on a fresh VM without any other application installed along with it (except SAM CONSOLE Linux which is also on java 8).
When ACM is upgraded to 10.1.2, ASL error appears while performing full discovery (DiscoverALL) and after restarting the INCHARGE-AM, INCHARGE-OI, and ACM servers. Following error message appears in ACM log:
ASL-W-ERROR_RULE_SOURCE-While executing rule set '/opt/InCharge1012ACM/ACM/smarts/rules/app-sig/standard-probe.asl' ASL-ERROR_ACTION-While executing action at: ASL-CALL_STACK_RULE- RuleName: CREATE_TOPOLOGY, Line: 224 ASL-CALL_STACK_RULE- RuleName: START, Line: 73 ASL-ERROR_INVOKE-While attempting to invoke operation 'makeSoftwareServiceOnHost' of object 'Application_SourceObjectFactory::Application-SourceObjectFactory-AS-IANA-smarts-broker_hostname=10.62.72.138-STANDARD' APPF-NULL_OBJECT-Null Object 'DXA_TopologySource::Application-SourceObjectFactory-AS-IANA-smarts-broker_hostname=10.62.72.138-STANDARD'