Topology Size
| Deployment | Number of Devices | Number of Events/Days | Number of Metrics/5 min |
|---|---|---|---|
| Extra Small | 2500 | 2500 | 1M |
Test Bed
| Topology and Events: Simulation Based on the Smarts Domain Manager repositories | Metrics: Simulation of Raw Metrics and Net Flows | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Deployment | Routers | Managed P&I | Hosts | VMs | Edges | Total No of Events=Number of Devices * 4 + 5k External Events (INCHARGE-OI) | Number of raw metrics from ( Smarts metric collector and VeloCloud collector) | Number of metrics from M&R Gateway(Flows+Metrics(SNMP, vCenter) | Number of Traffic Flows(Native) |
| Extra Small | 1500 | 30K | 200 | 500 | 300 | 15K | 570K | 400K | 350 |
Test Results for Notifications Console
| Sl No. | Test Case | Expected Result | Measured Result |
|---|---|---|---|
| 1 | Data synchronization of Smarts 10.X notifications with VMware Telco Cloud Operations (All Smarts 10.X notifications and their states should be synchronized with VMware Telco Cloud Operations). Measured with end to end (SAM to VMware Telco Cloud Operations UI without manual refresh). | 15 min | 12 min 30 sec |
| 2 | Time taken for all notifications to show up in Notification Console (Measured to launch first 100 events). | 5s | 3s |
| 3 | Time taken to populate data in a newly added notification view with set of filters. (Add 5 notification views with set of filters.) | 3s | 3s |
| 4 | Time taken to populate data in a newly added notification console with set of filters. (Add 5 notification consoles with set of filters.) | 3s | 4s |
| 5 | Time taken to populate data in a newly added summary view with set of filters. (Add 5 notification summary views with set of filters.) | 3s | 4s |
| 6 | Time taken to expand any root-cause notification to show all impacted notifications. (For XS, S, M RCA with - 16, 24, 25 impacts respectively.) | 3s | 2s |
| 7 | Time taken to launch and show browse details (all tabs should refresh with data) of a selected notification. (For XS, S, M RCA with - 16, 24, 25 Impacts respectively.) | 3s | 6s |
| 8 | Time taken to refresh the notification console with filter set on any column with string data | 3s | 3s |
| 9 | Time taken to Ack, Take Ownership of bulk (100) notifications. | 5s | 16s |
Test Results for Topology Maps
| Sl No. | Test Case | Expected Result | Measured Result |
|---|---|---|---|
| 1 | Data synchronization of Smarts 10.X topology with VMware Telco Cloud Operations (All Smarts 10.X topology and their relations should be synchronized with VMware Telco Cloud Operations). | 5 min | 3 min |
| 2 | Time taken for Smarts 10.X topology to show up in Topology Browser. | 5s | 2s |
| 3 | Time taken to expand classes with more than 1000 instances in Topology Browser. | 5s | 4s |
| 4 | Time taken to show the instances of class with more than 1000 instances in Topology Map drop down box. | 5s | 2s |
| 5 | Time taken to render a Topology map of a selected instance with ~50 connected/associated instances (example: VirtualCenter). | 5s | 7s |
| 6 | Time taken to expand selected map in #5 test case with 2 hop counts (Physical Connectivity Map) Nodes ~ 500, Edges ~ 10k. | 5s | 4s |
| 7 | Time taken to display the details (all tabs should refresh with data) of any topology node in #6 test case. | 5s | 4s |
Test Results for Dashboards & Reports
| Sl No. | All Test Cases ( Tested with 2 Weeks of Data and Default Time) | Expected Result | Measured Result |
|---|---|---|---|
| 1 | Time taken to render Network Summary dashboard (Landing Page) | 5s | 6s |
| 2 | Time taken to refresh Network Summary dashboard | 5s | 5s |
| 3 | Time taken to render Inventory dashboard (Network Inventory) | 5s | 11s |
| 4 | Time taken to refresh Inventory dashboard | 5s | 10s |
| 5 | Time taken to render SDWAN dashboard | 5s | 5s |
| 6 | Time taken to refresh SDWAN dashboard | 5s | 4s |
| 7 | Time taken to render KPI dashboard | 5s | 1s |
| 8 | Time taken to refresh KPI dashboard | 5s | 1s |
| 9 | Time taken to render “vCenter” dashboard | 5s | 4s |
| 10 | Time taken to refresh vCenter dashboard | 5s | 2s |
| 11 | Time taken to render VMware Telco Cloud Operations MnR Gateway Traffic Flow dashboard | 5s | 4s |
| 12 | Time taken to refresh VMware Telco Cloud Operations MnR Gateway Traffic Flow dashboard | 5s | 2s |
| 13 | Time taken to render VMware Telco Cloud Operations MnR Gateway Traffic Flow Flow Top N Summary | 5s | 3s |
| 14 | Time taken to refresh VMware Telco Cloud Operations MnR Gateway Traffic Flow, Flow Top N Summary | 5s | 2s |
| 15 | Time taken to render VMware Telco Cloud Operations MnR Gateway Traffic Flow Flow Trends | 5s | 3s |
| 16 | Time taken to refresh VMware Telco Cloud Operations MnR Gateway Traffic Flow Flow Trends | 5s | 2s |
| 17 | Time taken to render VMware Telco Cloud Operations Native Netflow dashboard | 5s | 3s |
| 18 | Time taken to refresh VMware Telco Cloud Operations Native Netflow dashboard | 5s | 2s |
Test Results for Collectors and Gateway
| Sl No. | Test Case | Expected Result | Measured Result |
|---|---|---|---|
| 1 | Number of VeloCloud discovery collectors (6hr polling period). | 1 | 1 |
| 2 | Number of VeloCloud monitoring collectors (5min polling period). | 1 | 1 |
| 3 | Number of VeloCloud performance collectors (5min polling period). | 1 | 1 |
| 4 | Number of NetFlow collectors (5min polling period). | 1 | 1 |
| 5 | Number of M&R Gateway collectors ( Traffic Flow, vCenter SP, and SNMP Custom Collector ( 1 GW Collector for Metric data and 1 GW Collector for traffice flow Event data). | 1 | 2 |
| 7 | Polling Period for Smarts Metric collector to collect the data from Domain Managers . | 300s | 210s |
Resource Utilization
| Node Name | Node CPU % | Node Memory % | POD Name (Services) | POD CPU utilization % | POD Memory utilization % |
|---|---|---|---|---|---|
| arangoworker | 78 | 34 | alerting-rest-9c76b8575-n9whb | 0.1 | 1.6 |
| apiservice-69dbf6c7b5-lx74z | 0.1 | 5.0 | |||
| arangodb-755f6b74bf-x9zwk | 0.3 | 9.4 | |||
| catalog-656d6b87c7-cbkgm | 0.2 | 1.4 | |||
| events-5f65fc5599-txjdd | 39.8 | 0.5 | |||
| jobmanager-5774f44f9b-5hzmm | 0.3 | 2.7 | |||
| k4m-rest-7f4b4db56b-fbxh7 | 0.2 | 1.5 | |||
| omega-enrichment-rest-7758795bbd-ch2bq | 0.1 | 1.1 | |||
| redis-59498cbf97-d9sw2 | 0.2 | 0.3 | |||
| taskmanager-85f6d7587c-nfv4p | 36.0 | 9.1 | |||
| tco-node-monitor-zc6qd | 0.1 | 0.1 | |||
| tco-self-monitoring-7c4cfcb95c-6scr8 | 0.1 | 0.1 | |||
| domainmanagers | 75 | 58 | aci-pm-6d45476944-prgd4 | 1.6 | 1.4 |
| analytics-service-546bcccb54-fpqgs | 0.1 | 0.4 | |||
| cisco-aci19125d-dm-699788d564-98z2b | 1.4 | 1.4 | |||
| cisco-acic6ce72-dm-5669875c87-5txck | 25.1 | 1.0 | |||
| collector-manager-6979f8c8f4-jjxrg | 0.2 | 0.8 | |||
| datamgmt | 0.2 | 0.2 | |||
| gateway-trafficflow-7465488df7-6dw7c | 0.6 | 1.1 | |||
| gateway-vcenter-custom-945cd9cbd-hn85k | 0.8 | 2.3 | |||
| kafka-edge-7554ffd95c-pmlfb | 1.3 | 21.1 | |||
| metrics-velocloud-7459f65b8d-xtsqp | 0.6 | 6.0 | |||
| metrics-viptela-5fc5c89679-hmdvb | 0.9 | 2.2 | |||
| smarts-event-data-smarts-sam-5c69bc565-vgd7x | 0.2 | 1.7 | |||
| smarts-metric-ampmnewus2500-7df46f656b-zmjxx | 9.4 | 2.5 | |||
| smarts-metric-incharge-am-pm-6bdd649f6-vmhng | 0.2 | 1.3 | |||
| smarts-topology-data-smarts-sam-5688fc7b4c-mwk2v | 0.2 | 1.3 | |||
| tco-node-monitor-vhwll | 0.2 | 0.1 | |||
| velocloud-sdwan336432-dm-c688bc-qmc2j | 0.4 | 3.3 | |||
| velocloud-sdwanaff28a-dm-78d8f8745d-p65jt | 9.2 | 4.6 | |||
| viptela-sdwan078fee-dm-79b9b765bd-wvspb | 0.2 | 1.1 | |||
| viptela-sdwan188886-dm-6dddfdb478-8rk82 | 20.8 | 1.0 | |||
| zookeeper-edge-5c799f7988-8v5ff | 0.2 | 0.7 | |||
| elasticworker | 8 | 40 | elasticsearch-69469cbfbb-rwlkf | 7.7 | 40.3 |
| tco-node-monitor-wj7xm | 0.2 | 0.1 | |||
| kafkaworker | 28 | 87 | kafka-79fd666d56-c8sxw | 12.1 | 30.9 |
| metric-5cd955dd54-jjhnm | 10.6 | 6.2 | |||
| metric-gateway-77b5c88dcb-llv7p | 1.3 | 10.7 | |||
| persistence-664bfb685d-fmvhj | 0.3 | 38.4 | |||
| tco-node-monitor-78kfx | 0.2 | 0.1 | |||
| topology-658794bf84-9wq6c | 0.1 | 1.1 | |||
| vmware-photon-os | 15 | 30 | authserver-747c44c757-ftw7q | 1.1 | 2.0 |
| grafana-df7d68496-cb2cf | 1.1 | 0.4 | |||
| nginx-59c76f6788-phr29 | 2.1 | 0.4 | |||
| omega-patching-rest-566cf497bb-6k7rk | 0.2 | 2.6 | |||
| tco-node-monitor-9kdd7 | 0.1 | 0.1 | |||
| zookeeper-7c5cc95c97-j67nw | 1.2 | 1.5 |
