Data Security

This topic describes how Tanzu Telemetry for Ops Manager transmits, stores, and secures collected data.

Overview

Tanzu Telemetry for Ops Manager collects data from installed products in your foundation. It does not collect any personal data or information related to an identified or identifiable person. For example, it does not collect data on passwords, private keys, or other authentication-type tools.

Security Measures

Data collected by the Tanzu Telemetry for Ops Manager undergoes rigorous vetting and validation. Any data that has not been approved will not be stored by VMware. Approved data is stored securely in Google Cloud Platform (GCP).

Tanzu Telemetry for Ops Manager secures data during transmission using the following measures:

  • HTTPS connections when communicating
  • User-specific credentials to authenticate HTTPS requests to VMware

Audit Mode

By default, data is only temporarily on disk before being sent. When Audit Mode is enabled, data will not be sent to VMware and will be collected on the telemetry-centralizer instance disk.

To access the data in Audit Mode, bosh ssh to the telemetry-centralizer instance. If data collection has occurred, it may be in two locations:

Log File

At the path /var/vcap/sys/log/telemetry-centralizer/audit.log, telemetry data will be newline separated containing a JSON parsable event. Not all possible telemetry may be in this file, as the telemetry is often event based.

Tar File

At the path /var/vcap/data/telemetry-collector/FoundationDetails_######.tar, data in the same format as the data from the Telemetry Collector will be present if collection has been successful in the last 24 hours. The file will contain data from Ops Manager and optionally the Usage service as JSON files separated by folders.

What Data is Collected

From Ops Manager
API Endpoint Data Collected
/api/v0/diagnostic_report
  • Ops Manager version
  • BOSH stemcell
  • IaaS type
  • Deployed and staged products, versions, and stemcells
  • BOSH director configuration details
  • BOSH releases
  • NTP servers is filtered out
  • Documentation
/api/v0/staged/products/:product_guid/properties
/api/v0/staged/products/:product_guid/resources
  • Resources
  • List of the compute and disk configurations for all jobs on the installed products
  • Documentation
/api/v0/vm_types
  • Details about VMs used on your IaaS, including:
    • Name
    • RAM
    • CPU
    • Ephemeral disk
    • Built in (true/false)
  • Documentation
/api/v0/deployed/products
  • List of all deployed tiles, including:
    • Name
    • Version
    • Product guid
  • Documentation
/api/v0/installations
  • Event history for tile changes, including:
    • Change type (e.g. - “upgrade”)
    • Start time
    • End time
    • Change status
  • Username is filtered out
  • Documentation
/api/v0/deployed/certificates
  • Details about deployed certificates:
    • Issuer
    • Valid start and end dates
    • Configurable
    • Property reference
    • Property type
    • Property id
  • Documentation
/api/v0/certificate_authorities
  • Details about certificate authorities:
    • Id
    • Issuer
    • Created and expired dates
    • Active status
  • Cert_pem and Nats_cert_pem are filtered out
  • Documentation
/api/v0/staged/pending_changes
  • List of all pending changes, including:
    • Product GUID
    • Product Action
    • Last Deployed State
    • Product Errands
    • Product Stemcells
  • Documentation
From Usage Service

Tanzu Telemetry for Ops Manager can be optionally configured to collect information about application instances, tasks, and service instances from the Usage Service.

API Endpoint Data Collected
/system_report/app_usages
  • System-wide app usage data:
    • App instance hours
    • Average app instances
    • Maximum app instances
    • Documentation
/system_report/task_usages
  • System-wide task usage data:
/system_report/service_usages
  • System-wide service usage data:
    • Service name
    • Service GUID
    • Duration in hours
    • Average service instances
    • Maximum service instances
    • Service plan usage:
      • Service plan GUID
      • Service plan duration in hours
      • Service plan average service instances
      • Service plan maximum service instances
      • Service plan name is filtered out
    • Documentation
From Cloud Controller API (CAPI)

Create App

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "create-app": {
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-11-28T02:00:50+00:00"
}

Create Build

App Id
Unique anonymized app id used to tie events to an app.*

Build Id
Unique anonymized build id used to tie events to a build.*

Lifecycle
Name of the build lifecycle used - ex. buildpack, docker

Stack
Name of stack used in build - cflinuxfs2, cflinuxfs3, windows

User Id
Unique anonymized user id used to tie events to a user.*
{
  "create-build": {
    "app-id": "anon-app-id",
    "build-id": "anon-build-id",
    "buildpacks": ["ruby_buildpack"],
    "lifecycle": "buildpack",
    "stack": "cflinuxfs3",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-06T05:16:51+00:00"
}

Build Completed

App Id
Unique anonymized app id used to tie events to an app.*

Build Id
Unique anonymized build id used to tie events to a build.*

Lifecycle
Name of the build lifecycle used - ex. buildpack, docker.

Stack
Name of stack used in build - cflinuxfs2, cflinuxfs3, windows.

Buildpacks
Names of buildpacks used - this is not a preset list as this list can be configured by each customer differently.
{
  "build-completed": {
    "lifecycle": "buildpack",
    "buildpacks": ["go_buildpack"],
    "stack": "cflinuxfs3",
    "app-id": "anon-app-id",
    "build-id": "anon-build-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-11-28T02:00:32+00:00"
}

Create Deployment

App Id
Unique anonymized app id used to tie events to an app.*

Strategy
Deployment strategy used - right now only limited to rolling - future values could be blue/green and stop/start.

User Id
Unique anonymized user id used to tie events to a user.*
{
  "create-deployment": {
    "app-id": "anon-app-id",
    "strategy": "rolling",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-06T23:15:54+00:00"
}

Bind Service

App Id
Unique anonymized app id used to tie events to an app.*

Service Id
Unique anonymized service id used to tie events to a service that exists on the platform.*

Service Instance Id
Unique anonymized service instance id used to tie events to a service instance bound to the app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "bind-service": {
    "app-id": "anon-app-id",
    "service-id": "anon-service-id",
    "service-instance-id": "anon-inst-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Create Task

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "create-task": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Delete App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "delete-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Restage App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

Lifecycle
Name of the build lifecycle used - ex. buildpack, docker.

Stack
Name of stack used in build - cflinuxfs2, cflinuxfs3, windows.

Buildpacks
The buildpacks that are used when an app is restaged (this is only for auto detected buildpacks).

User Id
Unique anonymized user id used to tie events to a user.*

{
  "restage-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "lifecycle": "some-lifecycle",
    "stack": "some-stack",
    "buildpacks": ["some-buildpack"],
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Restart App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "restart-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Rolled Back App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

Revision id
Unique anonymized revision id used to tie events to an revision.*

Strategy
Deployment strategy used - right now only limited to rolling - future values could be blue/green and stop/start.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "rolled_back_app": {
    "api_version": "some-version",
    "app_id": "anon-app-id",
    "revision_id": "anon-revision-id",
    "strategy": "strategy-id",
    "user_id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Scale App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

Disk in MB
Amount in MB an app is scaled in disk storage.

Instance Count
Number of app instances.

Memory in MB
Amount in MB an app is scaled in memory.

Process Type
The process that was scaled.

User Id
Unique anonymized user id used to tie events to a user.*
{
  "scale-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "disk-in-mb": 1024,
    "instance-count": 1,
    "memory-in-mb": 1024,
    "process-type": "some-process-type",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Start App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "start-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Stop App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "stop-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Update App

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "update-app": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

Upload Package

API Version
Version of the Cloud Controller API.*

App Id
Unique anonymized app id used to tie events to an app.*

User Id
Unique anonymized user id used to tie events to a user.*
{
  "upload-package": {
    "api-version": "some-version",
    "app-id": "anon-app-id",
    "user-id": "anon-user-id"
  },
  "telemetry-source": "cloud_controller_ng",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}
From On Demand Service Broker(s)

On Demand Broker Event

Operation
The operation type describing the event.

Item
The component involved in the event.

Service Instances -> total
Count of service instances.

Service Instances per Plan -> plan_id
An operator provided name that is used to identify a plan of a service offering, by the developers on a given foundation.

Service Instances per Plan -> total
Count of service instances for the corresponding plan.

Name
An operator provided name that is used to identify a service offering.

{
  "event": {
    "operation": "some-operation",
    "item": "instance"
  },
  "service_instances": {
    "total": 2
  },
  "service_instances_per_plan": {
    "plan_id": "some-plan-id",
    "total": 1
  },
  "service_offering": {
    "name": "some-service-name"
  },
  "telemetry-source": "on_demand_broker",
  "telemetry-time": "2019-12-02T21:52:55+00:00"
}

*Some information is anonymized using a SHA256 hash due to the possible sensitivity of the data. "Anonymization" is the process of concealing sensitive/personal/valuable information by replacing it with a non-invertible hash of the data.

check-circle-line exclamation-circle-line close-line
Scroll to top icon