Removing unneeded minions from a Carbon Black EDR cluster reduces the overall cluster size and simplifies the cluster deployment. This improves cluster performance by reducing network overhead.

Best Practices

Adhere to these best practices before attempting to remove minions from a Carbon Black EDR cluster.

Consult the VMware Carbon Black EDR Server Operating Environment Requirements Guide to make sure that the reduced cluster has the capacity to support the total number of endpoints.

  • The remaining minions must have enough disk space to contain the full data.
    • Calculate the daily usage of the disk per sensor and re-calculate the required disk space to new minions for full retention. For example:

      Assume you want to collapse the six-minion cluster that has 40K endpoints down to three minions with 30-day retention. Carbon Black EDR v6.0+ supports this because each minion still has less than the maximum of 18,750 endpoints.

      You calculate the current daily disk usage per sensor to be 20MB (dividing total cluster data volume storage by current daily retention and number of endpoints).

      The new total storage requires a minimum of 20MB * 40K (endpoints) * 30 (days) = 24 TB (or 8 TB per node).

      Add 20-30% to the available disk space on top of the calculated amount to allow for increased sensor activity in the future.

    • Additional extra disk space must be provided for desired cold storage in days.
  • The primary cluster node must have enough disk space to contain binary files for all removed minions. Binary files typically take less space than events, but should still be taken into account. You can calculate the required extra space on the primary node by logging into Carbon Black EDR, clicking Server Dashboard in the left navigation bar, and viewing Storage Statistics for the minions. To obtain the maximum additional space required on the primary node, add up the total binary size on the minions that are to be removed.
  • To support the sensor load, the remaining minions must have sufficient CPU and memory.

Read-Only Minions

Before removing minions from a Carbon Black EDR cluster, you must convert them to a read-only state.

Read-only minions are searchable throughout the API and user interface, but are not used for sensor checkins or event/datastore pushes. When minions are in the read-only state, the system copies the binary files to the primary node.

The event data is not added to the minions and existing data is purged periodically as in normal operations. Therefore, the read-only minions become completely inactive after the retention period. After the desired data retention period expires, you can safely remove the read-only minions.

You can revert read-only minions to read-write minions at any time.

Remove Minions

Perform the following procedure to mark minions as read-only and remove them. Removed minions are no longer part of the cluster and can be shut down and de-provisioned.

You must first stop the cluster service, set the minions as read-only, and restart the cluster service. Then, stop the cluster service again and remove the minion. Finally, restart the cluster service.

Prerequisites

Note:
  • The node_id parameter can be found in /etc/cb/cluster.conf.
  • Only read-only minions can be removed.
  • The system can remove a minion only if all storefiles have been copied to the primary node. If this is not the case, an error message states how many more files must be moved.

Procedure

  1. Stop the cluster service.
    /usr/share/cb/cbcluster stop
  2. For each minion to remove, run the following command:
    /usr/share/cb/cbcluster change-node -N {node_id} -R True
  3. (Optional) If you have an eventless primary node and want it to contain events, run the following command:
    /usr/share/cb/cbcluster change-node -E True
  4. Start the cluster service.
    /usr/share/cb/cbcluster start
  5. Stop the cluster service.
    /usr/share/cb/cbcluster stop
  6. For each minion to remove, run the following command:
    /usr/share/cb/cbcluster remove-node -N {node_id}
  7. Start the cluster service.
    /usr/share/cb/cbcluster start