Replica pruning occurs after the blocks are safely stored in the Full Copy Client. The pruning operation involves two steps. First, collect the latest pruneable block IDs from all the Replica and the Full Copy Client nodes. Secondly, create the prune command, which includes a list of the latest pruneable block IDs.

Upon receiving the prune command, all the Replica nodes verify the list of the latest pruneable block IDs and prune up to the minimal pruneable block ID.

The Full Copy Client, which contains the blockchain data starting from the genesis block, is not pruned.

Note:

Data pruning on Replica nodes is irreversible. After the data is pruned from the Replica nodes, the data is no longer available to the Client nodes. As a best practice, back up the Replica nodes before initiating a pruning process.

Pruning is enabled by default. You can configure pruning using block numbers to keep or a time parameter. The default configuration is to keep the data from the last two weeks and prune the rest.

The default Concord container pruning parameter values are as follows:

  • pruning_enabled: true

  • pruning_duration_to_keep_minutes: 20160

  • pruning_num_blocks_to_keep: 0

You can edit the pruning parameters on each Replica node using the command docker exec -it concord /bin/bash and  vi /concord/config/application.config.

Prerequisites

Procedure

  1. (Optional) Start the operator container.
    image=$(docker images --format"{{.Repository}}:{{.Tag}}"| grep"operator");docker run -d --name=operator -v/config/daml-ledger-api/concord-operator:/operator/config-local-v/config/daml-ledger-api/concord-operator:/concord/config-public-v/config/daml-ledger-api/config-local/cert:/config/daml-ledger-api/config-local/cert-v/config/daml-ledger-api/config-public:/operator/config-public$image
  2. (Optional) Verify the current pruning parameter values.
    sudo docker exec-it concord cat/concord/config/application.config | grep-iE 'pruning_'

    The default number of blocks or the duration to keep the blocks must be set to 0.

  3. (Optional) Run the configuration command to change the parameter values.
    sudo docker exec-it concord sed-i 's/<parameter>: <original value>/<parameter>: <new value>/g'/concord/config/application.config

    For example, you can change the duration time to keep stale key values from two weeks to one week.

    sudo docker exec-it concord sed-i 's/pruning_duration_to_keep_minutes: 20160/pruning_duration_to_keep_minutes: 10080/g'/concord/config/application.config
  4. (Optional) Start all the Replica nodes for the new pruning parameters to take effect.

    curl -X POST 127.0.0.1:8546/api/node/management?action=start

  5. Verify the Replica nodes status.
    sudo docker exec-it telegraf curl -s http://concord:9891/metrics| grep-ia last_block | tail-1
    sudo docker exec-it concord sh -c './concord-ctl status get state-transfer'| grepfetchingState
    sudo docker exec-it concord sh -c './concord-ctl status get replica'| grep-ia 'lastStableSeqNum'
    sudo docker logs --since 1m -f concord | grep-ia addBlock | cut-d '|'-f 3,10
  6. Identify the latest block for pruning in each Replica or Full Copy Client node.

    If a Replica node is lagging, it shows a smaller large pruneable block than the rest. Wait for the node to catch up using state transfer or data restore.

    sudo docker exec-it operator sh -c './concop prune latestPruneableBlock'

    The output contains the Replica or Full Copy Client node block_id and signature.

  7. Stop containers except for the agent and operator on all the Client nodes.

    curl -X POST 127.0.0.1:8546/api/node/management?action=stop

  8. Initiate the pruning process.
    sudo docker exec-it operator sh -c './concop prune start'{"pruned_block":1002,"succ":true}
  9. Check whether the pruning process is successful on all the Replica nodes.
    sudo docker exec-it operator sh -c './concop prune status'

    The output for each Replica node contains in_progress and last_pruned_block.

  10. Start all the containers on each Client node.
    curl -X POST 127.0.0.1:8546/api/node/management?action=start
  11. Log in to Wavefront.
  12. Verify the Merkle keys, versioned keys, and immutable keys metrics for each block.

    The metrics report shows the total number of stale and pruned keys for each category. The metrics indicate that the number of pruned keys has increased after a successful pruning process, and the number of stale keys has decreased.

Results

When the pruning process is successful, you can see the following changes in the system:

  • The genesis block changes to a larger value.

  • RocksDB disk size decreases after some time.

  • Keys related to active smart contracts are not affected regardless of their age.

What to do next

You can prune outdated data on the Client node. See Pruning Client Nodes on vSphere.