etcd Health

In this topic you can find different etcd commands to run on control-plane nodes.

Execute the following command to check the performance of etcd node.

[tco@node1 log]$ sudo ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/admin-node1.pem --key=/etc/ssl/etcd/ssl/admin-node1-key.pem check perf


 60 / 60 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00% 1m0s
PASS: Throughput is 150 writes/s
PASS: Slowest request took 0.077849s
PASS: Stddev is 0.004243s
PASS 

Execute the following command to list down the members of etcd cluster.

[tco@node1 log]$ sudo ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/admin-node1.pem --key=/etc/ssl/etcd/ssl/admin-node1-key.pem  member list --write-out="json" | jq


{
  "header": {
    "cluster_id": 6628172681314379000,
    "member_id": 5715020266194329000,
    "raft_term": 3
  },
  "members": [
    {
      "ID": 4649502993639389000,
      "name": "etcd2",
      "peerURLs": [
        https://10.225.66.153:2380
      ],
      "clientURLs": [
        https://10.225.66.153:2379
      ]
    },
    {
      "ID": 5715020266194329000,
      "name": "etcd1",
      "peerURLs": [
        https://10.225.67.36:2380
      ],
      "clientURLs": [
        https://10.225.67.36:2379
      ]
    },
    {
      "ID": 10732259215949154000,
      "name": "etcd3",
      "peerURLs": [
        https://10.225.66.226:2380
      ],
      "clientURLs": [
        https://10.225.66.226:2379
      ]
    }
  ]
}

Execute the following command to verify the endpoint status of etcd.

[tco@node1 log]$ sudo ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/admin-node1.pem --key=/etc/ssl/etcd/ssl/admin-node1-key.pem endpoint --cluster status --write-out=json | jq


[
  {
    "Endpoint": https://10.225.66.153:2379,
    "Status": {
      "header": {
        "cluster_id": 6628172681314379000,
        "member_id": 4649502993639389000,
        "revision": 809355,
        "raft_term": 3
      },
      "version": "3.5.6",
      "dbSize": 161087488,
      "leader": 4649502993639389000,
      "raftIndex": 865985,
      "raftTerm": 3,
      "raftAppliedIndex": 865985,
      "dbSizeInUse": 21581824
    }
  },
  {
    "Endpoint": https://10.225.67.36:2379,
    "Status": {
      "header": {
        "cluster_id": 6628172681314379000,
        "member_id": 5715020266194329000,
        "revision": 809355,
        "raft_term": 3
      },
      "version": "3.5.6",
      "dbSize": 161058816,
      "leader": 4649502993639389000,
      "raftIndex": 865985,
      "raftTerm": 3,
      "raftAppliedIndex": 865985,
      "dbSizeInUse": 21573632
    }
  },
  {
    "Endpoint": https://10.225.66.226:2379,
    "Status": {
      "header": {
        "cluster_id": 6628172681314379000,
        "member_id": 10732259215949154000,
        "revision": 809355,
        "raft_term": 3
      },
      "version": "3.5.6",
      "dbSize": 161112064,
      "leader": 4649502993639389000,
      "raftIndex": 865985,
      "raftTerm": 3,
      "raftAppliedIndex": 865985,
      "dbSizeInUse": 21569536
    }
  }
] 

Execute the following command to check the overall health of the etcd cluster.

[tco@node1 log]$ sudo ETCDCTL_API=3 /usr/local/bin/etcdctl --endpoints=https://127.0.0.1:2379 --cacert=/etc/ssl/etcd/ssl/ca.pem --cert=/etc/ssl/etcd/ssl/admin-node1.pem --key=/etc/ssl/etcd/ssl/admin-node1-key.pem endpoint --cluster health --write-out=json | jq


[
  {
    "endpoint": https://10.225.66.226:2379,
    "health": true,
    "took": "18.429913ms"
  },
  {
    "endpoint": https://10.225.66.153:2379,
    "health": true,
    "took": "18.769281ms"
  },
  {
    "endpoint": https://10.225.67.36:2379,
    "health": true,
    "took": "19.441441ms"
  }
] 

Execute the following command to gather other etcd related information.

[tco@node1 log]$ sudo curl  --cacert /etc/ssl/etcd/ssl/ca.pem --cert /etc/ssl/etcd/ssl/admin-node1.pem --key /etc/ssl/etcd/ssl/admin-node1-key.pem https://localhost:2379/metrics