To deploy the Metric Exporters on the cluster, you have to perform the following actions.

Procedure

  1. To deploy cAdvisor for monitoring Windows containers using the YAML, see cAdvisor YAML Definition.
    Note: Ensure that port 4914 is set as the host port.
    apiVersion: apps/v1 # apps/v1beta2 in Kube 1.8, extensions/v1beta1 in Kube < 1.8
    kind: DaemonSet
    metadata:
      name: cadvisor
      namespace: kube-system
      labels:
        app: cadvisor
      annotations:
          seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
    spec:
      selector:
        matchLabels:
          app: cadvisor
      template:
        metadata:
          labels:
            app: cadvisor
            version: v0.31.0
        spec:
          tolerations:
          - key: node-role.kubernetes.io/master
            effect: NoSchedule
          containers:
          - name: cadvisor
            image: google/cadvisor:v0.31.0
            resources:
              requests:
                memory: 250Mi
                cpu: 250m
              limits:
                cpu: 400m
            volumeMounts:
            - name: rootfs
              mountPath: /rootfs
              readOnly: true
            - name: var-run
              mountPath: /var/run
              readOnly: true
            - name: sys
              mountPath: /sys
              readOnly: true
            - name: docker
              mountPath: /var/lib/docker  #Mouting Docker volume 
              readOnly: true
            - name: disk
              mountPath: /dev/disk
              readOnly: true
            ports:
              - name: http
                containerPort: 8080 #Port exposed
                hostPort : 4194 #Host's port - Port to expose your cAdvisor DaemonSet on each node
                protocol: TCP
          automountServiceAccountToken: false
          terminationGracePeriodSeconds: 30
          volumes:
          - name: rootfs
            hostPath:
              path: /
          - name: var-run
            hostPath:
              path: /var/run
          - name: sys
            hostPath:
              path: /sys
          - name: docker
            hostPath:
              path: /var/lib/docker #Docker path in Host System
          - name: disk
            hostPath:
              path: /dev/disk
  2. Use CstatsExporter to fetch the windows container metrics from this link.
    You can fetch the container metrics in two ways.
  3. Deploy directly with docker using the following command.
    1. Run docker from here.
    2. run --rm -p 9030:9030 -v \\.\pipe\docker_engine:\\.\pipe\docker_engine projects.registry.vmware.com/vrops_metric_exporters/cstatsexporter@sha256:1547a44857612c616ab14cde945326ffb9497abd5541737ff7fd4f3e2af83226
  4. Deploy the cstats container as Deployment in Kubernetes for each worker node.
    1. Create cstatsexporter.yaml file as given in the following proxy or jumper.
      Daemaonset type:
      
      apiVersion: apps/v1
      kind: DaemonSet
      metadata:
        name: cstatsexporter
        namespace: kube-system
        labels:
          app: cstatsexporter
        annotations:
            seccomp.security.alpha.kubernetes.io/pod: 'docker/default'
      spec:
        selector:
          matchLabels:
            app: cstatsexporter
        template:
          metadata:
            labels:
              app: cstatsexporter
              version: v0.31.0
          spec:
            containers:
            - name: cstatsexporter
              image: projects.registry.vmware.com/vrops_metric_exporters/cstatsexporter@sha256:1547a44857612c616ab14cde945326ffb9497abd5541737ff7fd4f3e2af83226
              volumeMounts:
              - name: docker
                mountPath: \\.\pipe\docker_engine
              ports:
                - name: http
                  containerPort: 9030
                  protocol: TCP
            automountServiceAccountToken: false
            terminationGracePeriodSeconds: 30
            volumes:
            - name: docker
              hostPath:
                path: \\.\pipe\docker_engine
                type: null
            nodeSelector:
              kubernetes.io/os: windows
            tolerations:
            - key: "windows"
              operator: "Equal"
              value: "2019"
              effect: "NoSchedule"
      ---
      apiVersion: v1
      kind: Service
      metadata:
        name: cstatsexporter
        namespace: kube-system
        labels:
          app: cstatsexporter
      spec:
        externalTrafficPolicy: Local
        ports:
          - port: 9030
            targetPort: 9030
            nodePort: 31198
        selector:
          app: cstatsexporter
        type: NodePort
      ---
  5. Run kubectl get all -o wide to see where the cstats container is created.
  6. Copy the IP of cstats container node and the port on which the cstats service is exposed.
  7. Add it as a job in the prometheus.yml file in the Prometheus server.
    Limitations
    • Metrics limitation is observed when CstatExporter is running as Daemonset with a single NodePort service.
    • Inconsistency is noticed in the behavior of the services where we sometimes do not get metric for some node for some period of time.
    • To get metrics for all the nodes consistently, deploy cstatsExporter as a Docker container on each windows node.