Managing Resource Usage with Quotas

Warning: This feature is a beta component and is intended for evaluation and test purposes only. Do not use this feature in a production environment. Product support and future availability are not guaranteed for beta components.

This topic describes how to review and restrict the usage of VMware Tanzu Kubernetes Grid Integrated Edition (TKGI) resources by TKGI users.

Overview

As an Tanzu Kubernetes Grid Integrated Edition administrator, you can set a limit on each user’s total resource allocation within Tanzu Kubernetes Grid Integrated Edition.

You manage resources in Tanzu Kubernetes Grid Integrated Edition by defining quotas for individual users with the TKGI API.

The quotas API endpoint allows you to restrict the total amount of memory and number of CPUs that a user can allocate in total across their deployed clusters.

In addition, you can limit the total number of clusters a user can provision within Tanzu Kubernetes Grid Integrated Edition.

To review overall resource usage and for individual users, you access the TKGI API usages endpoint.

Note: Quota settings affect only non-admin user accounts. A quota applied to an admin user account is ignored.

Set up Your API Access Token

The curl commands in this topic use an access token environment variable to authenticate into the TKGI API.

1. To export your access token into an environment variable, run the following command:

   tkgi login -a TKGI-API -u USER-ID -p 'PASSWORD' -k; \
   export YOUR-ACCESS-TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
   

   Where:

   • TKGI-API is the FQDN of your TKGI API endpoint. For example, api.tkgi.example.com.
   • USER-ID is your Tanzu Kubernetes Grid Integrated Edition user ID.
   • PASSWORD is your Tanzu Kubernetes Grid Integrated Edition password.
   • YOUR-ACCESS-TOKEN is the name of your access token environment variable.

   For example:

   $ tkgi login -a tkgi.my.lab -u alana -p 'psswrdabc123...!' -k; \
   export my_token=$(bosh int ~/.pks/creds.yml --path /access_token)
   

   Note: If your operator has configured Tanzu Kubernetes Grid Integrated Edition to use a SAML identity provider, you must include an additional SSO flag to use the above command. For information about the SSO flags, see the section for the above command in TKGI CLI. For information about configuring SAML, see Connecting Tanzu Kubernetes Grid Integrated Edition to a SAML Identity Provider

Manage Quotas

This section describes how to add, modify and delete user quotas.

Add a Quota

To enforce a quota on a specific user, run the following command:

curl -k -X POST \
-H "Authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json" \
-d \
'{
    "owner": "USER-ID",
    "limit": {
      "cpu": MAX-CPU,
      "memory": MAX-MEM,
      "cluster": MAX-CLUSTER
      }
    }' \
https://TKGI-API:9021/v1/quotas

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• USER-ID is the user account ID to enforce the quota restriction on.
• MAX-CPU is the maximum total amount of CPU resources that the user can allocate to containers and pods. If MAX-CPU is set to 0, the user cannot create clusters.
• MAX-MEM is the maximum total amount of memory, in gigabytes, that the user can allocate to containers and pods. If MAX-MEM is set to 0, the user cannot create clusters.
• MAX-CLUSTER is the maximum number of clusters that the user can provision. This value must greater than or equal to 1.
• TKGI-API is the FQDN of your TKGI API server.

For example:

$ user=exampleuser
$ tkgi login -a tkgi.my.lab -u $user -p 'psswrdabc123...!' -k; export TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
$ curl -k -X POST \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d \
'{
    "owner": "cody",
    "limit": {
        "cpu": 4,
        "memory": 5,
        "cluster": 10
    }
  }' \
https://example.com:9021/v1/quotas

Modify an Existing Quota

To modify a specific user’s existing quota, run the following command:

curl -k -X PATCH  \ 
-H "Authorization: Bearer $YOUR-ACCESS-TOKEN" \
-H "Content-Type: application/json" \
-d \
'{ 
    "owner": "USER-ID", 
    "limit": { 
      "cpu": MAX-CPU, 
      "memory": MAX-MEM,
      "cluster": MAX-CLUSTER    
      } 
    }' \
https://TKGI-API:9021/v1/quotas/USER-ID

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• USER-ID is the user account ID to enforce the quota restriction on.
• MAX-CPU is the maximum total amount of CPU resources that the user can allocate to containers and pods. If MAX-CPU is set to 0, the user cannot create clusters.
• MAX-MEM is the maximum total amount of memory, in gigabytes, that the user can allocate to containers and pods. If MAX-MEM is set to 0, the user cannot create clusters.
• MAX-CLUSTER is the maximum number of clusters that the user can provision. This value must greater than or equal to 1.
• TKGI-API is the FQDN of your TKGI API server. For example, api.tkgi.example.com.

For example:

$ user=exampleuser
$ tkgi login -a tkgi.my.lab -u $user -p 'psswrdabc123...!' -k; export TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
$ curl -k -X PATCH \
-H "Authorization: Bearer $TOKEN" \
-H "Content-Type: application/json" \
-d \
'{ 
    "owner": "cody", 
    "limit": {  
        "cpu": 2, 
        "memory": 3,
        "cluster": 6
    }
  }' \
https://example.com:9021/v1/quotas/$user

Delete a Quota

To delete a specific user’s existing quota, run the following command:

curl -k -X DELETE -H "Authorization: Bearer $YOUR-ACCESS-TOKEN" \
https://TKGI-API:9021/v1/quotas/USER-ID

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• TKGI-API is the FQDN of your TKGI API server.
• USER-ID is the user account ID to enforce the quota restriction on.

For example:

$ user=exampleuser
$ tkgi login -a tkgi.my.lab -u $user -p 'psswrdabc123...!' -k; export TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
$ curl -k -X DELETE -H "Authorization: Bearer $TOKEN" \
https://example.com:9021/v1/quotas/$user
{
  "body":"The quota owner named: 'exampleuser' not found."
}

View Quotas

The TKGI API quotas endpoint reports on resource usage quotas in the JSON format.

View Quotas for a Single User

To list the resource quota restrictions currently applied to a single user, run the following command:

curl -k -H "Authorization: Bearer $YOUR-ACCESS-TOKEN" \
https://TKGI-API:9021/v1/quotas/USER-ID

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• TKGI-API is the FQDN of your TKGI API server.
• USER-ID is the user account ID to report on.

For example:

$ user=exampleuser
$ tkgi login -a tkgi.my.lab -u $user -p 'psswrdabc123...!' -k; export TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
$ curl -k -H "Authorization: Bearer $TOKEN" \
https://example.com:9021/v1/quotas/$user 
{
  "owner":"cody",
  "limit":{
      "cpu":2,
      "memory":1.0,
      "cluster": 6
  }
}

View All Quotas

To list all current resource and cluster quota restrictions, run the following command:

curl -k -H "Authorization: Bearer $YOUR-ACCESS-TOKEN" \
https://TKGI-API:9021/v1/quotas

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• TKGI-API is the FQDN of your TKGI API server.

For example:

$ user=exampleuser
$ tkgi login -a tkgi.my.lab -u $user -p 'psswrdabc123...!' -k; export TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
$ curl -k -H "Authorization: Bearer $TOKEN" \
https://example.com:9021/v1/quotas
[
  {
    "owner":"cody",
    "limit":{
        "cpu":2,
        "memory":1.0,
        "cluster": 6
    }
  }
]

Error Message When User Exceeds Cluster Quota

If a user has exceeded their set cluster creation quota, then the following error message appears when the user attempts to create a cluster.

Error: You do not have enough privileges to perform this action. 
Please contact the TKGI administrator.

View Usage

The TKGI API usages endpoint returns resource usage per user in the JSON format.

View Resource Usage by User

To list the current resource usage of a single user, run the following command:

curl -k -H "Authorization: Bearer $YOUR-ACCESS-TOKEN" https://TKGI-API:9021/v1/usages/USER-ID

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• TKGI-API is the FQDN of your TKGI API server.
• USER-ID is the user account ID whose resource utilization you want to view.

View All Resource Usage

To list the current resource utilization for all users and clusters, run the following command:

curl -k -H "Authorization: Bearer $YOUR-ACCESS-TOKEN" \
https://TKGI-API:9021/v1/usages

Where:

• YOUR-ACCESS-TOKEN is your access token environment variable.
• TKGI-API is the FQDN of your TKGI API server.

For example:

$ user=exampleuser
$ tkgi login -a tkgi.my.lab -u $user -p 'psswrdabc123...!' -k; export TOKEN=$(bosh int ~/.pks/creds.yml --path /access_token)
$ curl -k -H "Authorization: Bearer $TOKEN" \
https://example.com:9021/v1/usages
[
  {
    "owner": "cody",
    "totals": {
      "cpu": 20,
      "memory": 52,
      "cluster": 2
    },
    "clusters": [
      {
        "name": "vsp1",
        "cpu": 12,
        "memory": 36
      }
    ]
  }
]