Getting Started with Amazon ECS

Note
Amazon ECS capabilities in the Tanzu CloudHealth are going to be deprecated soon.

VMware Tanzu CloudHealth digests CloudWatch ECS events to report cluster cost allocation and utilization.

ECS generates CloudWatch Events for ECS state changes. Tanzu CloudHealth digests these events in order to report cluster cost allocation and utilization.

The Tanzu CloudHealth ECS module uses CloudWatch to gather ECS Events and store them in an S3 bucket that you create. An IAM Policy, specifying a designated Tanzu CloudHealth Role, is associated with the S3 bucket, permitting Tanzu CloudHealth to retrieve Events from the bucket.

Note

Tanzu CloudHealth currently does not support Fargate cluster metrics.
If you are using Kubernetes as your orchestration solution, see Getting Started with Kubernetes in Tanzu CloudHealth.

Prerequisites

You are using an Amazon ECS orchestration solution for managing containers.
You have privileges to create an IAM role and S3 bucket in the AWS Console.
You have privileges for creating a CloudFormation Stack in your AWS Account.

Step 1: Enable ECS Module in AWS Accounts

For which accounts should you enable the ECS Module? All AWS accounts that are configured in the Tanzu CloudHealth Platform and own ECS infrastructure that you want to analyze using Tanzu CloudHealth.

Log in to the Tanzu CloudHealth Platform. From the menu, select Setup > AWS Accounts and edit each account for which you want to enable the ECS Module.
Expand the Optional section of the account setup form and switch on the ECS option. Enter the ECS Bucket Name and ECS Bucket Prefix (if you configured a custom prefix). You can enter the default values that Tanzu CloudHealth generates for each account. If you want to use another name for the bucket, copy that name into a text file so that you can retrieve it later for the subsequent steps.
Click Save Account.

Step 2 - Create CloudFormation Stack in AWS Account

Note Repeat these steps for each AWS Account whose ECS infrastructure you want to analyze using Tanzu CloudHealth.

Download the CloudFormation template from this location and save it locally as ecs-event-pipeline-generic-template.json.

The template performs the following operations in your AWS Account.

Creates an AWS CloudWatch Rule
Creates an AWS Kinesis Firehose
Creates an AWS S3 Bucket
Creates an AWS S3 Bucket Policy
- Tanzu CloudHealth Role to Get and List Objects
Creates AWS IAM Roles for:
- CloudWatch to Assume Role as the CloudWatch Events Principal
- Kinesis Firehose to Assume Role as the Kinesis Service Principal
Creates AWS IAM Policies for:
- Putting ECS Events into the Kinesis Firehose
- Putting ECS Events into the S3 Bucket

What data does Tanzu CloudHealth gather? Tanzu CloudHealth gathers two categories of data: (a) What node-level hardware resources are available in terms of Memory, CPU, and Disk. (b) What workloads are running in the cluster, and their resource allocation, measured in terms of Memory, CPU, and Disk.

Option 1: Create CloudFormation Stack Using AWS CLI

Run this command for each AWS Account.

Enter the AWS Account ID, S3 Bucket name, and S3 Bucket prefix in the placeholders in the command.

export ACCOUNT_ID=[place ACCOUNT ID here]
export BUCKET_NAME=[place BUCKET NAME here]
export BUCKET_PREFIX=[place BUCKET PREFIX here]
aws cloudformation create-stack
  --stack-name cht-ecs-event-stream-bucket-stack-${ACCOUNT_ID} \
  --parameters ParameterKey=S3Bucket,ParameterValue=${BUCKET_NAME} ParameterKey=S3BucketPrefix,ParameterValue=${BUCKET_PREFIX} \
  --template-body file://./ecs-event-pipeline-generic-template.json \
  --capabilities CAPABILITY_NAMED_IAM

Option 2: Create CloudFormation Stack Using AWS Console

Create a new CloudFormation Stack by selecting the downloaded template from your local machine.
Specify details such as the Stack Name.
Specify any additional options such as tags for the Stack.
Review the configuration, and check the box for I acknowledge that AWS CloudFormation might create IAM resources with custom names.

Step 3: Update Read-Only IAM Policy

Provide Tanzu CloudHealth read-only access to your Amazon ECS infrastructure and the S3 Buckets that store ECS Events.

Switch to the Tanzu CloudHealth Platform. On the Setup > AWS Accounts, edit the AWS account for which you just enabled the ECS Module. Scroll to the bottom of the Account Edit page and click Generate Policy. If the policy is not updated, Tanzu CloudHealth will be unable to access your ECS infrastructure and the ECS Events stored in the S3 bucket.
In the IAM Access Policy dialog box, click Select All and copy the contents to the clipboard. The updated policy contains two sets of read-only privileges. This set provides Tanzu CloudHealth read-only access to the ECS infrastructure in the AWS Account. "ecs:List*", "ecs:Describe*"

This set provides Tanzu CloudHealth read-only access to the S3 bucket that stores ECS Events.
```
{
 "Effect": "Allow",
 "Action": [
   "s3:Get*",
   "s3:List*"
 ],
 "Resource": [
   "arn:aws:s3:::cht-ecs-event-stream-bucket-<AWS-Account-Number>",
   "arn:aws:s3:::cht-ecs-event-stream-bucket-<AWS-Account-Number>/*"
 ]
},
```
Switch to the AWS Console and navigate to Services > IAM. From the left menu, select Policies and locate the IAM Access Policy you are using for the Tanzu CloudHealth platform.
In the Permissions tab, click JSON and paste the policy you copied from the Tanzu CloudHealth platform into the editor. Then click Save.

How long does it take for ECS clusters to be discovered and listed in the Tanzu CloudHealth Platform? Tanzu CloudHealth polls for changes to the IAM Policy every 15 min. The two lines you enter in the policy allow Tanzu CloudHealth to get a list of all ECS clusters in your AWS Accounts. Depending on when the polling last occurred with respect to the timing of changes to the policy, you might need to wait at least 15 min for your ECS clusters to appear in the Tanzu CloudHealth Platform.

Step 4: View ECS Clusters in Tanzu CloudHealth

In the Tanzu CloudHealth Platform, from the left menu, select Setup > Containers > Clusters. Once your ECS clusters are discovered, they appear on the page.

What if no clusters appear on this page?

If clusters are not listed on this page, Tanzu CloudHealth is still in the process of reading your ECS infrastructure. Return to this page later to see if the list is populated.

The CloudFormation Stack that you previously deployed in your AWS Account allows Tanzu CloudHealth to immediately start collecting metadata from the orchestrator. The Status of the cluster switches to Healthy once Tanzu CloudHealth starts receiving data.

Historical information is not backfilled.

Meaningful visualizations appear in the Tanzu CloudHealth platform approximately 24 hours after the Stack has been deployed and has started pushing data.

Cluster Status Meaning

On the Setup > Containers > Clusters page, clusters can have one of three statuses:

Unknown: The Collector has never successfully contacted Tanzu CloudHealth.
Healthy: The Collector is successfully deployed on the container cluster and has successfully pushed data to the Tanzu CloudHealth platform at least once during the last hour.
Unhealthy: The Collector has contacted Tanzu CloudHealth in the past, but not during the last 1 hour.

Tanzu CloudHealth currently does not support Fargate cluster metrics.

Next Step: Configure Cluster Reports

Automate Amazon ECS Setup in Tanzu CloudHealth

Step 1 - Create and Configure Kinesis Firehose Delivery Stream

Configure Kinesis Firehose to stream ECS-related CloudWatch events to an S3 bucket.

Repeat these steps in each region in which you have ECS usage, and ensure that you use the same S3 bucket and IAM role for all the regions you configure.

In the AWS Console, select Services > Kinesis Firehose. From the left menu, select Data Firehose and click Create delivery stream.
On the Name and source page, enter a name for the delivery stream and specify Source as Direct PUT to other sources.
Click Next.
In the Process records page, retain the default selections.
- Record transformation: Disabled
- Record format conversion: Disabled Then click Next.
On the Choose destination page, set Destination to Amazon S3, specify or create an S3 bucket, and specify an S3 prefix as ecs-event-stream/<account_id>/.
On the Configure Settings page, set the following values:
- Buffer size: 16 MB
- Buffer interval: 60 seconds
- S3 Compression: GZIP
- S3 Encryption: Disabled
- Error Logging: Enabled
- IAM Role: Create or specify an IAM Role for Kinesis Firehose delivery. Then click Next.
Review the delivery specifications and click Create delivery stream.

Step 2: Create an AWS CloudWatch rule

Configure an ECS CloudWatch rule and connect Kinesis Firehose to the CloudWatch rule.

Repeat these steps in each region in which you have ECS usage, and ensure that you use the same IAM role for all the regions you configure.

In the AWS Console, select Services > CloudWatch. From the left menu, select Rules and click Create rule. Set the following values:
- Select Event Pattern
- Specify Elastic Container Service (ECS)
- Set Event Type to State Change
- Select Specific detail type(s): ECS Container Instance State Change and ECS Task State Change
Click Add target from the section on the right and specify these values:
- From the dropdown, select Firehose delivery stream
- Select the delivery stream that you created in Step 1.
- Retain the remaining default values in order to create an IAM role for Kinesis Firehose to have access to ECS CloudWatch events. Then click Configure details.
On the Configure rule details page, specify these settings:
- Name: cht-ecs-event-stream-ecs-to-firehose-rule-<insert-region-name>
- Description: CloudWatch Events Rule configured to deliver all ECS events to Kinesis Firehose
- Status: Enabled

Step 3: Update AWS Account Settings in Tanzu CloudHealth

Generate a policy that provides the IAM Role associated with the Tanzu CloudHealth Platform access to the S3 bucket that stores the Kinesis Firehose streams.

In the Tanzu CloudHealth Platform, from the left menu, select Setup > AWS Accounts and edit each account for which you want to enable the container ECS module.
Expand the Optional section of the account configuration page and enable ECS.
Enter the ECS Bucket Name and ECS Bucket Prefix (if you configured a custom prefix).

Scroll to the bottom of the page and click Generate Policy. Copy the contents of the IAM Access Policy dialog box to the clipboard. The policy should include the following definition:

"ecs:List*",
"ecs:Describe*",
{
   "Effect": "Allow",
   "Action": [
       "s3:Get*",
        "s3:List*"
    ],
    "Resource": [
        "arn:aws:s3:::&lt;yourbucketname&gt;",
        "arn:aws:s3:::&lt;yourbucketname&gt;/*"
    ]
}

Close the IAM Access Policy dialog box and click Save Account.

Step 4: Update AWS Account Settings in Tanzu CloudHealth

Update the policy for the IAM Role associated with the Tanzu CloudHealth Platform.

Log in to the AWS Console for the targeted account as a user who has permission to edit an IAM role.
Navigate to Services > IAM. From the left menu, select Policies. Search for and locate the policy for the IAM Role associated with the Tanzu CloudHealth Platform. Click Edit Policy.
Paste the policy definition that you copied in the Step 3. Save the policy.

Step 5: Update S3 Bucket Policy

Add the appropriate S3 Bucket Policy to allow Tanzu CloudHealth to retrieve data from the bucket.

In the AWS Console, select Services > S3. Search for the S3 bucket you created in Step 1.
Select the Permissions tab and click Bucket Policy.

Copy and paste the following policy document into the Bucket policy editor. Then click Save.

{
"Version": "2012-10-17",
"Id": "cht-ecs-bucket-policy",
"Statement": [
    {
        "Sid": "",
        "Effect": "Allow",
        "Principal": {
            "AWS": "arn:aws:iam::146708650527:role/CloudHealth-IAM-Role"
        },
        "Action": [
            "s3:Get*",
            "s3:List*"
        ],
        "Resource": "arn:aws:s3:::<insert bucket name>"
        }
    ]
}

In the Tanzu CloudHealth Platform, from the left menu, select Setup > Containers > Clusters. The clusters for all the account and region combinations you configured appear on this page and should indicate a healthy status.