Create AWS Resources for Tanzu Application Platform
To install Tanzu Application Platform (commonly known as TAP) within the Amazon Web Services (AWS) Ecosystem, you must create several AWS resources. Use this topic to learn how to create:
- An Amazon Elastic Kubernetes Service (EKS) cluster to install Tanzu Application Platform.
- Identity and Access Management (IAM) roles to allow authentication and authorization to read and write from Amazon Elastic Container Registry (ECR).
- ECR Repositories for the Tanzu Application Platform container images.
Creating these resources enables Tanzu Application Platform to use an IAM role bound to a Kubernetes service account for authentication, rather than the typical username and password stored in a Kubernetes secret strategy. For more information, see this AWS documentation.
This is important when using ECR because authenticating to ECR is a two-step process:
- Retrieve a token using your AWS credentials.
- Use the token to authenticate to the registry.
To increase security, the token has a lifetime of 12 hours. This makes storing it as a secret for a service impractical because it has to be refereshed every 12 hours.
Using an IAM role on a service account mitigates the need to retrieve the token at all because it is handled by credential helpers within the services.
Prerequisites
Before installing Tanzu Application Platform on AWS, you need:
-
An AWS Account. You need to create all of your resources within Amazon Web Services, so you need an Amazon account. For more information, see How do I create and activate a new AWS account?. You need your account ID for this walkthrough.
-
AWS CLI. This walkthrough uses the AWS CLI to both query and configure resources in AWS, such as IAM roles. For more information, see this AWS documentation.
-
eksctlcommand line. Theeksctlcommand line helps you manage the life cycle of EKS clusters. This guide uses it to create clusters. To installeksctl, see the eksctl documentation.
Export environment variables
Variables are used throughout this guide. To simplify the process and minimize the opportunity for errors, export these variables:
export AWS_ACCOUNT_ID=012345678901
export AWS_REGION=us-west-2
export EKS_CLUSTER_NAME=tap-on-aws
Where:
| Variable | Description |
|---|---|
| AWS_ACCOUNT_ID | Your AWS account ID |
| AWS_REGION | The AWS region you are going to deploy to |
| EKS_CLUSTER_NAME | The name of your EKS Cluster |
Create an EKS cluster
To create an EKS cluster in the specified region, run:
eksctl create cluster --name $EKS_CLUSTER_NAME --managed --region $AWS_REGION --instance-types t3.xlarge --version 1.24 --with-oidc -N 5
Creating the control plane and node group can take anywhere from 30-60 minutes.
NoteThis step is optional if you already have an existing EKS Cluster v1.23 or later with OpenID Connect (OIDC) authentication enabled. For more information about how to enable the OIDC provider, see AWS documentation.
Install EBS CSI driver
As a requirement for Tanzu Application Platform, EBS CSI driver is no longer installed by default starting from EKS 1.23. For more information about how to install EBS CSI driver, see AWS documentation.
Create the container repositories
ECR requires that the container repositories are already created. For Tanzu Application Platform, you need to create two repositories:
- A repository to store the Tanzu Application Platform service container images.
- A repository to store Tanzu Build Service Base OS and Buildpack container images.
To create these repositories, run:
aws ecr create-repository --repository-name tap-images --region $AWS_REGION
aws ecr create-repository --repository-name tap-build-service --region $AWS_REGION
Name the repositories any name you want, but remember the names for when you later build the configuration.
Create the workload container repositories
Similar to the two repositories created earlier for the platform, you must create repositories for each workload that Tanzu Application Platform creates before creating any workloads so that a repository is available to upload container images and workload bundles. This is because AWS ECR does not support automatically creating container repositories on initial push. For more information, see the AWS repository in GitHub.
When installing Tanzu Application Platform, you must specify a prefix for all workload registries. This topic uses tanzu-application-platform as the default value, but you can customize this value in the profile configuration created in Install Tanzu Application Platform package and profiles on AWS.
To use the default value, create two workload repositories for each workload with the following format:
tanzu-application-platform/WORKLOADNAME-NAMESPACE
tanzu-application-platform/WORKLOADNAME-NAMESPACE-bundle
For example, to create these repositories for the sample workload tanzu-java-web-app in the default namespace, you can run the following ECR command:
aws ecr create-repository --repository-name tanzu-application-platform/tanzu-java-web-app-default --region $AWS_REGION
aws ecr create-repository --repository-name tanzu-application-platform/tanzu-java-web-app-default-bundle --region $AWS_REGION
NoteThe default Supply Chain Choreographer method of storing Kubernetes configuration is RegistryOps, which requires the
bundlerepository. If you enabled the GitOps capability, this repository is not required. For more information about the differences between RegistryOps and GitOps, see Use GitOps or RegistryOps with Supply Chain Choreographer.
Create IAM roles
By default, the EKS cluster is provisioned with an EC2 instance profile that provides read-only access for the entire EKS cluster to the ECR registry within your AWS account. For more information, see this AWS documentation.
However, some of the services within Tanzu Application Platform require write access to the container repositories. To provide that access, create IAM roles and add the ARN to the Kubernetes service accounts that those services use. This ensures that only the required services have access to write container images to ECR, rather than a blanket policy that applies to the entire cluster.
You must create two IAM Roles:
-
Tanzu Build Service: Gives write access to the repository to allow the service to automatically upload new images. This is limited in scope to the service account for kpack and the dependency updater.
-
Workload: Gives write access to the entire ECR registry with a prepended path. This prevents you from having to update the policy for each new workload created.
To create the roles, you must establish two policies:
-
Trust Policy: Limits the scope to the OIDC endpoint for the Kubernetes cluster and the Kubernetes service account you attach the role to.
-
Permission Policy: Limits the scope of actions the role can take on resources.
NoteThese policies attempt to achieve a least privilege model. Review them to confirm they adhere to your organization’s policies.
To simplify this walkthrough, use a script to create these policy documents and the roles. This script outputs the files and then creates the IAM roles by using the policy documents.
Run:
# Retrieve the OIDC endpoint from the Kubernetes cluster and store it for use in the policy.
export OIDCPROVIDER=$(aws eks describe-cluster --name $EKS_CLUSTER_NAME --region $AWS_REGION --output json | jq '.cluster.identity.oidc.issuer' | tr -d '"' | sed 's/https:\/\///')
cat << EOF > build-service-trust-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDCPROVIDER}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${OIDCPROVIDER}:aud": "sts.amazonaws.com"
},
"StringLike": {
"${OIDCPROVIDER}:sub": [
"system:serviceaccount:kpack:controller",
"system:serviceaccount:build-service:dependency-updater-controller-serviceaccount"
]
}
}
}
]
}
EOF
cat << EOF > build-service-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ecr:DescribeRegistry",
"ecr:GetAuthorizationToken",
"ecr:GetRegistryPolicy",
"ecr:PutRegistryPolicy",
"ecr:PutReplicationConfiguration",
"ecr:DeleteRegistryPolicy"
],
"Resource": "*",
"Effect": "Allow",
"Sid": "TAPEcrBuildServiceGlobal"
},
{
"Action": [
"ecr:DescribeImages",
"ecr:ListImages",
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:BatchGetRepositoryScanningConfiguration",
"ecr:DescribeImageReplicationStatus",
"ecr:DescribeImageScanFindings",
"ecr:DescribeRepositories",
"ecr:GetDownloadUrlForLayer",
"ecr:GetLifecyclePolicy",
"ecr:GetLifecyclePolicyPreview",
"ecr:GetRegistryScanningConfiguration",
"ecr:GetRepositoryPolicy",
"ecr:ListTagsForResource",
"ecr:TagResource",
"ecr:UntagResource",
"ecr:BatchDeleteImage",
"ecr:BatchImportUpstreamImage",
"ecr:CompleteLayerUpload",
"ecr:CreatePullThroughCacheRule",
"ecr:CreateRepository",
"ecr:DeleteLifecyclePolicy",
"ecr:DeletePullThroughCacheRule",
"ecr:DeleteRepository",
"ecr:InitiateLayerUpload",
"ecr:PutImage",
"ecr:PutImageScanningConfiguration",
"ecr:PutImageTagMutability",
"ecr:PutLifecyclePolicy",
"ecr:PutRegistryScanningConfiguration",
"ecr:ReplicateImage",
"ecr:StartImageScan",
"ecr:StartLifecyclePolicyPreview",
"ecr:UploadLayerPart",
"ecr:DeleteRepositoryPolicy",
"ecr:SetRepositoryPolicy"
],
"Resource": [
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tap-build-service",
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tap-images"
],
"Effect": "Allow",
"Sid": "TAPEcrBuildServiceScoped"
}
]
}
EOF
cat << EOF > workload-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"ecr:DescribeRegistry",
"ecr:GetAuthorizationToken",
"ecr:GetRegistryPolicy",
"ecr:PutRegistryPolicy",
"ecr:PutReplicationConfiguration",
"ecr:DeleteRegistryPolicy"
],
"Resource": "*",
"Effect": "Allow",
"Sid": "TAPEcrWorkloadGlobal"
},
{
"Action": [
"ecr:DescribeImages",
"ecr:ListImages",
"ecr:BatchCheckLayerAvailability",
"ecr:BatchGetImage",
"ecr:BatchGetRepositoryScanningConfiguration",
"ecr:DescribeImageReplicationStatus",
"ecr:DescribeImageScanFindings",
"ecr:DescribeRepositories",
"ecr:GetDownloadUrlForLayer",
"ecr:GetLifecyclePolicy",
"ecr:GetLifecyclePolicyPreview",
"ecr:GetRegistryScanningConfiguration",
"ecr:GetRepositoryPolicy",
"ecr:ListTagsForResource",
"ecr:TagResource",
"ecr:UntagResource",
"ecr:BatchDeleteImage",
"ecr:BatchImportUpstreamImage",
"ecr:CompleteLayerUpload",
"ecr:CreatePullThroughCacheRule",
"ecr:CreateRepository",
"ecr:DeleteLifecyclePolicy",
"ecr:DeletePullThroughCacheRule",
"ecr:DeleteRepository",
"ecr:InitiateLayerUpload",
"ecr:PutImage",
"ecr:PutImageScanningConfiguration",
"ecr:PutImageTagMutability",
"ecr:PutLifecyclePolicy",
"ecr:PutRegistryScanningConfiguration",
"ecr:ReplicateImage",
"ecr:StartImageScan",
"ecr:StartLifecyclePolicyPreview",
"ecr:UploadLayerPart",
"ecr:DeleteRepositoryPolicy",
"ecr:SetRepositoryPolicy"
],
"Resource": [
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tap-build-service",
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tanzu-application-platform/tanzu-java-web-app",
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tanzu-application-platform/tanzu-java-web-app-bundle",
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tanzu-application-platform",
"arn:aws:ecr:${AWS_REGION}:${AWS_ACCOUNT_ID}:repository/tanzu-application-platform/*"
],
"Effect": "Allow",
"Sid": "TAPEcrWorkloadScoped"
}
]
}
EOF
cat << EOF > workload-trust-policy.json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::${AWS_ACCOUNT_ID}:oidc-provider/${OIDCPROVIDER}"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"${OIDCPROVIDER}:sub": "system:serviceaccount:default:default",
"${OIDCPROVIDER}:aud": "sts.amazonaws.com"
}
}
}
]
}
EOF
# Create the Tanzu Build Service Role
aws iam create-role --role-name tap-build-service --assume-role-policy-document file://build-service-trust-policy.json
# Attach the Policy to the Build Role
aws iam put-role-policy --role-name tap-build-service --policy-name tapBuildServicePolicy --policy-document file://build-service-policy.json
# Create the Workload Role
aws iam create-role --role-name tap-workload --assume-role-policy-document file://workload-trust-policy.json
# Attach the Policy to the Workload Role
aws iam put-role-policy --role-name tap-workload --policy-name tapWorkload --policy-document file://workload-policy.json
