Tanzu Application Platform v1.9.1 release notes

This topic contains release notes for Tanzu Application Platform v1.9.1.

Release Date: 09 May 2024

v1.9.1 Breaking changes

This release includes the following changes, listed by component and area.

v1.9.1 Breaking changes: Tanzu Application Platform

Tanzu Application Platform releases have migrated from VMware Tanzu Network to the Broadcom Support Portal and Broadcom registry. Using VMware Tanzu Network to install or upgrade Tanzu Application Platform is no longer supported.

Before you upgrade, you must relocate the Tanzu Application Platform images from the Broadcom registry tanzu.packages.broadcom.com to your own registry. Make sure you relocate the images to your container image registry as part of the instructions in Upgrade Tanzu Application Platform.

v1.9.1 Security fixes

For the list of security fixes in this Tanzu Application Platform release, see Security fixes.

v1.9.1 Resolved issues

The following issues, listed by component and area, are resolved in this release.

v1.9.1 Resolved issues: Supply Chain

The Workload page in the Supply Chain UI now loads at a normal speed when there are more than 100 workloads to display.
The Workload Details page, accessed by clicking on a workload name, now loads in the Supply Chain UI when there are more than 100 workloads to display.

v1.9.1 Resolved issues: Tanzu Developer Portal

For the Runtime Resource View plug-in, resources all across the cluster are no longer needlessly queried when a Backstage namespace annotation is specified on the entity.

v1.9.1 Known issues

This release has the following known issues, listed by component and area.

v1.9.1 Known issues: Tanzu Application Platform

Installing or upgrading Tanzu Application Platform packages might fail initially, but reconcile automatically within a few seconds. There is no current workaround because this is part of the reconciliation loop that Tanzu Cluster Essentials tooling performs.
On Azure Kubernetes Service (AKS), the Datadog Cluster Agent cannot reconcile the webhook, which leads to an error. For troubleshooting information, see Datadog agent cannot reconcile webhook on AKS.
The Tanzu Application Platform integration with Tanzu Service Mesh does not work on vSphere with TKR v1.26. For more information about this integration, see Set up Tanzu Service Mesh. As a workaround, you can apply the label to update pod security on a TKr v1.26 Kubernetes namespace as advised by the release notes for TKr 1.26.5 for vSphere 8.x. However, applying this label provides more than the minimum necessary privilege to the resources in developer namespaces.

v1.9.1 Known issues: API Auto Registration

Registering conflicting groupId and version with API portal:

If you create two CuratedAPIDescriptors with the same groupId and version combination, both reconcile without throwing an error, and the /openapi?groupId&version endpoint returns both specifications. If you are adding both specifications to the API portal, only one of them might show up in the API portal UI with a warning indicating that there is a conflict. If you add the route provider annotation for both of the CuratedAPIDescriptors to use Spring Cloud Gateway, the generated API specspecification includes API routes from both CuratedAPIDescriptors.

You can see the groupId and version information from all CuratedAPIDescriptors by running:
```
$ kubectl get curatedapidescriptors -A

NAMESPACE           NAME         GROUPID            VERSION   STATUS   CURATED API SPEC URL
my-apps             petstore     test-api-group     1.2.3     Ready    http://AAR-CONTROLLER-FQDN/openapi/my-apps/petstore
default             mystery      test-api-group     1.2.3     Ready    http://AAR-CONTROLLER-FQDN/openapi/default/mystery
```
When creating an APIDescriptor with different apiSpec.url and server.url, the controller incorrectly uses the API spec URL as the server URL. To avoid this issue, use server.url only.

v1.9.1 Known issues: App Last Mile Catalog

The app-config-web, app-config-server, and app-config-worker components do not allow developers to override the default application ports. This means that applications that use non-standard ports do not work. To work around this, you can configure ports by providing values to the resulting Carvel package. This issue is planned to be fixed in a future release.

v1.9.1 Known issues: Application Configuration Service

Application Configuration Service is not compatible with the version of Flux CD in this release.

v1.9.1 Known issues: Application Live View

On the Run profile, Application Live View fails to reconcile if you use a non-default cluster issuer while installing through Tanzu Mission Control.

v1.9.1 Known issues: Artifact Metadata Repository Observer and CloudEvent Handler

Periodic reconciliation or restarting of the AMR Observer causes reattempted posting of ImageVulnerabilityScan results. There is an error on duplicate submission of identical ImageVulnerabilityScans you can ignore if the previous submission was successful.

v1.9.1 Known issues: Bitnami Services

If you try to configure private registry integration for the Bitnami Services after having already created a claim for one or more of the services using the default configuration, the updated private registry configuration does not appear to take effect. This is due to caching behavior in the system which is not accounted for during configuration updates. For a workaround, see Troubleshoot Bitnami Services.

v1.9.1 Known issues: Cartographer Conventions

Before Tanzu Application Platform v1.9, the cartographer.tanzu.vmware.com package contained two products: Cartographer and Cartographer Conventions. In Tanzu Application Platform v1.9.0 the Cartographer Conventions product is removed from the cartographer.tanzu.vmware.com package and is distributed in its own package named cartographer.conventions.apps.tanzu.vmware.com.

When you upgrade to Tanzu Application Platform v1.9, an issue might occur when installing the new package for Cartographer Conventions. The upgrade might fail to reconcile and show error messages similar to the following:
```
Resource 'clusterrole/cartographer-conventions-manager-role (rbac.authorization.k8s.io/v1) cluster' is already associated with a different app 'cartographer.app'
```
This message might appear more than once, and it can refer to several resources.

These errors appear when kapp-controller on the cluster tries to install the new Cartographer Conventions package before the Cartographer package has reconciled. The new package for Cartographer Conventions tries to install resources that the existing Cartographer package still owns.

Although it looks like the upgrade fails, if you wait a few minutes, kapp-controller finishes the installation and the packages will reconcile successfully. The system works normally after the reconciliation is complete.

This error does not occur on a new installation of Tanzu Application Platform v1.9.
While processing workloads with large SBOMs, the Cartographer Convention controller manager pod can fail with the status CrashLoopBackOff or OOMKilled. For information about how to increase the memory limit for both the convention server and webhook servers, including app-live-view-conventions, spring-boot-webhook, and developer-conventions/webhook, see Troubleshoot Cartographer Conventions.

v1.9.1 Known issues: Contour

A renewed CA certificate might not get updated in the CA section of Envoy and Contour certificate’s secrets

v1.9.1 Known issues: Crossplane

The Crossplane validatingwebhookconfiguration is not removed when you uninstall the Crossplane package. To workaround, delete the validatingwebhookconfiguration manually by running kubectl delete validatingwebhookconfiguration crossplane.

v1.9.1 Known issues: Services Toolkit

An error occurs if additionalProperties is true in a CompositeResourceDefinition. For more information and a workaround, see Troubleshoot Services Toolkit.

v1.9.1 Known issues: Supply Chain

Components cannot have more than one resumption defined. When there are multiple resumptions, WorkloadRuns are not correctly created after resumptions trigger changes. The current workaround is to assess all triggers in a single resumption.
Tanzu Supply Chain currently does not include support for Red Hat OpenShift. This means you cannot individually install components for Tanzu Supply Chain and Managed Resource Controller. You also cannot install the Authoring profile that includes those components as standard. Support for Red Hat OpenShift is planned for a later release.
Tanzu Supply Chain currently does not include support for CA certificates in the Out of the Box components. However, you can edit the components to support CA certificates and use them to construct a new Supply Chain. Support for CA certificates as standard is planned for future versions of Tanzu Supply Chain.
When you select the Supply Chains tab in Tanzu Developer Portal, you might encounter an error related to data.packaging.carvel.dev. The error message is related to permission issues and JSON parsing errors. The error message indicates that the user system:serviceaccount:tap-gui:tap-gui-viewer cannot list resource packages in the API group data.packaging.carvel.dev at the cluster scope. Additionally, an unexpected non-whitespace character is reported after JSON at position 4.

As a temporary workaround, apply an RBAC configuration that includes the get, watch, and list permissions for the resources in the data.packaging.carvel.dev API group. This workaround must not be mandated for supply chains that do not generate Carvel packages.

To eliminate the error message, configure RBAC to allow access to the Carvel package resource as follows:
```
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
- apiGroups: [data.packaging.carvel.dev]
 resources: [packages]
 verbs: ['get', 'watch', 'list']
```
In the Workload Details page in the Supply Chain UI, the config writer step takes longer than 20 seconds to load when more than 149 workloads are displayed in the Supply Chain UI.

v1.9.1 Known issues: Supply Chain Choreographer

The template for the external-deliverable-template does not respect the gitops_credentials_secret parameter. The value is not present on the deliverable if it is provided in the workload parameter gitops_credentials_secret or the supply chain tap-value ootb_supply_chain*.gitops.credentials_secret. As a workaround, operators must provide the value as a tap-value for the delivery: ootb_delivery_basic.source.credentials_secret. The supply chain’s GitOps credentials must authenticate to the same repository as the delivery’s source credentials. If a deliverable must use a secret different from that specified by the delivery tap-value, the deliverable must be manually altered when being copied to the Run cluster. Add the secret name as a source_credentials_secret parameter on the deliverable.
By default, Server Workload Carvel packages generated by the Carvel package supply chains no longer contain OpenAPIv3 descriptions of their parameters. These descriptions were omitted to keep the size of the Carvel Package definition under 4 KB, which is the size limit for the string output of a Tekton Task. For information about these parameters, see Carvel Package Supply Chains.
When using the Carvel Package Supply Chains, if the operator updates the parameter carvel_package.name_suffix, existing workloads incorrectly output a Carvel package to the GitOps repository that uses the old value of carvel_package.name_suffix. You can ignore or delete this package.
If the size of the resulting OpenAPIv3 specification exceeds a certain size, approximately 3 KB, the Supply Chain does not function. If you use the default Carvel package parameters, this issue does not occur. If you use custom Carvel package parameters, you might encounter this size limit. If you exceed the size limit, you can either deactivate this feature, or use a workaround. The workaround requires enabling a Tekton feature flag. For more information, see the Tekton documentation.

v1.9.1 Known issues: Supply Chain Security Tools - Policy

Supply Chain Security Tools - Policy defaults to The Update Framework (TUF) enabled due to incorrect logic. This might cause the package to not reconcile correctly if the default TUF mirrors are not reachable. To work around this, explicitly configure policy controller in the tap-values.yaml file to enable TUF:
```
policy:
  tuf_enabled: true
```

v1.9.1 Known issues: Supply Chain Security Tools - Scan

When using Supply Chain Security Tools (SCST) - Scan 2.0 with a ClusterImageTemplate, the value for the scanning image is overwritten with an incorrect default value from ootb_supply_chain_testing_scanning.image_scanner_cli in the tap-values.yaml file for templates other than Trivy. You can prevent this by setting the value in your tap-values.yaml file to the correct image. For example, for the Grype image packaged with Tanzu Application Platform:
```
ootb_supply_chain_testing_scanning:
  image_scanner_template_name: image-vulnerability-scan-grype
  image_scanning_cli:
    image: registry.example.com/tanzu-application-platform/tap-packages@sha256:feb1cdbd5c918aae7a89bdb2aa39d486bf6ffc81000764b522842e5934578497
```
The Snyk scanner outputs an incorrectly created date, resulting in an invalid date. If the workload is in a failed state due to an invalid date, wait approximately 10 hours and the workload automatically goes into the ready state. For more about this issue information, see the Snyk GitHub repository.
Recurring scan has a maximum of approximately 5000 container images that can be scanned at a single time due to size limits configMaps.
If the supply chain container image scanning is configured to use a different scanner or scanner version than the recurring scanning, the vulnerabilities displayed in Tanzu Developer Portal might be inaccurate.
SCST - Scan 1.0 fails with the error secrets 'store-ca-cert' not found during deployment by using Tanzu Mission Control with a non-default issuer. For how to work around this issue, see Deployment failure with non-default issuer.

v1.9.1 Known issues: Supply Chain Security Tools - Store

SCST - Store returns an expired certificate error message when a CA certificate expires before the app certificate. For more information, see CA Cert expires.
When outputting CycloneDX v1.5 SBOMs, the report is found to be an invalid SBOM by CycloneDX validators. This issue is planned to be fixed in a future release.
SCST - Store automatically detects PostgreSQL database index corruptions. If SCST - Store finds a PostgresSQL database index has been corrupted, SCST - Store will automatically attempt to repair, which might cause reconciliation during package updates. When this happens, the included Postgres database might take some time to complete the repair and accept connections. For more information, see Fix Postgres Database Index Corruption.
If CA Certificate data is included in the shared Tanzu Application Platform values section, do not configure AMR Observer with CA Certificate data.
When observer.deploy_through_tmc is true, properties are auto-configured for Tanzu Mission Control (TMC). This causes the MultiClusterPropertyCollector resource to overwrite existing Tanzu Application Platform values for Observer.

When using Let’s Encrypt ACME issuers, the resultant Kubernetes secret resource does not contain a ca.crt property. Therefore, when the MultiClusterPropertyCollector resource creates the Observer package configuration values secret, the required ca_cert_data is empty.

To work around this issue, add the Certificate Authority (CA) Certificate to the shared.ca_cert_data key in the Tanzu Application Platform installation values.

v1.9.1 Known issues: Tanzu Build Service

During upgrades a large number of builds might be created due to buildpack and stack updates. Some of these builds might fail due to transient network issues, causing the workload to be in an unhealthy state. This resolves itself on subsequent builds after a code change and does not affect the running application.

If you do not want to wait for subsequent builds to run, you can manually trigger a build. For instructions, see Troubleshooting.

v1.9.1 Known issues: Tanzu Developer Portal

If you do not configure any authentication providers, and do not allow guest access, the following message appears when loading Tanzu Developer Portal in a browser:
```
No configured authentication providers. Please configure at least one.
```
To resolve this issue, see Troubleshooting.
Ad-blocking browser extensions and standalone ad-blocking software can interfere with telemetry collection within the VMware Customer Experience Improvement Program and restrict access to all or parts of Tanzu Developer Portal. For more information, see Troubleshooting.
ScmAuth is a Backstage concept that abstracts Source Code Management (SCM) authentication into a package. An oversight in a recent code-base migration led to the accidental exclusion of custom ScmAuth functions. This exclusion affected some client operations, such as using Application Accelerators to create Git repositories on behalf of users.
The back-end Kubernetes plug-in reports failure in multicluster environments. In a multicluster environment when one request to a Kubernetes cluster fails, backstage-kubernetes-backend reports a failure to the front end. This is a known issue with upstream Backstage and it applies to all released versions of Tanzu Developer Portal. For more information, see this Backstage code in GitHub. This behavior arises from the API at the Backstage level. There are currently no known workarounds. There are plans for upstream commits to Backstage to resolve this issue.

v1.9.1 Known issues: Tanzu Developer Tools for IntelliJ

The error com.vdurmont.semver4j.SemverException: Invalid version (no major version) is shown in the error logs when attempting to perform a workload action before installing the Tanzu CLI apps plug-in.
If you restart your computer while running Live Update without terminating the Tilt process beforehand, there is a lock that incorrectly shows that Live Update is still running and prevents it from starting again. For the fix, see Troubleshooting.
Workload actions and Live Update do not work when in a project with spaces in its name, such as my app, or in its path, such as C:\Users\My User\my-app. For more information, see Troubleshooting.
An EDT Thread Exception error is logged or reported as a notification with a message similar to "com.intellij.diagnostic.PluginException: 2007 ms to call on EDT TanzuApplyAction#update@ProjectViewPopup". For more information, see Troubleshooting.

v1.9.1 Known issues: Tanzu Developer Tools for Visual Studio

Clicking the red square Stop button in the Visual Studio top toolbar can cause a workload to fail. For more information, see Troubleshooting.

v1.9.1 Component versions

The following table lists the Tanzu Application Platform package versions included with this release. For open source component versions in this Tanzu Application Platform release, see Open source component versions.

Component Name	Version
API Auto Registration	0.5.0
API portal	1.5.0
Application Accelerator	1.9.1
Application Configuration Service	2.3.1
Application Live View APIServer	1.9.1
Application Live View back end	1.9.1
Application Live View connector	1.9.1
Application Live View conventions	1.9.1
Application Single Sign-On	5.1.5
Artifact Metadata Repository Observer	0.5.0
AWS Services	0.3.0
Bitnami Services	0.5.0
Carbon Black Scanner for SCST - Scan (beta)	1.4.0
Cartographer Conventions	0.9.0
cert-manager	2.7.4
Cloud Native Runtimes	2.5.3
Contour	2.2.0
Crossplane	0.5.0
Default Roles	1.1.0
Developer Conventions	0.16.1
External Secrets Operator	0.9.4+tanzu.3
Flux CD Source Controller	1.1.2+tanzu.3
Grype Scanner for SCST - Scan	1.9.1
Local Source Proxy	0.2.1
Managed Resource Controller (beta)	0.2.11
Namespace Provisioner	0.6.2
Out of the Box Delivery - Basic	0.16.6
Out of the Box Supply Chain - Basic	0.16.6
Out of the Box Supply Chain - Testing	0.16.6
Out of the Box Supply Chain - Testing and Scanning	0.16.6
Out of the Box Templates	0.16.6
Service Bindings	0.12.0
Service Registry	1.3.2
Services Toolkit	0.14.0
Snyk Scanner for SCST - Scan (beta)	1.3.0
Source Controller	0.9.0
Spring Boot conventions	1.9.1
Spring Cloud Gateway	2.1.10
Supply Chain Choreographer	0.9.0
Supply Chain Security Tools - Policy Controller	1.6.4
Supply Chain Security Tools - Scan	1.9.1
Supply Chain Security Tools - Scan 2.0	0.4.0
Supply Chain Security Tools - Store	1.9.0
Tanzu Application Platform Telemetry	0.7.0
Tanzu Build Service	1.13.0
Tanzu CLI	1.3.0
Tanzu Developer Portal	1.9.2
Tanzu Developer Portal Configurator	1.9.2
Tanzu Supply Chain (beta)	0.2.31
Tekton Pipelines	0.50.3+tanzu.4