RSS
 

Known limitations

This topic lists known limitations and issues with Services Toolkit.

Service Resource Replication Limitations

Service Resource Replication limitations are listed as follows.

Updates to Secrets are not replicated

Currently, after a Secret is replicated from a Service Cluster to a Workload Cluster, any further updates to the original Secret in the Service Cluster are not propagated to the replica Secret in the Workload Cluster. VMware aims to remove this limitation in a future release of Services Toolkit.

Service API Projection Limitations

Service API Projection limitations are listed as follows.

Unable to project Core Kubernetes APIs

API projection via does not work with core Kubernetes APIs such as Secrets. This means that use cases such as Direct Service References or Cloud Service Provider use cases, support such as Consuming AWS RDS on TAP, will not work when combined with usage of the kubectl-scp plugin as shown in Dedicated Service Clusters.

CRD and Aggregation layer conflict

VMware uses api-aggregation as the mechanism to project APIs. After an API is registered by using this aggregation layer (the APIService is available), even if you create a CRD pointing to the same path, the aggregation layer still proxies the requests. If you do it the other way around, first create the CRD and then “project” the API (or register the APIService). That way the APIService is not available.

Local CRD is created before Service Resource API is projected

For example, you create rabbitmqclusters.rabbitmq.com/v1beta1 on your workload cluster by creating a CustomResourceDefinition before projecting the rabbitmq.com/v1beta1 API. When you try to project the rabbitmq.com/v1beta1 API, the APIService v1beta1.rabbitmq.com is not ready.

rabbitmqclusters.rabbitmq.com CRD status:

status:
  acceptedNames:
    categories:
    - all
    kind: RabbitmqCluster
    listKind: RabbitmqClusterList
    plural: rabbitmqclusters
    shortNames:
    - rmq
    singular: rabbitmqcluster
  conditions:
  - lastTransitionTime: "2021-08-18T13:01:31Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: "2021-08-18T13:01:31Z"
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1beta1

rabbitmq.com-v1beta1-api-group-import ClusterAPIGroupImport status:

status:
  conditions:
  - lastTransitionTime: "2021-08-18T13:01:47Z"
    message: apiservices.apiregistration.k8s.io "v1beta1.rabbitmq.com" already exists
    reason: APIServiceNotReady
    status: "False"
    type: APIServiceReady
  - lastTransitionTime: "2021-08-18T13:01:47Z"
    message: apiservices.apiregistration.k8s.io "v1beta1.rabbitmq.com" already exists
    reason: APIServiceNotReady
    status: "False"
    type: Ready
  observedGeneration: 1

To use Service API Projection on your cluster when you don’t have any Custom Resources provisioned from this CRD, delete the local CRD and delete/recreate the ClusterAPIGroupImport.

When local CRD is created after Service Resource API is projected

When local CRD is created after Service Resource API is projected, the APIService is available but the rabbitmqclusters.rabbitmq.com CRD does not show any errors on the status. This can be confusing as when you provision or delete a Custom Resource because the requests are proxied and run on the linked Service cluster, not on your local cluster.

rabbitmqclusters.rabbitmq.com CRD status:

status:
  acceptedNames:
    categories:
    - all
    kind: RabbitmqCluster
    listKind: RabbitmqClusterList
    plural: rabbitmqclusters
    shortNames:
    - rmq
    singular: rabbitmqcluster
  conditions:
  - lastTransitionTime: "2021-08-18T09:40:35Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: "2021-08-18T09:40:35Z"
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1beta1

rabbitmq.com-v1beta1-api-group-import ClusterAPIGroupImport status:

status:
  conditions:
  - lastTransitionTime: "2021-08-18T13:10:48Z"
    status: "True"
    type: APIServiceReady
  - lastTransitionTime: "2021-08-18T13:10:48Z"
    status: "True"
    type: Ready
  observedGeneration: 1

No built-in support for cluster-scoped requests against projected APIs in the Workload Cluster

By default, Services Toolkit does not support projection of cluster-scoped requests in the Workload Cluster. It supports namespace-scoped requests only.

This poses a problem with certain controllers watching these APIs in the Workload Cluster, for example, Service Binding implementation in GitHub. They might require cluster-scoped read access verbs on projected APIs in the Workload Cluster.

There is a workaround for these types of scenarios:

VMware provides a ClusterRole by using the kubectl-scp plug-in’s federate command on the Service Cluster.

For example:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: "example"
rules:
- apiGroups:
    - rabbitmq.com
  resources:
    - rabbitmqcluster
  verbs: ["get", "list", "watch"]

The ClusterRole is then bound to the Proxy Service Account on the Service Cluster.

This workaround has significant implications:

  • It represents a potential attack vector in which a malicious user operating in Workload Cluster A might obtain the secret access token used by the Proxy and, in turn, use that token to perform read actions (e.g. get/watch/list) on resources in the Service Cluster that are owned by an entirely different Workload Cluster B. In other words, this workaround circumvents proper isolation of projected resources between different Workload Clusters.
  • It’s confusing to the App Operator who might see resources that belong to non-existing namespaces.
  • Projected resources belonging to a Workload Cluster A are potentially being leaked to users in Workload Cluster B. It’s similar to the security issue stated earlier in this list, but different in that the user doesn’t even have to have any sort of malicious intent.

Future versions of the Services Toolkit add first-class support for cluster-scoped requests against projected APIs and, therefore, remove the need for the laid out workaround and its problematic characteristics.

Service Resource Claims Limitations

Service Resource Claims limitations are listed as follows.

Can only claim service resources that adhere to the Kubernetes Binding specification

Currently, a ResourceClaim is only successful in claiming a service resource if that service resource adheres to the Provisioned Service duck type in GitHub or if directly referring to a compatible secret. Future iterations of Services Toolkit might loosen this requirement by using an extension of the ResourceClaim function or another API.

Can only claim service resources once

Currently, only a single ResourceClaim can successful claim a service resource. If a second ResourceClaim is created for the same service resource, it fails with ResourceAlreadyClaimed. Future iterations of Services Toolkit might allow shared service resources.

check-circle-line exclamation-circle-line close-line
Scroll to top icon