This page lists known limitations and issues with Services Toolkit.

Service Resource Replication Limitations

Limitation 1: Updates to Secrets are not automatically replicated

Currently, after a Secret has been replicated from a Service Cluster to a Workload Cluster, any further updates to the original Secret in the Service Cluster are not propagated to the replica Secret in the Workload Cluster. We are aiming to remove this limitation in a future release of Services Toolkit.

Service API Projection Limitations

Limitation 1: CRD and Aggregation layer conflict

We use api-aggregation as the mechanism to project APIs. Once an API is registered via this aggregation layer (the APIService is available), even if you create a CRD pointing to the same path, the requests will still be proxied by the aggregation layer. If you do it the other way around, as in first create the CRD and then "project" the API (or register the APIService), the APIService won't be available.

Behaviour when local CRD is created before Service Resource API has been projected

For example, let's say you created rabbitmqclusters.rabbitmq.com/v1beta1 on your workload cluster by creating a CustomResourceDefinition before you project the rabbitmq.com/v1beta1 API. When you try to project it, the APIService v1beta1.rabbitmq.com won't be ready:

rabbitmqclusters.rabbitmq.com CRD status:

status:
  acceptedNames:
    categories:
    - all
    kind: RabbitmqCluster
    listKind: RabbitmqClusterList
    plural: rabbitmqclusters
    shortNames:
    - rmq
    singular: rabbitmqcluster
  conditions:
  - lastTransitionTime: "2021-08-18T13:01:31Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: "2021-08-18T13:01:31Z"
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1beta1

rabbitmq.com-v1beta1-api-group-import ClusterAPIGroupImport status:

status:
  conditions:
  - lastTransitionTime: "2021-08-18T13:01:47Z"
    message: apiservices.apiregistration.k8s.io "v1beta1.rabbitmq.com" already exists
    reason: APIServiceNotReady
    status: "False"
    type: APIServiceReady
  - lastTransitionTime: "2021-08-18T13:01:47Z"
    message: apiservices.apiregistration.k8s.io "v1beta1.rabbitmq.com" already exists
    reason: APIServiceNotReady
    status: "False"
    type: Ready
  observedGeneration: 1

The workaround in this case, if you want to use Service API Projection on your cluster (and you don't have any Custom Resources provisioned from this CRD) is to delete the local CRD and delete/recreate the ClusterAPIGroupImport.

Behaviour when local CRD is created after Service Resource API is projected

If you did things in the other order however, the APIService will be available but also the rabbitmqclusters.rabbitmq.com CRD won't show any errors on the status, which can be confusing as when you provision/delete a Custom Resource, the requests will be proxied and will run on the linked Service cluster, not on your local cluster.

rabbitmqclusters.rabbitmq.com CRD status:

status:
  acceptedNames:
    categories:
    - all
    kind: RabbitmqCluster
    listKind: RabbitmqClusterList
    plural: rabbitmqclusters
    shortNames:
    - rmq
    singular: rabbitmqcluster
  conditions:
  - lastTransitionTime: "2021-08-18T09:40:35Z"
    message: no conflicts found
    reason: NoConflicts
    status: "True"
    type: NamesAccepted
  - lastTransitionTime: "2021-08-18T09:40:35Z"
    message: the initial names have been accepted
    reason: InitialNamesAccepted
    status: "True"
    type: Established
  storedVersions:
  - v1beta1

rabbitmq.com-v1beta1-api-group-import ClusterAPIGroupImport status:

status:
  conditions:
  - lastTransitionTime: "2021-08-18T13:10:48Z"
    status: "True"
    type: APIServiceReady
  - lastTransitionTime: "2021-08-18T13:10:48Z"
    status: "True"
    type: Ready
  observedGeneration: 1

Limitation 2: No built-in support for cluster-scoped requests against projected APIs in the Workload Cluster

By default Services Toolkit 0.4.0 does not support projection of cluster-scoped requests in the Workload Cluster. It supports namespace-scoped requests only.

This poses a problem with certain controllers watching these APIs in the Workload Cluster, e.g. Service Binding implementation. They might require cluster-scoped read access verbs on projected APIs in the Workload Cluster.

There is a workaround for these types of scenarios:

We provide a ClusterRole through our prototypical kubectl-scp plugin's federate command on the Service Cluster. For example:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
 name: "example"
rules:
- apiGroups:
   - rabbitmq.com
 resources:
   - rabbitmqcluster
 verbs: ["get", "list", "watch"]

The ClusterRole is then bound to the Proxy Service Account on the Service Cluster.

This workaround has significant implications to be aware of:

  • It represents a potential attack vector in which a malicious user operating in Workload Cluster A might obtain the secret access token used by the Proxy and, in turn, use that token to perform read actions (e.g. get/watch/list) on resources in the Service Cluster that are owned by an entirely different Workload Cluster B. In other words, this workaround circumvents proper isolation of projected resources between different Workload Clusters.
  • It's confusing to the App Operator who might see resources that belong to non-existing namespaces.
  • Projected resources belonging to a Workload Cluster A are potentially being leaked to users in Workload Cluster B. It's similar to the security issue stated earlier in this list, but different in that the user doesn't even have to have any sort of malicious intent.

Future versions of the Services Toolkit will add first-class support for cluster-scoped requests against projected APIs and, thus, remove the need for the laid out workaround and its problematic characteristics.

Service Resource Claims Limitations

Limitation 1: Only can claim service resources that adhere to the Kubernetes Binding specification

Currently, a ResourceClaim will only be successful in claiming a service resource if that service resource adheres to the Provisioned Service duck type or if directly referring to a compatible Secret. Eventually future iterations of the Services Toolkit will loosen this requirement through an extension of the ResourceClaim functionality or another API.

Limitation 2: Only can claim service resources in the same namespace as your application

Currently, a ResourceClaim will only be successful in claiming a service resource if that service resource is present within the same namespace as the ResourceClaim. Eventually future iterations of the Services Toolkit will enable cross-namespace claiming of service resources.

check-circle-line exclamation-circle-line close-line
Scroll to top icon