Here you will find troubleshooting tips and instructions for Spring Cloud Gateway for Kubernetes.
Techniques for troubleshooting
This section provides techniques to gather more information about the state of your Spring Cloud Gateway Operator.
Get scg-operator events
In case of errors, events are published for the Operator and the Gateway components (gateways, mappings and route configs). You can display them using the describe option:
$ kubectl describe springcloudgateway my-gateway
$ kubectl describe springcloudgatewaymapping my-gateway-mapping
$ kubectl describe springcloudgatewayrouteconfig my-gateway-route-config
For example, in case some expected routes are unavailable, using kubectl describe springcloudgatewaymapping may show the referenced gateway is not present. Creating the missing Gateway instance would fix the issue.
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning NotFound <unknown> SpringCloudGatewayController Specified SpringCloudGateway resource "demo-gateway" is not found / not ready
Another useful event to look for when troubleshooting is RouteUpdateException. This event is triggered when a Route is not valid. For example, when the filter name is wrong:
apiVersion: "tanzu.vmware.com/v1"
kind: SpringCloudGatewayRouteConfig
metadata:
name: test-gateway-routes
spec:
routes:
- uri: https://example.com
predicates:
- Path=/test/**
filters:
- InvalidFilter=1
title: "my-test-route"
If we get the events in the namespace, we can see how everything succeeded but we have now an extra Warning with the RouteUpdateException explaining that the route my-test-route is wrong:
$ kubectl get events --watch
LAST SEEN TYPE REASON OBJECT MESSAGE
11s Normal SuccessfulCreate statefulset/my-gateway create Pod my-gateway-0 in StatefulSet my-gateway successful
0s Normal RoutesUpToDate pod/my-gateway-0 Pod "my-gateway-0-1/my-gateway-0" is RoutesUpToDate with all routes
0s Normal Created springcloudgateway/my-gateway SpringCloudGateway resource my-gateway is Created
0s Warning RouteUpdateException /my-gateway-0 Failed to update route with title 'my-test-route' and uri 'https://example.com' due to: 'Pod update failed, request to http://10.244.1.4:8090/actuator/gateway/routes/my-gateway-0-1-mapping-0 failed. Response code 400, message Bad Request'
0s Normal Created springcloudgatewaymapping/mapping Routes specified in SpringCloudGatewayRouteConfig "initial-route-config" is Created on pod "my-gateway-0-1/my-gateway-0"
Check scg-operator logs
You can access the scg-operator logs by running.
$ kubectl logs deployment.apps/scg-operator --namespace spring-cloud-gateway
Common problems and their solutions
This section describes how to resolve common problems.
Unresponsive scg-operator
If you find that scg-operator won't fully start or respond, and none of the above techniques help point out the problem, you could try increasing the requested CPU resource by 100m by running:
$ kubectl edit deployment.apps/scg-operator
Alternatively, you can restart the deployment by running:
$ kubectl rollout restart deployment.apps/scg-operator
Spring Cloud Gateway Custom Resource Definitions present after uninstalling
If there are problems while uninstalling, sometimes the Custom Resource Definitions don't get deleted. After uninstalling, you can check if there are any of the Spring Cloud Gateway Custom Resource Definitions remaining by running:
$ kubectl get customresourcedefinitions
NAME CREATED AT
springcloudgatewaymappings.tanzu.vmware.com 2021-02-17T11:52:09Z
springcloudgatewayrouteconfigs.tanzu.vmware.com 2021-02-17T11:28:12Z
springcloudgateways.tanzu.vmware.com 2021-02-17T11:28:12Z
If any of these three appear, you can manually delete them by running:
$ kubectl delete crd springcloudgatewaymappings.tanzu.vmware.com
$ kubectl delete crd springcloudgatewayrouteconfigs.tanzu.vmware.com
$ kubectl delete crd springcloudgateways.tanzu.vmware.com
Failing to pull images
When running the installation script ./scripts/install-spring-cloud-gateway.sh and you see errors pulling an image:
Events from from installation namespace:
LAST SEEN TYPE REASON OBJECT MESSAGE
2m Normal Scheduled pod/scg-operator-7c6b749b9-hbrkx Successfully assigned spring-cloud-gateway/scg-operator-7c6b749b9-hbrkx to kind-control-plane
36s Normal Pulling pod/scg-operator-7c6b749b9-hbrkx Pulling image "my.registry/scg-operator:1.0.1"
2m Normal SuccessfulCreate replicaset/scg-operator-7c6b749b9 Created pod: scg-operator-7c6b749b9-hbrkx
2m Normal ScalingReplicaSet deployment/scg-operator Scaled up replica set scg-operator-7c6b749b9 to 1
36s Warning Failed pod/scg-operator-7c6b749b9-hbrkx Failed to pull image "my.registry/scg-operator:1.0.1": rpc error: code = Unknown desc = failed to pull and unpack image "my.registry/scg-operator:1.0.1": failed to resolve reference "my.registry/scg-operator:1.0.1": unexpected status code [manifests 1.0.1]: 401 Unauthorized
36s Warning Failed pod/scg-operator-7c6b749b9-hbrkx Error: ErrImagePull
12s Normal BackOff pod/scg-operator-7c6b749b9-hbrkx Back-off pulling image "my.registry/scg-operator:1.0.1"
12s Warning Failed pod/scg-operator-7c6b749b9-hbrkx Error: ImagePullBackOff
Error installing Spring Cloud Gateway operator
Check to make sure you created an image pull secret (with default name spring-cloud-gateway-image-pull-secret) to allow authentication with your registry. See the Installation page for a step-by-step guide.
