This topic contains troubleshooting and known issues for Supply Chain Security Tools - Store.

CrashLoopBackOff from password authentication failed

Symptom

Supply Chain Security Tools - Store does not start up. You see the following error in the metadata-store-app Pod logs:

$ kubectl logs pod/metadata-store-app-* -n metadata-store -c metadata-store-app
...
[error] failed to initialize database, got error failed to connect to `host=metadata-store-db user=metadata-store-user database=metadata-store`: server error (FATAL: password authentication failed for user "metadata-store-user" (SQLSTATE 28P01))

Solution

If you see the error above, you have changed the database password between deployments, which is not supported. To change the password, see Persistent Volume Retains Data below.

Warning: Changing the database password deletes your Supply Chain Security Tools - Store data.

Persistent Volume Retains Data

Symptom

If Supply Chain Security Tools - Store is deployed, deleted, and then redeployed the metadata-store-db Pod fails to start up if the database password changed during redeployment. This is due to the persistent volume used by postgres retaining old data, even though the retention policy is set to DELETE.

Solution

To redeploy the app either use the same database password or follow the steps below to erase the data on the volume:

  1. Deploy metadata-store app through kapp.
  2. Verify that the metadata-store-db-* Pod fails.
  3. Run:

    kubectl exec -it metadata-store-db-<some-id> -n metadata-store /bin/bash
    

    Where <some-id> is the ID generated by Kubernetes and appended to the Pod name.

  4. Run rm -rf /var/lib/postgresql/data/* to delete all database data. This is the path found in postgres-db-deployment.yaml.
  5. Delete the metadata-store app through kapp.
  6. Deploy the metadata-store app through kapp.

Missing Persistent Volume

Symptom

After Store is deployed, metadata-store-db Pod could fail for missing volume while postgres-db-pv-claim pvc is in PENDING state. This issue could be occurring because the cluster where Store is deployed does not have storageclass defined. storageclass's provisioner is responsible for creating the persistent volume after metadata-store-db attaches postgres-db-pv-claim.

Solution

  1. Verify that your cluster has storageclass by running kubectl get storageclass.
  2. Create a storageclass in your cluster before deploying Store, for example:

    # This is the storageclass that Kind uses
    kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/master/deploy/local-path-storage.yaml
    # set the storage class as default
    kubectl patch storageclass local-path -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}}}'
    

Querying local path source reports

Symptom

If a source report has a local path as the name -- for example, /path/to/code -- the leading / on the resulting repository name causes the querying packages and vulnerabilities to return the following error from the client lib and the CLI: { "message": "Not found" }.

The URL of the resulting HTTP request is properly escaped: for example, /api/sources/%2Fpath%2Fto%2Fdir/vulnerabilities.

The rbac-proxy used for authentication handles this URL in a way that the response is a redirect: for example, HTTP 301\nLocation: /api/sources/path/to/dir/vulnerabilities. The Client Lib follows the redirect, making a request to the new URL which does not exist in the Supply Chain Security Tools - Store API, resulting in the error message above.

No support for installing in custom namespaces

All of our testing uses the metadata-store namespace. Using a different namespace breaks authentication and certificate validation for the metadata-store API.

Deploying for Internal/External customers

Using imgpkg copy command copies the image bundle along with all images it references to a target repository.

This is useful as it ensures the images used in the bundle are always available, even if the images in the original location are no longer available: there is now a copy in the target repository.

  • Deploy the bundle by running:

    imgpkg copy -b <bundle-image-registry>:<bundle-version-tag> --to-repo <target-image-registry-repo>/
    

    Where: + <bundle-image-registry> is the registry where you copy the image bundle. + <bundle-version-tag> is the image bundle version tag to copy. + <target-image-registry-repo> is where you paste the image bundle and all associated images.

check-circle-line exclamation-circle-line close-line
Scroll to top icon