VMware Cloud Director Object Storage Extension 3.1 Release Notes

VMware Cloud Director Object Storage Extension 3.1 \| 27 JUN 2024 \| Build 24042320 Check for additions and updates to these release notes.

VMware Cloud Director Object Storage Extension 3.1 | 27 JUN 2024 | Build 24042320

Check for additions and updates to these release notes.

What's New

VMware Cloud Director Object Storage Extension version 3.1 includes the following new features:

MinIO Support

Service providers can prepare a MinIO cluster in an external Kubernetes cluster, then use the VMware Cloud Director Object Storage Extension deployment flow to provision the MinIO integration in VMware Cloud Director.
Client IP Forwarding

You can forward the VMware Cloud Director Object Storage Extension API client IPs to the underlying storage platform. You can use client IP forwarding by bucket owners to define client IP allow or deny policies on buckets. Client IP forwarding is not activated by default and it depends on the storage platform's capability to accept forwarded client IPs.
Kubernetes Backup and Restore Enhancement

VMware Cloud Director Object Storage extension refreshes the Kubernetes Backup and Restore UI, provides more details of individual backups and restores, and it simplifies the states and transition for Kubernetes backup.
OSIS Enhancement

VMware Cloud Director Object Storage Extension's OSIS (Object Storage Interoperability Service) now supports the S3-compliant storage vendors which assign tenant specific S3 endpoints for newly created storage tenants. OSIS also allows to onboard storage tenants in an asynchronous manner.
This release resolves CVE-2024-22276.

For more information on this vulnerability and its impact on VMware by Broadcom products, see VMSA-2024-0015.

Product Support Notices

Deprecation of localized languages
Beginning with the next major release, we will be reducing the number of supported localization languages. The three supported languages will be:
- Japanese
- Spanish
- French
The following languages will no longer be supported: Brazilian Portuguese, German, Italian, Korean, Simplified Chinese, Traditional Chinese.

Impact:
- Users who have been using the deprecated languages will no longer receive updates or support in these languages.
- All user interfaces, help documentation, and customer support will be available only in English or in the three supported languages mentioned above.

Localization

VMware Cloud Director Object Storage Extension 3.1 is available in the following languages:

English
French
German
Italian
Spanish
Brazilian Portuguese
Japanese
Korean
Traditional Chinese
Simplified Chinese

Upgrade

You can upgrade directly to VMware Cloud Director Object Storage Extension 3.1 from versions 2.X. See Upgrading VMware Cloud Director Object Storage Extension.

From VMware Cloud Director Object Storage Extension 2.2.3, the CentOS 7 operating system is no longer supported. Verify that your operating system is supported before installing or upgrading to VMware Cloud Director Object Storage Extension 3.1.

If you plan to install VMware Cloud Director Object Storage Extension on a new operating system, you can migrate your existing configuration by using the ose config export and ose config import commands.

Caveats and Limitations

When you access a tenant organization as a cloud provider, you can see only local resources

When you access a tenant organization, you can see only the local resources of this organizaiton. When you open the VMware Cloud Director Object Storage Extension Dashboard or Buckets page, you can see and select only the local organizations.
S3 API requests authenticated with application credentials do not support the following use cases:
- Accessing a shared bucket if another user grants you permissions for the bucket.
- Deleting multiple objects simultaneously with a single API request.
- Copying objects from buckets that you own.
If you are using ECS storage, you cannot remove object tags.

When you try to remove an object tag, the operation fails with an error.
VMware Cloud Director and the underlying storage systems have different limitations on user names. To use VMware Cloud Director Object Storage Extension, user names must comply with both the requirements of VMware Cloud Director and the underlying storage system. A best practice is to use short user names (under 50 bytes) and to use alphanumeric characters.
- If you are using Cloudian storage, the maximum length of user IDs is 255 bytes.
- If you are using Dell ECS, the maximum length of user IDs is 64 bytes.
Bucket synchronization supports up to 10 million objects per a single synchronization job

When the cloud provider enables bucket synchronization for a tenant in the provider portal, the synchronization can support up to 10 million objects for the tenant. VMware Cloud Director Object Storage Extension does not support the synchronization for more than 10 million objects per a single bucket synchronization job.
If you are using ECS storage, S3 API, or the Find a Bucket feature, you cannot visit a bucket that belongs to a different tenant organization in the ECS platform.

Product Documentation

All documentation is available on the VMware Cloud Director Object Storage Extension Documentation page.

Known Issues

New - After upgrading VMware Cloud Director Object Storage Extension from version 2.2.X to version 3.X, the service fails to start

After you complete the upgrade of VMware Cloud Director Object Storage Extension from version 2.2.X to version 3.X, the service fails start with the following error:

Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource
Workaround:
1. Run the following command:ose args set -k spring.flyway.ignore-migration-patterns -v *:missing.
2. Restart the VMware Cloud Director Object Storage Extension service by running the following command: ose service restart.
New - Creating a VMware Cloud Director Object Storage Extension add-on instance fails with an error

When you attempt to create a VMware Cloud Director Object Storage Extension add-on instance, the process fails with the following error:

Can't create role by name: Object Storage Operator

As a result, the role Object Storage Operator disappears from the UI.
Workaround:
1. Login to the VMware Cloud Director host.
2. Run the following commands:
```
su - postgres 
\c vcloud
```
3. Delete the following line: delete from org_member where member_name = 'object storage operator';.
New - Upgrading VMware Cloud Director Object Storage Extension to version 3.1 results in an error

After you upgrade to VMware Cloud Director Object Storage Extension 3.1, in approximately ten minutes, the following error appears:

The kubeconfig is invalid

The issue is observed, because the kubeconfig file of the protected cluster needs more than ten minutes to sync after the upgrade.

Workаround: Wait for over ten minutes after you upgrade to version 3.1 and the message automatically disappears.
Deleting an object from an existing bucket after upgrading to VMware Cloud Director Object Storage Extension version 3.0, fails with an error

If you upgrade to VMware Cloud Director Object Storage Extension version 3.0, then try to delete an object from an existing bucket, the process fails with the following error:

Failed to exchange user info between Cloud Director and storage platform.

The issue is observed if the tenant user who attempts the operation has a user name that contains special characters.
Workaround:
1. Navigate to the Postgres Database that VMware Cloud Director Object Storage Extension uses.
2. In the table bucket_info, in the storage_user_id for buckets column, add the encoded tenant user name.
  
  You can find the encoded user name in the table platform_user_mapping, when you select the platform_user_id corresponding to user_name.
VMware Cloud Director authentication fails after you switch to remote region

If you are in a multisite, multi-region environment and you switch to a remote region in a remote organization with a different name, the VMware Cloud Director authentication fails.

Workaround: None.
Backing up an entire cluster fails

When you try to back up a Kubernetes cluster, where a pod contains persistent volumes in the primary node, the process enters a partially failed status.

Workaround: Activate Scheduling Pods in the Kubernetes Control plane primary nodes by running the following commands:

kubectl taint nodes --all node-role.kubernetes.io/master-

kubectl taint nodes --all node-role.kubernetes.io/control-plane-
The Kubernetes cluster protection status remains as Restoring

After performing a restore task in the target Kubernetes cluster, the cluster protection status remains as Restoring. The problem might occur when VMware Cloud Director Object Storage Extension continues to monitor the status of the restoring task and the state of the task remains in an InProgress state.

Workaround: Click the Refresh button on the cluster protection card.
The S3 service of VMware Cloud Object Storage Extension is unavailable

When you start or view VMware Cloud Object Storage Extension, the VMware Cloud Object Storage Extension service is active, but the S3 service is unavailable, with the following error message in the log file:

S3_TOKEN_AUTH_ERROR

The issue is observed if the time gap between the S3 client and the VMware Cloud Object Storage Extension VM is over 20 seconds.

Workaround 1: Change the gap time between the S3 client and the VMware Cloud Object Storage Extension VM to less than 20 seconds, for example, NTP for the VMware Cloud Object Storage Extension VM.

Workaround 2: Run the command oss.s3.request-expire-time=3600 and restart the VMware Cloud Object Storage Extension service.
Region metrics on the provider portal's tenant onboarding page does not distinguish region specific metrics data

With multi-region deployment, when multiple regions are activated for a tenant organization, active region cards show the global consumption metrics, not region-specific data. The problem is observed, because region specific metrics is not supported yet.

Workaround: None.