Object Metadata 2.0

Starting with VMware Cloud Director 10.4, metadata is available in the /cloudapi REST api. It is the same concept as in the legacy /api space but has an improved and simplified structure and rules.

The following object types have metadata(expanding list):

• Defined entities

Object metadata gives cloud operators and tenants a flexible way to associate user-defined properties (key=value pairs) with objects. New in this version:

• File metadata
• Simplified access control model
• Namespace as a secondary optional unique key
• Configurable lifecycle on cloned objects
• Simplified rules for the FIQL expression, and enhanced capabilities and performance when searching by metadata
• A new and equivalent representation of the fields projection for metadata of the /api space
• Optimistic concurrency control

Basic structure

There currently are four entry kinds with different value types: String, Number, Boolean and File. Here is an example of the core structure. For more details, please refer to the openapi specification. Example:

• Synchronous - String, Number, Boolean

  POST /1.0.0/entities/{id}/metadata
  

  {
    "persistent": false,
    "readOnly": true,
    "keyValue": {
    "domain": "TENANT",
    "key": "test.str.3",
    "value": {
       "value": "string",
       "type": "StringEntry"
       }
    }
  }
  

  Response:
  200 Created

  {
    "id": <entry-urn>
    <rest of the entry fields as specified in the request>
  }
  

• Asynchronous - File

  POST /1.0.0/entities/{id}/metadata
  

  {
    "persistent": false,
    "readOnly": true,
    "keyValue": {
    "domain": "TENANT",
    "key": "test.str.3",
    "value": {
      "type": "FileEntry",
      "value": {
        "name": "cloud_director.png",
        "size": 41071,
        "mimeType": "image/png"
        }
      }
    }
  }
  

  Response:
  202 Accepted
  file upload link header
  task link header

Entries are unique on the fields: domain, namespace, key, within the context of the main object that they are attached to. Once created, the entry will have a unique URN id. The id is then used to access individual entries for view/update/delete operations. You can update only the value and the persistent fields of the entry. If you want to change any other fields, you must re-create it.

GET /1.0.0/entities/{id}/metadata/{entry-urn}

PUT /1.0.0/entities/{id}/metadata/{entry-urn}

DELETE /1.0.0/entities/{id}/metadata/{entry-urn}

There can be up to 50 entries per entity across namespaces and domains.

File metadata

A new kind of metadata has been introduced which is intended for storage of small bits of data – up to 32 MB per entry. The GET address of such a resource is stable (you do not have to first request а download transfer) and the binary response payload will include a Content-Type header with the value of the mimeType of the entry as created/updated. In this way you can store for example an image, whose location can be embedded in a html <img> tag as a source for a UI extension.

The lifecycle of the File entries is slightly different from the other kinds, because there is an asynchronous binary content upload step. Therefore, only for this kind the creation operation returns a task, and the uniqueness validation is performed upon task(upload) completion. The binary content upload itself is done through a link header returned to the creation request to track the task and upload the content. You must match the Content-Length header of this request, to the size field in the FileDescriptor of the metadata entry.

In relation to this kind of metadata a new quota type has been added – Total metadata file size, which enables providers to configure the amount of storage a tenant can consume through this feature.

Access control model

The access control model is still based on domain, however it has a different behavior compared to the /api space metadata. File entries have an additional layer of authorization through a new right (Metadata File Entry: Create/Modify) which enables only specific users to work with this kind of resource(create/update/delete).

Each user has an access level to the metadata, which is defined based on the formula: min(main object access level, entry access level). The entry access level is defined based on the domain and readOnly fields.
The domain field allows providers to share entries created by them to their tenants. The readOnly field specifies whether they should additionally be able to modify the entries.

Currently, there are only two valid values for the domain in the following order: TENANT < PROVIDER. Users cannot place an entry in a domain which is greater than the one they operate in or define a readOnly entry in their own domain.
For example, a provider can place an entry in the TENANT domain with readOnly=true to allow a tenant user to view it (but not modify it). However, a tenant user cannot create an entry in the PROVIDER domain, or a readOnly entry in the TENANT domain.

In summary:

• In order to be able to view the metadata of the object, you must at least have read access level to it and the entries must be in a domain up to- or below the one you are operating in
• In order to be able to create/update the metadata of the object, you must at least have write access level to it.
• In order to be able to update metadata, it must be in a domain up to- or below the one you are operating in and not readOnly. Additionally, for file entries you must have the right Metadata File Entry: Create/Modify

Namespace

Think of the namespace as an optional secondary key by which entries are unique. It provides a more obvious mechanism to group entries by vendor.

Configurable lifecycle on cloned objects

You can specify if the metadata of the object should be copied over on certain clone type of operations on the object. The list of operations depends on the type of the object. Entries with their persistent field set to true will be copied on such operations automatically.

Searching by metadata

In the query service of the /api space, searching by metadata has two aspects:

• a filter for entry existence
• a projection to include specific entries in the representation of the main object

These have equivalent representations in the new version – filter and summary.

Metadata Filter

Various api endpoints which allow searching for instances of an object type having metadata now have an additional query parameter – metadata. For example:

GET /1.0.0/entities/types/{vendor}/{nss}/{version}?metadata=

Users can specify a FIQL expression selecting by metadata as if the keys and values are regular properties of the object. The structure of a single entry selection is the following: [namespace|]key operator value. The namespace is optional and must be separated from the key by the pipe symbol. Therefore, the pipe symbol is a restricted character and cannot be used in the namespace or key of the entries. You can search by wildcard and key-only expressions. Examples:

• ?metadata=org:vmware|test:key:8=='string' – search by a string entry, ie when searching for a string always enclose in single quotes
• ?metadata=test.key.3==42 – search by a number entry
• ?metadata=test.key.3==* - search by key
• ?metadata=test.key.*==* - search by key starting with test.key.
• ?metadata=test.key.3=lt=43
• ?metadata=test.key.4=='str*' – search by a string entry with a value starting with str There are no limitations on the size of the FIQL expression or the number of different key-values in it.
  The resulting objects will only contain metadata accessible to the user. For example, a tenant user who searches for test.key, which is only present on an entry in the PROVIDER domain will not get any results back.

Example of filtering both at a core object property and metadata:

GET /1.0.0/entities/types/{vendor}/{nss}/{version}?metadata=test.key.3==42&filter=name==testEntity

This functionality should not be confused with the individual entries search on a particular object. The api GET …/{object-id}/metadata supports a regular filter query parameter, which enables you to search for particular entries on the object by key (note the syntax which is a consequence of the resource structure: ?filter=keyValue.key==test.key).

Metadata Summary

Instead of specifying a particular projection of which entries to include in the payload, users will now receive additional link headers in the response of the previous search operation. The response of such a link will contain a portion of a simplified representation of the metadata of the previously retrieved main objects. Consumers of this api must fetch all the links and merge the result in order to retrieve the total metadata summary. Example:

• Link: /1.0.0/metadataSummaries/rHLWh2aJRxazDCACHOaRqlqaXw7JdE0avCSxVYnG_IZBFhs1UTxP_6j7amOI8hHU>;rel=”related";title=1;type="application/json";model="MetadataSummaries"

• response payload is a map of object urn to array of KeyValue:

• {
  "urn:…ac72d687-6689-4716-b30c-20021ce691aa": [
    {
      domain:,
      namespace:,
      key:,
      value:
    }, …
  ],…
  }
  

The title property of the link contains an index for the links returned. Each link can (optionally) be ordered on the title.
The response content will include only metadata to which the user has access to at the time of request. If the configuration of the objects/entries related to the access control model changes between requests, you will potentially receive different contents.

Optimistic concurrency control

Users can optionally enable safe concurrent modifications of entries via the standard If-Match and ETag headers. When an entry is retrieved, an ETag header is returned, representing the current state id. When an If-Match header with that id is added to an update/delete request, the system will only perform the operation if the current state matches the requested reference state.