GemFire Vector Database

This topic walks through giving your VMware Tanzu GemFire for Tanzu Application Service service instances access to the GemFire Vector Database (Vector DB) Extension.

GemFire Vector DB is an innovative extension that leverages GemFire’s powerful in-memory data capabilities to provide lightning-fast storage and retrieval of vector data, ideal for machine learning and AI applications. For more information, see the VMware GemFire Vector Database Product Documentation.

Prerequisites

The following are prerequisites to use GemFire Vector DB with your GemFire for Tanzu Application Service service instances.

GemFire Vector DB is supported only on VMs with 2 or more CPU cores. While creating plans, operators must choose the vm_types accordingly.
A GemFire for Tanzu Application Service service instance. For more information, see Create or Delete a Service Instance.
A service key for the GemFire for Tanzu Application Service service instance. For more information, see Create a Service Key.

Retrieve the Service Key

GemFire Vector DB is accessible using REST APIs and user credentials for the Authorization header. To find the REST URL and credentials, you must access the service key you created for your GemFire for Tanzu Application Service service instance.

For example, for a service key named “test-c1-key.key” created on a service instance named “test-c1”, you would run the following command:

$ cf service-key test-c1 test-c1-key.key

Extract username and password from a role of “users” group. If the foundation has UAA configured, users must use their assigned credentials, which will not appear in the service key output. For example, you can use the username and password for the cluster_operator role to create the Authorization header for GemFire Vector Database API:
```
{
  "password": "EZSy2WVHpCOtZLjZuNpVUA",
  "roles": [
    "cluster_operator"
  ],
  "username": "cluster_operator_2I5YlL2zEF8zkHYNYCaMw"
}
```
To create the Authorization header by using the Base64 encoded HTTP Basic Authentication mechanism, see the GemFire Vector DB documentation.

Copy the vectordb URL from the service key urls group. For example:

"urls": {
  "gfsh": "https://cloudcache-7a081d54-823b-45dd-990d-a88088be5c05.sys.coil-1464369.cf-app.com/gemfire/v1",
  "management": "https://cloudcache-7a081d54-823b-45dd-990d-a88088be5c05.sys.coil-1464369.cf-app.com/management/docs",
  "pulse": "https://cloudcache-7a081d54-823b-45dd-990d-a88088be5c05.sys.coil-1464369.cf-app.com/pulse",
  "vectordb": "https://gemfire-server-25d504b6-549c-4126-92ba-a4b53cb9fd5f.sys.mute-1513053.cf-app.com/gemfire-vectordb/v1"
},

cf service-key dev dev-key
Getting key dev-key for service instance dev as admin...

{
  "credentials": {
    "distributed_system_id": "0",
    "gfsh_login_string": "connect --url=https://cloudcache-32f36132-5b92-4b97-a458-75dc906c5931.sys.iris-1594600.za3fcc275.shep.cf-app.com/gemfire/v1 --user=cluster_operator_VH3vTL6Ti2YYPDM39ksc9w --password=6VlLxuiM7WSPfuUjFJERPg --skip-ssl-validation",
    "locators": [
      "0c129cd6-4598-4286-8404-385838251bdc.locator-server.iris-1594600-services-subnet.service-instance-32f36132-5b92-4b97-a458-75dc906c5931.bosh[55221]"
    ],
    "remote_cluster_info": {
      "recursors": {
        "iris-1594600-services-subnet.service-instance-32f36132-5b92-4b97-a458-75dc906c5931.bosh": [
          "10.0.8.5:1053"
        ]
      },
      "remote_locators": [
        "0c129cd6-4598-4286-8404-385838251bdc.locator-server.iris-1594600-services-subnet.service-instance-32f36132-5b92-4b97-a458-75dc906c5931.bosh[55221]"
      ],
      "trusted_sender_credentials": [
        {
          "password": "4M3x9fngMR9FGEYuL0eFvA",
          "username": "gateway_sender_Q8lpVoaBEsabf05rAlYYQA"
        }
      ]
    },
    "tls-enabled": "false",
    "urls": {
      "gfsh": "https://cloudcache-32f36132-5b92-4b97-a458-75dc906c5931.sys.iris-1594600.za3fcc275.shep.cf-app.com/gemfire/v1",
      "management": "https://cloudcache-32f36132-5b92-4b97-a458-75dc906c5931.sys.iris-1594600.za3fcc275.shep.cf-app.com/management/docs",
      "pulse": "https://cloudcache-32f36132-5b92-4b97-a458-75dc906c5931.sys.iris-1594600.za3fcc275.shep.cf-app.com/pulse",
      "vectordb": "https://gemfire-server-32f36132-5b92-4b97-a458-75dc906c5931.sys.iris-1594600.za3fcc275.shep.cf-app.com/gemfire-vectordb/v1"
    },
    "users": [
      {
        "password": "6VlLxuiM7WSPfuUjFJERPg",
        "roles": [
          "cluster_operator"
        ],
        "username": "cluster_operator_VH3vTL6Ti2YYPDM39ksc9w"
      },
      {
        "password": "VZdFe2c6AhuGgq367Uq1mw",
        "roles": [
          "developer"
        ],
        "username": "developer_Xjl574YVY5qsSrJ9sFTjHw"
      },
      {
        "password": "ouZ2o7LoR4lvDLsibGmZg",
        "roles": [
          "readonly"
        ],
        "username": "readonly_nJFdX1wSx6bfM3mMPWbuFg"
      }
    ],
    "wan": {}
  }
}

Using the REST endpoints to access GemFire Vector DB

This section provides some examples of how to use the REST endpoints to confirm access to the Vector Database Extension on the service instance.

Create Index:

Request:

curl -X POST 'gemfire-server-25d504b6-549c-4126-92ba-a4b53cb9fd5f.sys.mute-1513053.cf-app.com/gemfire-vectordb/v1/indexes' \
-H 'Authorization: Basic Y2x1c3Rlcl9vcGVyYXRvcl8xcjZjYmYxUzBXOW1TbTRTeE9CZ1E6YnF2VVdReXVwQUZyUFliNjJsNWdZUQ==' \
-H 'Content-Type: application/json' \
-d '{"name": "test_index_1"}' \
-w '\nHTTP Status Code: %{http_code}\n'

Response:

HTTP Status Code: 201

Retrieve Index:

Request:

curl -X GET 'gemfire-server-25d504b6-549c-4126-92ba-a4b53cb9fd5f.sys.mute-1513053.cf-app.com/gemfire-vectordb/v1/indexes/test_index_1' \
-H 'Authorization: Basic Y2x1c3Rlcl9vcGVyYXRvcl8xcjZjYmYxUzBXOW1TbTRTeE9CZ1E6YnF2VVdReXVwQUZyUFliNjJsNWdZUQ==' \
-H 'Content-Type: application/json' \
-w '\nHTTP Status Code: %{http_code}\n'

Response:

{
"name" : "test_index_1",
"fields" : [ ],
"beam-width" : 100,
"max-connections" : 16,
"vector-similarity-function" : "COSINE",
"buckets" : 4,
"number-of-embeddings" : 0
}
HTTP Status Code: 200