This topic details the specifications for DataScript utilities functions.

The following utilities functions are available in DataScripts:

Function

Description

avi.utils.base64_decode( string )

Decode content

avi.utils.base64_encode( string )

Encode content

avi.utils.get_geo_from_ip(ip_string, [flag])

Used to extract geolocation database information

avi.utils.ip.netaddr (ipaddr, netmask)

Accepts an IP address and netmask and returns the network address

avi.utils.murmur_hash ( string), avi.utils.sha1_hash (string), avi.utils.md5_hash (string)

Hash content

avi.utils.rand_bytes( num_bytes )

Generate cryptographically secure random bytes

avi.utils.get_ip_reputation

Used to check IP reputation of a given IP address

avi.utils.ip_in_subnet

Used to evaluate IP address netmask.
Note:

DataScripts can hash arbitrary data using one of three functions supporting their corresponding named hashing methods: MurmurHash, SHA-1, and MD5. For more information, see DataScript: avi.utils.murmur_hash ( string), avi.utils.sha1_hash (string), avi.utils.md5_hash (string).

avi.utils.base64_decode

Function

avi.utils.base64_decode( string )

Description

DataScripts can encode and decode arbitrary data with base64. This can be used for modification of data sent over the wire, or for inspection and security use cases.

Events

  • HTTP_REQ

  • HTTP_RESP

Parameter

string is the string to be decoded.

Returns

The decoded string

Related

avi.utils.base64_encode()

Example

The following example grabs the HTTP basic auth username and adds it as a user ID tag within VMware NSX Advanced Load Balancer, which can be used for log search, persistence, or access control.

if string.beginswith(avi.http.get_header("Authorization"), "Basic ") then
  auth = string.sub(avi.http.get_header("Authorization"), 7)
  decoded = avi.utils.base64_decode(auth)
  username = string.sub(decoded, 0, string.find(decoded, ":") - 1)
  avi.http.set_userid(username)
end

avi.utils.base64_encode

Function

avi.utils.base64_encode( string )

Description

DataScripts can encode and decode arbitrary data with base64. This can be used for modification of data sent over the wire, or for inspection and security use cases.

Events

  • HTTP_REQ

  • HTTP_RESP

Parameter

string is the string to encode.

Returns

The encoded string

Related

avi.utils.base64_decode()

avi.utils.get_geo_from_ip

NSX Advanced Load Balancer includes a geolocation database for identifying the origin of clients. This fixed database is based on the MaxMind IP-country and IP-autonomous system number (ASN) data. The database is maintained on NSX Advanced Load Balancer Controllers and incorporates database updates when the Controllers are upgraded.

The DataScript function avi.utils.get_geo_from_ip is used to retrieve geolocation information, autonomous system (AS) number, country code, and location (latitude, longitude, region, and city) of the queried IP address.

Note:

  • The database used for this API is equivalent to the default one maintained on the NSX Advanced Load Balancer Controller, it cannot be overridden or augmented with IP groups as mentioned in Geolocation Database.

  • It is mandatory to attach System-GeoDB profile to the virtual service Datascripts that make use of the function get_geo_from_ip.

  • The VSDataScriptSet that contains the DataScript using this function must refer to Geolocation Database.

Function

  • avi.utils.get_geo_from_ip (ip_string, [flag])

  • avi.utils.get_geo_from_ip(ip_string,"COUNTRY")

  • avi.utils.get_geo_from_ip(ip_string,"ASN")

  • avi.utils.get_geo_from_ip(ip_string,"LOCATION")

  • avi.utils.get_geo_from_ip(ip_string,"ALL")

Description

For a given IP address, returns a geolocation value for a given IPv4/v6 address ip_string according to flag. This value is retrieved from the geolocation database.

Prerequisites

A minimum of 4GB RAM in the service engine

At least 100 MB of extra shared memory must be configured on the Service Engine group using it. This can be set through the extra_shared_config_memory parameter in the SE group, as illustrated below:

[admin:example-ctrl]: > configure serviceenginegroup Default-Group extra_shared_config_memory 100
[admin:example-ctrl]: serviceenginegroup > save

A restart of the SEs in this group is required for the memory changes to take effect.

If this memory is not set, any virtual services configured with a DataScript using this function will be put into a fault state. In the NSX Advanced Load Balancer UI, the error message will resemble the below:

Events

All HTTP events

Parameters

  • ip_string is a string containing the the query IPv4/v6 address for which geodatabase information is queried.

  • flag is a string that can be set to "COUNTRY", "ASN", "LOCATION", or "ALL" default as "COUNTRY" if none is provided.

Returns

  1. Returns nil if:

    • ip_string contains an IPv6, internal/ invalid address, or

    • ip_string is not in the geolocation database (unassigned address), or

    • flag is set incorrectly (something other than "COUNTRY", "ASN", "LOCATION", or "ALL"), or

  2. Returns a string containing an ISO 3166-1 alpha-2 country code (e.g., "AU" for Australia) if flag is set to "COUNTRY" or if the flag parameter is absent.

  3. Returns a 32-bit ASN code (e.g., 1449 for AS1449 PayPal, Inc) if flag is set to "ASN".

  4. Returns a table containing four string elements, latitude, longitude, region and city (e.g. ["44.9566", "34.1116", "Crimea", "Simferopol"]) if flag is set to "LOCATION"

  5. Returns a table containing all geo information available, along with any user mapping matches, for the target IP if flag is set to "ALL"

Note:

The database used for this function is equivalent to the default one maintained on the NSX Advanced Load Balancer Controller. It cannot be overridden or augmented with IP groups, as mentioned in the Geolocation Database article.

Example 1

The below DataScript adds a geo header that looks like:

  • "Geo: not found" if the IP address is not found in the geolocation database, or

  • "Geo: <ASN>" if it is found.

client_ip = avi.vs.client_ip()
asn = avi.utils.get_geo_from_ip(client_ip, "ASN")
geo_header = "not found"
if asn ~= nil then
geo_header = asn
end
avi.http.add_header("Geo", geo_header)

Example 2

 
    { [“ATTRIBUTE_COUNTRY_CODE“] = “US“,
      [“ATTRIBUTE_COUNTRY_NAME“] = “United States“,
      [“ATTRIBUTE_CONTINENT_NAME“] = “North America“,
      [“ATTRIBUTE_CONTINENT_CODE“] = “NA“,
      [“ATTRIBUTE_LATITUDE“] = “37.3583“,
      [“ATTRIBUTE_LONGITUDE“] = “-122.1081“,
      [“ATTRIBUTE_USER_DEFINED_MAPPING“] = { [1] = “Example Mapping 1“, [2] = “Example Mapping 2", etc. }
    }

The length for country code is limited to exact 2 characters, and region/city is capped to 36 characters respectively.

avi.utils.ip.netaddr

Function

avi.utils.ip.netaddr(ip_addr, netmask))

Description

Accepts an IPv4 or IPv6 IP address and netmask and returns the network address.

Events

HTTP_REQ

HTTP_RESP

Parameter

  • ip_addris a string containing the IP address.

  • netmask is an integer containing the network mask.

Returns

If

  • ip_addr or mask is nil

  • ip_addr contains "NIL", or

  • ip_addr contains an invalid IPv6 or IPv4 address,

then the function returns NIL

else the function returns the IPv6 or IPv4 address of the network.

Example

avi.utils.ip.netaddr ("43.225.52.40", 24) returns 43.225.52.0

avi.utils.ip.netaddr ("2001:0db8:85a3:0000:0000:8a2e:0370:7334", 53) returns 1:2001:db8:85a3::

avi.utils.murmur_hash (), avi.utils.sha1_hash (), avi.utils.md5_hash ()

Function

avi.utils.murmur_hash ( string ), avi.utils.sha1_hash ( string ), avi.utils.md5_hash ( string )

Description

DataScripts can hash arbitrary data using one of three functions supporting their correspondingly named hashing methods: MurmurHash, SHA-1, and MD5.

Events

HTTP_REQ

HTTP_RESP

Parameter

string is the string to be hashed.

Returns

The hashed string

Example

Consider this 5-line code snippet:

if avi.http.get_path() == "/echo_listen_port" then
 avi.vs.log("Murmur-hash = " .. avi.utils.murmur_hash("How is the weather?"))
 avi.vs.log("SHA1-hash = " .. avi.utils.sha1_hash("How is the weather?"))
 avi.vs.log("MD5-hash = " .. avi.utils.md5_hash("How is the weather?"))
end

The output yielded for curl -vvv http://10.160.33.200/echo_listen_port was as follows:

[string "DS1"]:2: Murmur-hash = 2050936584
[string "DS1"]:3: SHA1-hash = 0171e8d98bf187af83fe7a9dbbb34ebc6629dcbe
[string "DS1"]:4: MD5-hash = c4150a2ad26a0c6345041f11ba914661

avi.utils.rand_bytes

Function

avi.utils.rand_bytes(num_bytes)

Description

Enables users to generate cryptographically secure random bytes of data.

Events

All

Parameter

num_bytes is an integer between 0 and 1024.

Returns

A string containing num_bytes random bytes of data.

Note:

Lua defines a string as a sequence of bytes. Therefore, manipulations valid for bytes within the ASCII range are likewise valid for bytes outside the ASCII range.Printing these random bytes will mostly likely show unrecognizable characters.Alternative methods such as math.random cannot be relied upon to return cryptographically secure random values, such as might be needed when creating UUIDs.

Example

Log 8 random bytes in the VS application log:

random_bytes = avi.utils.rand_bytes(8)
avi.vs.log(random_bytes)

avi.utils.get_ip_reputation

You can use Webroot DB for IP reputation check in L7 DataScript using Lua function.

is_good, reputation_type = avi.utils.get_ip_reputation(ip_addr)

First return value for is_good is true/false. This indicates if the given IP is of good reputation.

The second return value is a bitmap of IP reputation type and is valid only if is_good is false. For instance, value 1 indicates Spam Source (bit 0 set), and value 17 indicates Spam Source and Scanner (bits 0 and 4 sets).

For more information, see the IP Reputation topic in the VMware NSX Advanced Load BalancerWAF Guide.

The function will use the IP reputation database configured for a VSDataScriptSet.

Note:

You can use API or CLI to configure VSDataScriptSet to use IPReputationDB.

The Lua function will accept both IPv4 and IPv6 addresses. However, it will always return true for IPv6 addresses because the IP reputation database currently contains only IPv4 address information.

The format of the ip_addr parameters is expected to be as returned by avi.vs.client_ip(), which is a presentation format, for instance, 1.2.3.4.

message VSDataScriptSet {
  ...
  optional string ip_reputation_db_uuid = 58 [
    (refers_to) = "IPReputationDB",
    ...
  ]
  ...
}

Configuration Workflow

When a script uses the new avi.utils.get_ip_reputation(ip_addr) function, IPReputationDB should be configured at the VSDataScriptSet level.