VMware Greenplum Backup and Restore utilities are released separately from VMware Greenplum (as of Backup and Restore v1.13.0), and are updated independently of the core server.
VMware Greenplum Backup and Restore is compatible with these Greenplum Database versions:
Release Date: July 26, 2024
VMware Greenplum Backup and Restore version 1.30.6 is a maintenance release that resolves several issues.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
Resolved Issues
gpbackup
will timeout if sending an email takes too long.Changed Features
gpbackup
supports backing up and restoring resource groups with the MEMORY_QUOTA
, IO_LIMIT
, and MIN_COST
parameters on Greenplum version 7.gpbackup
now always backs up CREATE DATABASE
statements so they can be restored with the --create-db
option.Resolved Issues
gpbackup_helper
would send data for the wrong table while performing a filtered gprestore
from a --single-data-file
backup using the --on-error-continue
flag.Changed Features
gpbackup_helper
now uses events to respond more quickly when writing data during table data restore.Resolved Issues
gprestore
with --resize-cluster
and either --copy-queue-size
or --jobs
would fail with the error No such file or directory
.Resolved Issues
No changes.
No changes.
No changes.
Release Date: May 23, 2024
VMware Greenplum Backup and Restore version 1.30.5 is a maintenance release that resolves several issues.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
Resolved Issues
gpbackup
might exit before cleanup functions complete leading to files and processes being left behind. Now gpbackup
ensures the cleanup functions complete before exiting.Changed Features
INFO
by default. File log level can be increased along with terminal level by using --verbose
or --debug
options.Resolved Issues
gprestore
could have up to a minute delay in writing data between each table during data restore, causing performance issues when restoring large numbers of tables.Resolved Issues
gprestore
might exit before cleanup functions complete leading to files and processes being left behind. Now gprestore
ensures the cleanup functions complete before exiting.Changed Features
COPY
program command when using --single-data-file
option to reduce the chance of hanging COPY
commands.INFO
by default. File log level can be increased along with terminal level by using --verbose
or --debug
options.No changes.
No changes.
No changes.
Resolved Issues
gpbackup_s3_plugin
with an endpoint would report errors such as NoSuchBucket
on some systems, by adding remove_duplicate_bucket
config option to toggle API behavior.Release Date: April 5, 2024
VMware Greenplum Backup and Restore version 1.30.4 is a maintenance release that resolves several issues.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
Resolved Issues
gpbackup
processes would halt when run in the background.GRANT
statements were not properly backed up for aggregate functions.Changed Features
gpbackup
with Greenplum 7.0.0 or later, gpbackup
will no longer log the progress of COPY
commands for each table.Resolved Issues
gpbackup_helper
processes would sometimes be left behind from gpbackup
or gprestore
after it completed or was terminated. Now the cleanup procedure will ensure the processes are stopped.gpbackup_helper
was creating pipes for data transfer with overly broad permissions. Now permissions are changed so only gpadmin user will have access to the pipes.gpbackup_helper
was reusing pipes multiple times for a larger to smaller --resize-cluster
restore resulting in hanging COPY
processes. Now all pipes have unique names and COPY
should not hang.gpbackup_helper
in multi data file mode was not closing file handles and may eventually fail due to too many files open.Changed Features
gpbackup_helper
logging around cleanup failures if gpbackup_helper
exits early.Resolved Issues
ENUM
distribution key could not be restored by gprestore.--jobs
option and executing serially.gprestore
may hang when using--resize-cluster
and --on-error-continue
because gprestore
was not creating skip files properly.Changed Features
gprestore
with Greenplum 7.0.0 or later, gprestore
will no longer log the progress of COPY
commands for each table.Resolved Issues
No changes.
No changes.
Resolved Issues
gpbackup_s3_plugin
with an endpoint would report errors such as NoSuchBucket: The specified bucket does not exist
.Release Date: January 12, 2024
VMware Greenplum Backup and Restore version 1.30.3 is a maintenance release that resolves several issues.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
Resolved Issues
gpbackup
was backing up constraints prior to backing up data, which led to performance issues during restore. Constraints are now created in postdata.No changes.
Resolved Issues
gprestore
generated invalid syntax for GRANT
statements on views, materialized views, and foreign tables, when using the --redirect-schema
option.No changes.
No changes.
No changes.
No changes.
Release Date: December 1st, 2023
VMware Greenplum Backup and Restore version 1.30.2 is a maintenance release that changes a feature, and resolves an issue.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
New Features
Historically, by default gpbackup
wrote backup files to one directory per segment. This changed briefly in gpbackup
1.30.0 and 1.30.1, when the default became to write the files to just one directory for all segments. However, this change led to performance regressions when restoring.
In this release, gpbackup
’s default behavior is to once again write files to one directory per segment.
If you prefer that gpbackup
write to just one directory for all segments instead, you may now pass gpbackup
the new --single-backup-dir
option.
With this option, gpbackup
places all files for a given backup set on a given host in the same directory, rather than in distinct, per-segment subdirectories under the configured backup directory.
For example, files that were previously created in the <backup-dir>/gpseg0/backups/<datestamp>/<timestamp>
or <backup-dir>/gpseg1/backups/<datestamp>/<timestamp>
directories will now all be created under <backup-dir>/backups/<datestamp>/<timestamp>
.
You may restore backups taken with this option as normal, but there will be a performance penalty when doing so on some systems. However this protects you from having to manually reorganize files to the correct segment when you pass the --resize-cluster
restore option to gprestore
.
You must use the --backup-dir
option with this option.
No changes.
No changes.
Resolved Issue
Resolved issues when traversing the new backup directory structure by updating dependencies.
No changes.
No changes.
No changes.
Release Date: November 14, 2023
VMware Greenplum Backup and Restore version 1.30.1 is a maintenance release that resolves two issues.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
No changes.
No changes.
Resolved Issue
Resolves an issue where gprestore
found an incorrect number of backup files due to a faulty regular expression.
Resolved Issue
Resolves an issue where gpbackup_manager
failed to find backup files when backups taken in the most recent format (post-1.30.0) and older format (pre-1.30.0) were present in the same directory.
No changes.
No changes.
No changes.
Release Date: October 12, 2023
VMware Greenplum Backup and Restore version 1.30.0 is a minor release that introduces new and changed features and resolves a number of issues.
This release includes the following utilities:
See Data Domain Boost for Data Domain Boost support information.
See Known Issues for a list of issues known to exist in this release.
See Differences Compared to Open Source Greenplum Backup and Restore for the list of features that are only in VMware Greenplum Backup and Restore.
See Release Numbering Conventions for a description of the VMware Greenplum Backup and Restore release numbering scheme.
New Features
When using gpbackup
with Greenplum 7.0.0 or later, gpbackup
writes both to standard output and to the log file the progress of the COPY
command for each table.
The --include-table
and --exclude-table
options now support schemas and tables containing a period in their name. The object containing the period must be set off in double quotes and the double quotes must be escaped with a backslash, as in the following schema example: \"public.staging\".sales
.
The --include-table-file
and --exclude-table-file
options now support schemas and tables containing a period in their name. The object containing the period must be set off in double quotes, as in the following schema example: "public.staging".sales
.
gpbackup
now includes the database name in the subject line of the email when /home/gpadmin/gp_email_contacts.yaml
is configured.
This release of gpbackup
adds new fields to the backup file's metadata to facilitate parallel restores of metadata if the --jobs
or --copy-queue-size
options are invoked with gprestore
. In addition, gpbackup
now wraps metadata restore statements in transactions to improve performance. This substantially speeds up metadata restores.
NoteThis applies to pre-data metadata only.
NOTEIf
--on-error-continue
is passed togprestore
, no parallelism or transactions are used for metadata restore, and it will behave exactly as it has historically.
This release introduces support for Enterprise Linux 9 (Red Hat 9, Rocky 9, Oracle RHCK 9).
Changed Features
When invoking the --backup-dir
option, backup file directories are no longer categorized by segment. Instead, all backup files are now placed in a directory structure such as the following: backup-dir/backups/date/timestamp.
When invoking the --include-table
and --no-inherits
options, only the table specified in the --include-table
option is backed up, even if that table's DDL is dependent on other inherited tables.
Resolved Issue
Resolves performance issues with an underlying catalog query when backing up INDEX information during a full backup.
No changes.
New Features
The --resize-cluster
option is now Generally Available. To enable this feature, take production backups with version 1.30.0 or later of gpbackup
.
When the directory specified with the backup-dir
option contains only the backup files of a single timestamp, the backup set will be restored without the need to invoke the --timestamp
option in the gprestore
command.
When using gprestore
with Greenplum 7.0.0 or later, gprestore
writes both to standard output and to the log file the progress of the COPY
command for each table.
gprestore
now includes the database name in the subject line of the email when /home/gpadmin/gp_email_contacts.yaml
is configured.
The --include-table
and --exclude-table
options now support schemas and tables containing a period in their name. The object containing the period must be set off in double quotes and the double quotes must be escaped with a backslash, as in the following schema example: \"public.staging\".sales
.
The --include-table-file
and --exclude-table-file
options now support schemas and tables containing a period in their name. The object containing the period must be set off in double quotes, as in the following schema example: "public.staging".sales
.
Resolved Issue
Resolves an issue where, when the --redirect-schema
option was passed, GRANT
and REVOKE
commands were not executing properly.
New Feature
The gpbackup_manager utility
now includes a new command, find-table
. When invoked with a fully qualified <schema.tablename>, gpbackup_manager
displays the successful backup sets that contain the specified table.
No changes.
No changes.
Resolved Issue
Resolves an issue where AWS bucket name paths were being duplicated when using an endpoint URL configuration, resulting in unexpected backup placement.
This release of gpbackup
and gprestore
supports Data Domain Boost for backup on Red Hat Enterprise Linux. This table lists the supported versions of DDOS, Data Domain Boost SDK, and BoostFS.
DDOS | Data Domain Boost | BoostFS | Anticipated End of Support* |
---|---|---|---|
7.10 | 7.10 | 7.10 | TBA |
7.9 | 7.9 | 7.9 | TBA |
7.8 | 7.8 | 7.8 | TBA |
7.7 | 7.7 | 7.7 | 31 Oct 2024 |
7.6 | 7.6 | 7.6 | 31 May 2024 |
7.5 | 7.5 | 7.5 | 28 Feb 2024 |
7.4 | 7.4 | 7.4 | 31 Dec 2023 |
7.2 | 7.2 | 7.2 | 31 May 2023 |
7.1 | 7.1 | 7.1 | 31 Jan 2023 |
*The Data Domain Boost plugin support matrix conforms to the end-of-support dates published by Dell.
NoteIn addition to the DDOS versions listed in the previous table,
gpbackup
andgprestore
support all minor patch releases (fourth digit releases) later than the certified version.
There are known gprestore 1.30.4
issues when using --copy-queue-size
(and --single-data-file
), especially in combination with --resize-cluster
. There is currently no workaround for this. It is recommended not to use the --copy-queue-size
optimization when performing resize restore with this version of gprestore.
When using gprestore
with the --incremental
and --data-only
command line options on Greenplum Database version 5.28.1 or earlier, the restore may fail during the truncate phase if a table in the restore set contains a foreign key referencing a different table. This Greenplum server side fix is included in version 5.28.2 and later, and does not require a change in gpbackup
or gprestore
.
Greenplum Backup and Restore versions 1.21.0 and 1.22.0 are incompatible with Centos 6 (glibc 2.12) and SLES 11 (glibc 2.11). The workaround is to upgrade to version 1.23.0 or later.
When attempting to restore data from Greenplum 4.3.x/5.x/6.x (prior to Greenplum v6.14.1 and v5.28.6), views with anyarray
typecasts are not restorable. From VMware Greenplum Backup and Restore v1.20.3, gpbackup generates an error when such views are detected.
Workaround: On the original cluster, identify any views that exhibit this symptom, using a query similar to:
SELECT relname AS anyarray_views
FROM pg_class WHERE relkind = 'v' AND oid >= 16384 AND (pg_get_viewdef(oid)
LIKE '%::anyarray%') IS TRUE;
Then select option 1 or 2:
gprestore
with the --on-error-continue
flag. The affected views will not be re-created. After the restore completes, re-create each VIEW
using your original VIEW
definition.VIEW
on the original Greenplum cluster. Take another backup of the older cluster and run gprestore
on the new cluster. Finally, recreate the VIEW
on the new cluster using the original VIEW
definition.When attempting to restore data from Greenplum 4.3.x/5.x into Greenplum 6.x, if the restore involved a table distributed by character(n)
or char(n)
using the legacy bpchar
hash operator, the restore would fail. This issue has been resolved in Greenplum Database 6.12. Upgrade to the latest Greenplum release to avoid this issue.
gprestore
does not support restoring a backup that contains partitioned tables where the table is created with a non-reserved keyword that is used as a partition name. For example, the non-reserved keyword at
is used as a partition name in this SUBPARTITION TEMPLATE
clause fragment:
...
SUBPARTITION TEMPLATE
( SUBPARTITION "at" VALUES ('usa'),
...
gpbackup
backs up the partitioned table, but gprestore
returns an error when attempting to restore the table.
Before performing a backup with gpbackup
, you must ensure that partitioned tables do not use any of these non-reserved keywords as a partition name:
ADD, ALTER, ALWAYS, AT, ATTRIBUTE, CATALOG, COMMENTS, CONFIGURATION, CONFLICT, CONTINUE, CURRENT, DATA, DAY, DENY, DEPENDS, DICTIONARY, DISCARD, DOCUMENT, DXL, EVENT, EXTENSION, FAMILY, FILESPACE, FILTER, FULLSCAN, FUNCTIONS, HOUR, IDENTITY, IGNORE, IMPORT, INITPLAN, INLINE, LABEL, LEAKPROOF, LOCKED, LOGGED, MAPPING, MATERIALIZED, METHOD, MINUTE, MONTH, NOCREATEEXTTABLE, OFF, ORDERED, ORDINALITY, OVER, PARALLEL, PARSER, PASSING, PLANS, POLICY, PROGRAM, RANDOMLY, READABLE, READS, RECURSIVE, REF, REFRESH, REJECT, REPLICATED, ROOTPARTITION, SECOND, SEQUENCES, SERVER, SKIP, SNAPSHOT, SQL, STANDALONE, STRIP, TABLES, TEXT, TRANSFORM, TYPES, UNLOGGED, VALIDATION, VARYING, VIEWS, WEB, WHITESPACE, WITHIN, WITHOUT, WRAPPER, WRITABLE, XML, YEAR, YES
A gprestore
operation using the --redirect-schema
option fails if gprestore
attempts to restore an index in a schema and the name of the index is the name of the schema followed by a '.' (period). For example, this CREATE INDEX
command creates the index named test.
on a table in the schema test
. The index is in the schema test
.
CREATE INDEX "test." ON test.mytbl USING btree (i);
If the index and table are backed up with gpbackup
, restoring the backup with this gprestore
command fails because gprestore
fails to restore the test.
index.
gprestore --timestamp <timestamp> --redirect-schema foo2
Beginning with versions 4.3.33 and 5.19, Greenplum Database checks that the distribution key for a table is a prefix of any unique index key. This policy is not in place for Greenplum Database versions before 4.3.33 or 5.19, and it is possible to create a backup of a database from one of these earlier versions with unique indexes that do not comply with the policy. When you restore such a backup to a Greenplum Database version that does enforce the policy:
gprestore
, gpdbrestore
, and pg_restore
utilities.VMware Greenplum Backup and Restore includes all of the functionality in the open source Greenplum Backup github repository and S3 storage plugin repository and adds:
gpbackup_manager
utilitygppkg
formatThe VMware Greenplum Backup and Restore distribution release number indicates the type of the release.
gpbackup
/gprestore
major release number. For example, given the release number, 1.16.0
, the major release number is 1
.1.16.0
, the minor release number is 16
. This number increments when new features are added to the gpbackup
/gprestore
utilities.gpbackup_manager
and the backup storage plugins, have been updated.The release versions of the components included in the distribution, such as gpbackup_manager
and the backup storage plugins, are separate from the distribution version, but follow the same numbering scheme as the distribution.