The S3 storage plugin application lets you use an Amazon Simple Storage Service (Amazon S3) location to store and retrieve backups when you run gpbackup and gprestore. Amazon S3 provides secure, durable, highly-scalable object storage.
To use the S3 storage plugin application, you specify the location of the plugin and the S3 login and backup location in a configuration file. When you run
gprestore, you specify the configuration file with the option
--plugin-config. For information about the configuration file, see S3 Storage Plugin Configuration File Format.
If you perform a backup operation with the
--plugin-config, you must also specify the
--plugin-config option when you restore the backup with
The configuration file specifies the absolute path to the Greenplum Database S3 storage plugin executable, connection credentials, and S3 location.
The S3 storage plugin configuration file uses the YAML 1.1 document format and implements its own schema for specifying the location of the Greenplum Database S3 storage plugin, connection credentials, and S3 location and login information.
The configuration file must be a valid YAML document. The
gprestore utilities process the control file document in order and use indentation (spaces) to determine the document hierarchy and the relationships of the sections to one another. The use of white space is significant. White space should not be used simply for formatting purposes, and tabs should not be used at all.
This is the structure of a S3 storage plugin configuration file.
[executablepath](#s3-exe-path): <<absolute-path-to-gpbackup_s3_plugin>> [options](#s3-options): [region](#s3-region): <<aws-region>> [endpoint](#s3-endpoint): <<S3-endpoint>> [aws\_access\_key\_id](#s3-id): <<aws-user-id>> [aws\_secret\_access\_key](#s3-key): <<aws-user-id-key>> [bucket](#s3-bucket): <<s3-bucket>> [folder](#s3-location): <<s3-location>> [encryption](#s3-encryption): [on|off]
$GPHOME/bin/gpbackup_s3_plugin. The plugin must be in the same location on every Greenplum Database host.
Required. Begins the S3 storage plugin options section.
: If this option is specified, the plugin ignores the
region option and does not use AWS to resolve the endpoint. When this option is not specified, the plugin uses the
region to determine AWS S3 endpoint.
: If this parameter is not specified, S3 authentication information from the session environment is used. See Notes.
aws_access_key_id. The S3 passcode for the S3 ID to access the S3 bucket location.
on, use connections that are secured with SSL. Set this option to
offto connect to an S3 compatible service that is not configured to use SSL.
: Any value other than
off is treated as
This is an example S3 storage plugin configuration file that is used in the next
gpbackup example command. The name of the file is
executablepath: $GPHOME/bin/gpbackup_s3_plugin options: region: us-west-2 aws_access_key_id: test-s3-user aws_secret_access_key: asdf1234asdf bucket: gpdb-backup folder: test/backup3
gpbackup example backs up the database demo using the S3 storage plugin. The absolute path to the S3 storage plugin configuration file is
gpbackup --dbname demo --plugin-config /home/gpadmin/s3-test-config.yaml
The S3 storage plugin writes the backup files to this S3 location in the AWS region us-west-2.
The S3 storage plugin application must be in the same location on every Greenplum Database host. The configuration file is required only on the master host.
When you perform a backup with the S3 storage plugin, the plugin stores the backup files in this location in the S3 bucket.
Where folder is the location you specified in the S3 configuration file, and datestamp and timestamp are the backup date and time stamps.
Using Amazon S3 to back up and restore data requires an Amazon AWS account with access to the Amazon S3 bucket. These are the Amazon S3 bucket permissions required for backing up and restoring data.
aws_secret_access_key are not specified in the configuration file, the S3 plugin uses S3 authentication information from the system environment of the session running the backup operation. The S3 plugin searches for the information in these sources, using the first available source.
For information about Amazon S3, see Amazon S3.
Parent topic: Using gpbackup Storage Plugins