Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

S3 Backup configuration for AGILITY

AGILITY provides local backup per default and it is suggested not to change this setting. However, it is possible to optionally enable remote S3 backup.

  1. Create a kubernetes secret named agility-db-backup with the remote S3 credentials.

    • Declare the S3 credentials as environment variables and the namespace where AGILITY is installed

    Code Block
    export NS=agility
    export S3_BACKUP_CREDENTIALS_FILE=/tmp/s3-backup.ini
    export S3_ACCESS_KEY=<Access key>
    export S3_ACCESS_KEY_SECRET=<Secret access key>
    • Store in a temporary file the S3 credentials with the following format

    Code Block
    cat <<EOF > ${S3_BACKUP_CREDENTIALS_FILE}
    [global]
    repo2-s3-key=${S3_ACCESS_KEY}
    repo2-s3-key-secret=${S3_ACCESS_KEY_SECRET}
    EOF
    • Create the kubernetes secret

    Code Block
    kubectl --namespace ${NS} create secret generic agility-db-backup --from-file=s3.conf=${S3_BACKUP_CREDENTIALS_FILE}
  2. Define backup override values file

    • Define the required information

    Code Block
    export S3_BUCKET_NAME=<bucket_name>
    export S3_ENDPOINT=<endpoint>
    export S3_REGION=<region>
    export S3_URI_STYLE=<host|path>

    As per pgBackRest documentation, S3_URI_STYLE should be either host or path. Usually path is the preferred option:

    S3 URI Style. The following URI styles are supported: host - Connect to bucket.endpoint host. path - Connect to endpoint host and prepend bucket to URIs.

    • Declare the file

    Code Block
    cd agility-charts
    cat <<EOF> agility-services-values-backup.yaml
    postgrescluster:
      pgbackrest:
        backup:
          bucket_s3:
            enabled: true
            bucket_name: ${S3_BUCKET_NAME}
            endpoint: ${S3_ENDPOINT}
            region: ${S3_REGION}
            path: /backups/pgbackrest/${NS}/agility-db/repo2
            uri_style: ${S3_URI_STYLE}
        otherSpecs:
          manual:
            repoName: repo2
            options:
            - --type=full
    EOF
  3. Run the Helm command to deploy agility-services:

    Code Block
    helm --namespace ${NS} upgrade --install --create-namespace agility-services ./agility-services --values agility-services-values-backup.yaml
  4. Once deployed, check backup configuration is healthy

    Code Block
    export DB_POD=$(kubectl --namespace ${NS} get pod -l "postgres-operator.crunchydata.com/role=master" --no-headers -o custom-columns=":metadata.name")
    kubectl --namespace ${NS} exec ${DB_POD} -c pgbackrest -- pgbackrest info --output=json | jq '.[].repo[] | select(.key == 2)'

    Output should be the following:

    Code Block
    {
      "cipher": "none",
      "key": 2,
      "status": {
        "code": 2,
        "message": "no valid backups"
      }
    }
  5. Enforce first full backup execution on the S3 bucket

    Code Block
    kubectl --namespace ${NS} annotate postgrescluster agility-db postgres-operator.crunchydata.com/pgbackrest-backup="$(date)"

    A new pod will be created, wait until it gets Completed status

    Code Block
    kubectl --namespace ${NS} get pod -l postgres-operator.crunchydata.com/pgbackrest-repo=repo2 -w

S3 Backup verification

Once S3 backup is configured, it is possible to verify the current status:

...

Code Block
{
  "cipher": "none",
  "key": 2,
  "status": {
    "code": 0,
    "message": "ok"
  }
}

Restore AGILITY

agility-services database can be restored in place.

...