This repository contains these plugins to support running Velero on GCP:
-
An object store plugin for persisting and retrieving backups on Google Cloud Storage. Content of backup is log files, warning/error files, restore logs.
-
A volume snapshotter plugin for creating snapshots from volumes (during a backup) and volumes from snapshots (during a restore) on Google Compute Engine Disks.
- Since v1.4.0, the snapshotter plugin can handle the volumes provisioned by CSI driver
pd.csi.storage.gke.io
.
- Since v1.4.0, the snapshotter plugin can handle the volumes provisioned by CSI driver
You can run Kubernetes on Google Cloud Platform in either:
- Kubernetes on Google Compute Engine virtual machines
- Google Kubernetes Engine
For common use-cases, take a look at the Examples page.
Below is a listing of plugin versions and respective Velero versions that are compatible.
Plugin Version | Velero Version |
---|---|
v1.6.x | v1.10.x |
v1.5.x | v1.9.x |
v1.4.x | v1.8.x |
v1.3.x | v1.7.x |
v1.2.x | v1.6.x |
v1.1.x | v1.5.x |
v1.1.x | v1.4.x |
v1.0.x | v1.3.x |
v1.0.x | v1.2.0 |
If you would like to file a GitHub issue for the plugin, please open the issue on the core Velero repo
To set up Velero on GCP, you:
You can also use this plugin to create an additional Backup Storage Location.
If you do not have the gcloud
and gsutil
CLIs locally installed, follow the user guide to set them up.
Velero requires an object storage bucket in which to store backups, preferably unique to a single Kubernetes cluster (see the FAQ for more details). Create a GCS bucket, replacing the <YOUR_BUCKET> placeholder with the name of your bucket:
BUCKET=<YOUR_BUCKET>
gsutil mb gs:https://$BUCKET/
If you run Google Kubernetes Engine (GKE), make sure that your current IAM user is a cluster-admin. This role is required to create RBAC objects. See the GKE documentation for more information.
To integrate Velero with GCP, create a Velero-specific Service Account:
-
View your current config settings:
gcloud config list
Store the
project
value from the results in the environment variable$PROJECT_ID
.PROJECT_ID=$(gcloud config get-value project)
-
Create a service account:
GSA_NAME=velero gcloud iam service-accounts create $GSA_NAME \ --display-name "Velero service account"
If you'll be using Velero to backup multiple clusters with multiple GCS buckets, it may be desirable to create a unique username per cluster rather than the default
velero
.Then list all accounts and find the
velero
account you just created:gcloud iam service-accounts list
Set the
$SERVICE_ACCOUNT_EMAIL
variable to match itsemail
value.SERVICE_ACCOUNT_EMAIL=$(gcloud iam service-accounts list \ --filter="displayName:Velero service account" \ --format 'value(email)')
These permissions are required by Velero to manage snapshot resources in the GCP Project.
```bash
ROLE_PERMISSIONS=(
compute.disks.get
compute.disks.create
compute.disks.createSnapshot
compute.snapshots.get
compute.snapshots.create
compute.snapshots.useReadOnly
compute.snapshots.delete
compute.zones.get
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.list
)
gcloud iam roles create velero.server \
--project $PROJECT_ID \
--title "Velero Server" \
--permissions "$(IFS=","; echo "${ROLE_PERMISSIONS[*]}")"
gcloud projects add-iam-policy-binding $PROJECT_ID \
--member serviceAccount:$SERVICE_ACCOUNT_EMAIL \
--role projects/$PROJECT_ID/roles/velero.server
gsutil iam ch serviceAccount:$SERVICE_ACCOUNT_EMAIL:objectAdmin gs:https://${BUCKET}
```
Note:
To allow Velero's Kubernetes Service Account to create signed urls for the GCS bucket,
add iam.serviceAccounts.signBlob
permissions above.
This is required if you want to run velero backup logs
:
the Velero server has access to GCS but the CLI does not.
This can be done in 2 different options.
This involves creating a Google Service Account Key and using it as --secret-file
during installation.
-
Create a service account key, specifying an output file (
credentials-velero
) in your local directory:gcloud iam service-accounts keys create credentials-velero \ --iam-account $SERVICE_ACCOUNT_EMAIL
Note that Google Service Account keys are valid for decades (no clear expiry date) - so store it securely or rotate them as often as possible or both.
This requires a GKE cluster with workload identity enabled.
-
Create Velero Namespace This is required because Kuberenetes Service Account (step 2) resides in a namespace
NAMESPACE=velero kubectl create namespace $NAMESPACE
-
Create Kubernetes Service Account This is required when binding to the Google Service Account. Namespace is already created in step 1 above.
KSA_NAME=velero kubectl create serviceaccount $KSA_NAME --namespace $NAMESPACE
-
Add IAM Policy Binding for Velero's Kubernetes service account to a GCP service account
gcloud iam service-accounts add-iam-policy-binding \ --role roles/iam.workloadIdentityUser \ --member "serviceAccount:[$PROJECT_ID].svc.id.goog[$NAMESPACE/$KSA_NAME]" \ [$GSA_NAME]@[$PROJECT_ID].iam.gserviceaccount.com
In this case:
[$NAMESPACE/$KSA_NAME]
are Kubernetes Namespace and Service Account created in step 1 and 2.PROJECT_ID
is the Google Project ID - Step 1 andGSA_NAME
is the name of the Google Service Account - Step 2.
For more information on configuring workload identity on GKE, look at the official GCP documentation for more details.
Download Velero
Install Velero, including all prerequisites, into the cluster and start the deployment. This will create a namespace called velero
, and place a deployment named velero
in it.
If using a Google Service Account Key:
velero install \
--provider gcp \
--plugins velero/velero-plugin-for-gcp:v1.6.0 \
--bucket $BUCKET \
--secret-file ./credentials-velero
If using Workload Identity:
You must add a service account annotation to the Kubernetes service account so that it will know which GCP service account to use. You can do this during installation with --sa-annotations
. Use the flag --no-secret
so that Velero will know not to look for a key file. You must also add the GCP service account name in --backup-location-config
.
velero install \
--provider gcp \
--plugins velero/velero-plugin-for-gcp:v1.6.0 \
--bucket $BUCKET \
--no-secret \
--sa-annotations iam.gke.io/gcp-service-account=[$GSA_NAME]@[$PROJECT_ID].iam.gserviceaccount.com \
--backup-location-config serviceAccount=[$GSA_NAME]@[$PROJECT_ID].iam.gserviceaccount.com \
Additionally, you can specify --use-node-agent
to enable node agent support, and --wait
to wait for the deployment to be ready.
(Optional) Specify additional configurable parameters for the --backup-location-config
flag.
(Optional) Specify additional configurable parameters for the --snapshot-location-config
flag.
(Optional) Customize the Velero installation further to meet your needs.
For more complex installation needs, use either the Helm chart, or add --dry-run -o yaml
options for generating the YAML representation for the installation.
If you are using Velero v1.6.0 or later, you can create additional GCP Backup Storage Locations that use their own credentials. These can also be created alongside Backup Storage Locations that use other providers.
It is not possible to use different credentials for additional Backup Storage Locations if you are pod based authentication such as Workload Identity.
- Velero 1.6.0 or later
- GCP plugin must be installed, either at install time, or by running
velero plugin add velero/velero-plugin-for-gcp:plugin-version
, replace theplugin-version
with the corresponding value
To configure a new Backup Storage Location with its own credentials, it is necessary to follow the steps above to create the bucket to use and to generate the credentials file to interact with that bucket. Once you have created the credentials file, create a Kubernetes Secret in the Velero namespace that contains these credentials:
kubectl create secret generic -n velero bsl-credentials --from-file=gcp=</path/to/credentialsfile>
This will create a secret named bsl-credentials
with a single key (gcp
) which contains the contents of your credentials file.
The name and key of this secret will be given to Velero when creating the Backup Storage Location, so it knows which secret data to use.
Once the bucket and credentials have been configured, these can be used to create the new Backup Storage Location:
velero backup-location create <bsl-name> \
--provider gcp \
--bucket $BUCKET \
--credential=bsl-credentials=gcp
The Backup Storage Location is ready to use when it has the phase Available
.
You can check this with the following command:
velero backup-location get
To use this new Backup Storage Location when performing a backup, use the flag --storage-location <bsl-name>
when running velero backup create
.