Google Cloud storage support

[githubzilla]

This pull request introduces support for Google Cloud Storage as an optional storage backend. Users can enable this feature by using the compile flag USE_GCP. Additionally, the USE_GCP flag can be used in conjunction with the USE_AWS flag to enable both the S3 and GCS storage backends simultaneously. To validate this feature, two new test suites, namely gcp_db_cloud_test and gcp_file_system_test, have been created.

Here are the steps to set up and run this feature:

# Setup Google Cloud storage client env
# according to https://cloud.google.com/cpp/docs/setup

# Init the Google Cloud client running environment and ADC
# provide the region/zone and project id
gcloud init
# login
gcloud auth application-default login

# Build the feature
USE_RTTI=1 USE_GCP=1 make all -j8

# Test the feature, since GCS does not support domain name style bucket name, you have to customize the bucket name
ROCKSDB_CLOUD_TEST_BUCKET_PREFIX=rockset- ROCKSDB_CLOUD_TEST_BUCKET_NAME=dbcloudtest GOOGLE_CLOUD_PROJECT=your_google_project_id ./gcp_file_system_test
ROCKSDB_CLOUD_TEST_BUCKET_PREFIX=rockset- ROCKSDB_CLOUD_TEST_BUCKET_NAME=dbcloudtest GOOGLE_CLOUD_PROJECT=your_google_project_id ./gcp_db_cloud_test

[dhruba]

Thanks for contributing this code! Can we get an idea of the status of this code, for example, is it being used in any application? Any insights into where/how/scale of thie code being used would be great.

[dhruba]

can we get rid of this change, this is not needed and makes the diff large in size.

[githubzilla]

Done

[githubzilla]

Thanks for contributing this code! Can we get an idea of the status of this code, for example, is it being used in any application? Any insights into where/how/scale of thie code being used would be great.

This pull request (PR) originates from MonographDB. In our current setup, we utilize RocksDB Cloud as the log server backend to store log items, and the S3 backend has been performing effectively. Now, we aim to expand our support to Google Cloud, which serves as the motivation for creating this PR. Although the integration work is still in progress, we anticipate its completion in the near future and are actively working towards its resolution.