forked from getsentry/self-hosted
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Add workstation configuration (getsentry#2968)
* Add workstation configuration These are prebuilt docker images for spinning up a local self-hosted image on the Google Cloud Workstation project. While primarily intended for internal development at Sentry, in theory these can be used by anyone with GCWS project to create a fresh workstation for developing self-hosted via a remote VSCode connection. Users who have GCWS properly configured will be able to use the forthcoming `workstations ...` command in the `sentry` dev CLI to create, manage, and destroy one-off or long-lived workstations in either the pre-install or post-install configuration. Note that the `sentry workstations ...` CLI has not yet landed in the `sentry` repo - those changes are coming soon! Issue: getsentry/team-ospo#240 * Fix shfmt complaints
- Loading branch information
1 parent
5aad393
commit da57b62
Showing
14 changed files
with
258 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#!/bin/bash | ||
# | ||
|
||
# Create getsentry folder and enter. | ||
mkdir /home/user/getsentry | ||
cd /home/user/getsentry | ||
|
||
# Pull down sentry and self-hosted. | ||
git clone https://github.com/getsentry/sentry.git | ||
git clone https://github.com/getsentry/self-hosted.git | ||
cd self-hosted |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
#!/bin/bash | ||
# | ||
|
||
# Install self-hosted. Assumed `200_download-self-hosted.sh` has already run. | ||
./install.sh --skip-commit-check --skip-user-creation --skip-sse42-requirements --no-report-self-hosted-issues | ||
|
||
# Apply CSRF override to the newly installed sentry settings. | ||
echo "CSRF_TRUSTED_ORIGINS = [\"https://9000-$WEB_HOST\"]" >>/home/user/getsentry/self-hosted/sentry/sentry.conf.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
#!/bin/bash | ||
# | ||
|
||
# Add a dot file to the home directory indicating that the setup has been completed successfully. | ||
# The host-side of the connection will look for this file when polling for completion to indicate to | ||
# the user that the workstation is ready for use. | ||
# | ||
# Works under the assumption that this is the last setup script to run! | ||
echo "ready_at: $(date -u +"%Y-%m-%dT%H:%M:%SZ")" >/home/user/.sentry.workstation.remote |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,160 @@ | ||
# Remote Self-Hosted Development on Google Cloud Workstation | ||
|
||
This document specifies how to set up remote workstation development for `self-hosted` using Google | ||
Cloud. While this feature is primarily intended for Sentry developers seeking to develop or test | ||
changes to `self-hosted`, in theory anyone with a Google Cloud account and the willingness to incur | ||
the associated costs could replicate the setup described here. | ||
|
||
The goal of remote workstations is to provide turn-key instances for developing on `self-hosted`, in | ||
either postinstall (the `/.install.sh` script has already run) or preinstall (it has not) modes. By | ||
using Ubuntu as a base image, we are able to provide a fresh development environment that is very | ||
similar to the Linux-based x86 instances that self-hosted is intended to be deployed to. | ||
|
||
Specifically, the goals of this effort are: | ||
|
||
- Create and manage turn-key virtual machines for development in either preinstall or postinstall | ||
mode quickly and with minimal manual user input. | ||
- Simulate real `self-hosted` deployment environments as faithfully as possible. | ||
- Create a smooth developer experience when using VSCode and GitHub. | ||
|
||
The last point is worth emphasizing: this tool is specifically optimized to work well with VSCode | ||
remote server (for the actual development) and GitHub (for pushing changes). Supporting any other | ||
workflows is an explicit ***non-goal*** of this setup. | ||
|
||
The instructions here are for how to setup workstations as an administrator (that is, the person in | ||
charge of managing and paying for the entire fleet of workstations). End users are expected to | ||
create, connect to, manage, and shut down workstations as needed via the the `sentry` developer CLI | ||
using the `sentry workstations ...` set of commands. For most use cases outside of | ||
Sentry-the-company, the administrator and end user will be the same individual: they'll configure | ||
their Google Cloud projects and billing, and then use them via `sentry workstations ...` commands on | ||
their local machine. | ||
|
||
## Configuring Google Cloud | ||
|
||
You'll need to use two Google Cloud services to enable remote `self-hosted` deployment: Google Cloud | ||
Workstations to run the actual virtual machines, and the Artifact Registry to store the base images | ||
described in the adjacent Dockerfiles. | ||
|
||
The rest of this document will assume that you are configuring these services to be used from the | ||
west coast of the United States (ie: `us-west1`), but a similar set of processes could be applied | ||
for any region supported by Google Cloud. | ||
|
||
### Creating an Artifact Registry | ||
|
||
You can create an artifact registry using the Google Cloud Platform UI | ||
[here](https://console.cloud.google.com/artifacts): | ||
|
||
![Create an Artifact Registry](./docs/img/create_artifcat_registry.png) | ||
|
||
The dialog should be straightforward. We'll name our new repository `sentry-workstation-us` and put | ||
it in `us-west1`, but you could change these to whatever options suit your liking. Leave the | ||
remaining configurations as they are: | ||
|
||
![Create a Repository](./docs/img/create_repository.png) | ||
|
||
### Setting up Cloud Workstations | ||
|
||
To use Google Cloud Workstations, you'll need to make at least one workstation cluster, and at least | ||
one configuration therein. | ||
|
||
Navigate to the services [control panel](https://console.cloud.google.com/workstations/overview). | ||
From here, you'll need to make one cluster for each region you plan to support. We'll make one for | ||
`us-west1` in this tutorial, naming it `us-west` for clarity: | ||
|
||
![Create a Workstation cluster](./docs/img/create_cluster.png) | ||
|
||
Now, create a new configuration for that cluster. There are a few choices to make here: | ||
|
||
- Do you want it to be a preinstall (ie, `./install.sh` has not run) or postinstall (it has) | ||
instance? | ||
- Do you want to use a small, standard or large resource allocation? | ||
- How aggressively do you want to auto-sleep and auto-shutdown instances? More aggressive setups | ||
will save money, but be more annoying for end users. | ||
|
||
For this example, we'll make a `postinstall` instance with a `standard` resource allocation, but you | ||
can of course change these as you wish. | ||
|
||
On the first panel, name the instance (we recommend using the convention | ||
`[INSTALL_KIND]-[SIZE]-[CLUSTER_NAME]`, so this one `postinstall-standard-us-west`) and assign it to | ||
the existing cluster: | ||
|
||
![Create a config](./docs/img/create_config_1.png) | ||
|
||
Next, pick a resource and cost saving configuration that makes sense for you. In our experience, an | ||
E2 instance is plenty for most day-to-day development work. | ||
|
||
![Create a config](./docs/img/create_config_2.png) | ||
|
||
On the third panel, select `Custom container image`, then choose one of your `postinstall` images | ||
(see below for how to generate these). Assign the default Compute Engine service account to it, then | ||
choose to `Create a new empty persistent disk` for it. A balanced 10GB disk should be plenty for | ||
shorter development stints: | ||
|
||
![Create a config](./docs/img/create_config_3.png) | ||
|
||
On the last screen, set the appropriate IAM policy to allow access to the new machine for your | ||
users. You should be ready to go! | ||
|
||
## Creating and uploading an image artifact | ||
|
||
Each Cloud Workstation configuration you create will need to use a Docker image, the `Dockerfile`s | ||
and scripts for which are found in this directory. There are two kinds of images: `preinstall` (ie, | ||
`./install.sh` has not run) and `postinstall` (it has). To proceed, you'll need to install the | ||
`gcloud` and `docker` CLI, then login to both and set your project as the default: | ||
|
||
```shell | ||
$> export GCP_PROJECT_ID=my-gcp-project # Obviously, your project is likely to have another name. | ||
$> gcloud auth application-default login | ||
$> gcloud config set project $GCP_PROJECT_ID | ||
$> gcloud auth print-access-token | docker login -u oauth2accesstoken --password-stdin https://us-docker.pkg.dev | ||
``` | ||
|
||
Next, you'll set some useful variables for this session (note: the code below assumes we are pushing | ||
to the `sentry-workstation-us` repository defined above): | ||
|
||
```shell | ||
$> export GROUP=sentry-workstation # Pick whatever name you like here. | ||
$> export REGION=us # Name your regions as you see fit - these are not tied to GCP definitions. | ||
$> export PHASE=pre # Use `pre` for preinstall, `post` for postinstall. | ||
$> export REPO=${GROUP}-${REGION} | ||
$> export IMAGE_TAG=${GROUP}/${PHASE}install:latest | ||
$> export IMAGE_URL=us-docker.pkg.dev/${GCP_PROJECT_ID}/${REPO}/${GROUP}/${PHASE}install:latest | ||
``` | ||
|
||
Now, build the docker image of your choosing: | ||
|
||
```shell | ||
$> docker build -t ${IMAGE_TAG} -f ./${PHASE}install/Dockerfile . | ||
$> docker image ls | grep "${GROUP}/${PHASE}install" | ||
``` | ||
|
||
Finally, upload it to the Google Cloud Artifact Registry repository of interest: | ||
|
||
```shell | ||
$> docker tag ${IMAGE_TAG} ${IMAGE_URL} | ||
$> docker push ${IMAGE_URL} | ||
``` | ||
|
||
## Creating and connecting to a workstation | ||
|
||
Once the Google Cloud services are configured and the docker images uploaded per the instructions | ||
above, end users should be able to list the configurations available to them using `sentry | ||
workstations config`: | ||
|
||
```shell | ||
$> sentry workstations configs --project=$GCP_PROJECT_ID | ||
|
||
NAME CLUSTER REGION MACHINE TYPE | ||
postinstall-standard-us-west us-west us-west1 e2-standard-4 | ||
postinstall-large-us-west us-west us-west1 e2-standard-8 | ||
preinstall-standard-us-west us-west us-west1 e2-standard-4 | ||
preinstall-large-us-west us-west us-west1 e2-standard-8 | ||
``` | ||
|
||
They will then be able to create a new workstation using the `sentry workstations create` command, | ||
connect to an existing one using `sentry workstations connect`, and use similar `sentry workstations | ||
...` commands to disconnect from and destroy workstations, as well as to check the status of their | ||
active connections. The `create` and `connect` commands will provide further instructions in-band on | ||
how to connect a their local VSCode to the remote server, use SSH to connect to the terminal | ||
directly, add a GitHub access token, or access their running `self-hosted` instance via a web | ||
browser. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
sudo wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor >packages.microsoft.gpg | ||
sudo install -D -o root -g root -m 644 packages.microsoft.gpg /etc/apt/keyrings/packages.microsoft.gpg | ||
sudo sh -c 'echo "deb [arch=amd64,arm64,armhf signed-by=/etc/apt/keyrings/packages.microsoft.gpg] https://packages.microsoft.com/repos/code stable main" > /etc/apt/sources.list.d/vscode.list' | ||
sudo rm -f packages.microsoft.gpg |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,32 @@ | ||
# Use a predefined image with no out-of-the-box IDE set up. | ||
FROM us-west1-docker.pkg.dev/cloud-workstations-images/predefined/base:latest | ||
|
||
# Modifications to the `/home` directory must take place outside of the Dockerfile. Add a startup | ||
# script to handle the cloning of the `sentry` and `self-hosted` repositories. | ||
COPY 200_download-self-hosted.sh /etc/workstation-startup.d/ | ||
COPY 201_install-self-hosted.sh /etc/workstation-startup.d/ | ||
COPY 299_setup-completed.sh /etc/workstation-startup.d/ | ||
|
||
RUN chmod -R +x /etc/workstation-startup.d | ||
|
||
# Avoid prompts from apt by setting it to non-interactive. | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
|
||
# Install VSCode and the GitHub CLI. | ||
RUN wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > vscode.gpg \ | ||
&& install -D -o root -g root -m 644 vscode.gpg /etc/apt/keyrings/vscode.gpg \ | ||
&& sh -c 'echo "deb [arch=amd64,arm64,armhf signed-by=/etc/apt/keyrings/vscode.gpg] https://packages.microsoft.com/repos/code stable main" > /etc/apt/sources.list.d/vscode.list' \ | ||
&& rm -f vscode.gpg \ | ||
&& apt-get update \ | ||
&& apt-get install -y code \ | ||
&& apt-get install -y curl \ | ||
&& curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | dd of=/usr/share/keyrings/gh.gpg \ | ||
&& chmod go+r /usr/share/keyrings/gh.gpg \ | ||
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gh.gpg] https://cli.github.com/packages stable main" > /etc/apt/sources.list.d/github-cli.list \ | ||
&& rm -f gh.gpg \ | ||
&& apt-get update \ | ||
&& apt-get install -y gh \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
# Reset the DEBIAN_FRONTEND environment variable. | ||
ENV DEBIAN_FRONTEND= |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,31 @@ | ||
# Use a predefined image with no out-of-the-box IDE set up. | ||
FROM us-west1-docker.pkg.dev/cloud-workstations-images/predefined/base:latest | ||
|
||
# Modifications to the `/home` directory must take place outside of the Dockerfile. Add a startup | ||
# script to handle the cloning of the `sentry` and `self-hosted` repositories. | ||
COPY 200_download-self-hosted.sh /etc/workstation-startup.d/ | ||
COPY 299_setup-completed.sh /etc/workstation-startup.d/ | ||
|
||
RUN chmod -R +x /etc/workstation-startup.d | ||
|
||
# Avoid prompts from apt by setting it to non-interactive. | ||
ENV DEBIAN_FRONTEND=noninteractive | ||
|
||
# Install VSCode and the GitHub CLI. | ||
RUN wget -qO- https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > vscode.gpg \ | ||
&& install -D -o root -g root -m 644 vscode.gpg /etc/apt/keyrings/vscode.gpg \ | ||
&& sh -c 'echo "deb [arch=amd64,arm64,armhf signed-by=/etc/apt/keyrings/vscode.gpg] https://packages.microsoft.com/repos/code stable main" > /etc/apt/sources.list.d/vscode.list' \ | ||
&& rm -f vscode.gpg \ | ||
&& apt-get update \ | ||
&& apt-get install -y code \ | ||
&& apt-get install -y curl \ | ||
&& curl -fsSL https://cli.github.com/packages/githubcli-archive-keyring.gpg | dd of=/usr/share/keyrings/gh.gpg \ | ||
&& chmod go+r /usr/share/keyrings/gh.gpg \ | ||
&& echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gh.gpg] https://cli.github.com/packages stable main" > /etc/apt/sources.list.d/github-cli.list \ | ||
&& rm -f gh.gpg \ | ||
&& apt-get update \ | ||
&& apt-get install -y gh \ | ||
&& rm -rf /var/lib/apt/lists/* | ||
|
||
# Reset the DEBIAN_FRONTEND environment variable. | ||
ENV DEBIAN_FRONTEND= |