Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integration test improvements #2858

Merged
merged 11 commits into from
Mar 7, 2024
Merged

Integration test improvements #2858

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
02ccedd
integration test improvements
hubertdeng123 Mar 6, 2024
07c059e
fix sentry send event
hubertdeng123 Mar 6, 2024
364d0d2
install self-hsoted as separate step in gh action
hubertdeng123 Mar 6, 2024
9dd9ac0
do not report self-hosted issues during install
hubertdeng123 Mar 6, 2024
c21576f
run jobs for compose v2.0.1 and v2.7.0
hubertdeng123 Mar 6, 2024
4308b2c
incorporate sentry dsn for testing into existing logic
hubertdeng123 Mar 6, 2024
b22f1eb
fix test type
hubertdeng123 Mar 7, 2024
9c4fbfc
remove retries for now
hubertdeng123 Mar 7, 2024
ffabc9f
comment onto using exec
hubertdeng123 Mar 7, 2024
2ced8fb
fix error reporting
hubertdeng123 Mar 7, 2024
7559e55
add error handling to test.sh
hubertdeng123 Mar 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 9 additions & 1 deletion .github/workflows/test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,16 @@ jobs:
strategy:
fail-fast: false
matrix:
test_type: ["initial-install", "customizations"]
compose_version: ["v2.0.1", "v2.7.0"]
include:
- compose_version: "v2.0.1"
compose_path: "/usr/local/lib/docker/cli-plugins"
- compose_version: "v2.7.0"
compose_path: "/usr/local/lib/docker/cli-plugins"
env:
COMPOSE_PROJECT_NAME: self-hosted-${{ strategy.job-index }}
SENTRY_DSN: https://[email protected]/6627632
steps:
- name: Checkout
uses: actions/checkout@v4
Expand All @@ -82,8 +85,13 @@ jobs:
sudo curl -L https://github.com/docker/compose/releases/download/${{ matrix.compose_version }}/docker-compose-`uname -s`-`uname -m` -o "${{ matrix.compose_path }}/docker-compose"
sudo chmod +x "${{ matrix.compose_path }}/docker-compose"

- name: Install self-hosted
run: |
export REPORT_SELF_HOSTED_ISSUES=1
./install.sh

- name: Integration Test
run: ./integration-test.sh
run: ./integration-test.sh --${{ matrix.test_type }}

- name: Inspect failure
if: failure()
Expand Down
4 changes: 2 additions & 2 deletions _integration-test/run.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,10 +41,10 @@ echo "${_endgroup}"
echo "${_group}Starting Sentry for tests ..."
# Disable beacon for e2e tests
echo 'SENTRY_BEACON=False' >>$SENTRY_CONFIG_PY
echo y | $dcr web createuser --force-update --superuser --email $TEST_USER --password $TEST_PASS
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docker run is slower than just an exec into a running container

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is but it is also intentional to keep the run separate from the main web process.

That said for a one-off thing like this, I think using exec is a good compromise if it saves a notable amount of time. I'd just want it documented with a brief comment.

$dc up -d
printf "Waiting for Sentry to be up"
timeout 90 bash -c 'until $(curl -Isf -o /dev/null $SENTRY_TEST_HOST); do printf '.'; sleep 0.5; done'
echo y | $dc exec web sentry createuser --force-update --superuser --email $TEST_USER --password $TEST_PASS
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if we exec into the web container but it isn't ready yet? Does docker wait until it is ready, or does this fail? If it's the former, we should echo "Waiting for Sentry..." before this runs, otherwise the user may be waiting a while. If it fails, we should add some sort of sync point to wait for the container to be up before trying this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker compose up will only succeed if the container healthcheck for web passes, so I don't think this will be a problem. That is performed on a previous line, so when the tests get to the createuser logic the web container will always be ready

printf "Waiting for Sentry to be up"
echo ""
echo "${_endgroup}"

Expand Down
8 changes: 4 additions & 4 deletions install/error-handling.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
echo "${_group}Setting up error handling ..."

export SENTRY_DSN='https://[email protected]/3'
export SENTRY_ORG=self-hosted
export SENTRY_PROJECT=installer
if [ -z "${SENTRY_DSN:-}" ]; then
export SENTRY_DSN='https://19555c489ded4769978daae92f2346ca@self-hosted.getsentry.net/3'
fi

$dbuild -t sentry-self-hosted-jq-local --platform="$DOCKER_PLATFORM" jq

jq="docker run --rm -i sentry-self-hosted-jq-local"
sentry_cli="docker run --rm -v /tmp:/work -e SENTRY_ORG=$SENTRY_ORG -e SENTRY_PROJECT=$SENTRY_PROJECT -e SENTRY_DSN=$SENTRY_DSN getsentry/sentry-cli"
sentry_cli="docker run --rm -v /tmp:/work -e SENTRY_DSN=$SENTRY_DSN getsentry/sentry-cli"

send_envelope() {
# Send envelope
Expand Down
34 changes: 20 additions & 14 deletions integration-test.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,24 +6,30 @@ rm -f sentry/enhance-image.sh
rm -f sentry/requirements.txt
export REPORT_SELF_HOSTED_ISSUES=0

echo "Testing initial install"
./install.sh
_integration-test/run.sh
_integration-test/ensure-customizations-not-present.sh
_integration-test/ensure-backup-restore-works.sh
test_option="$1"

echo "Make customizations"
cat <<EOT >sentry/enhance-image.sh
if [[ "$test_option" == "--initial-install" ]]; then
echo "Testing initial install"
_integration-test/run.sh
_integration-test/ensure-customizations-not-present.sh
_integration-test/ensure-backup-restore-works.sh
elif [[ "$test_option" == "--customizations" ]]; then
echo "Testing customizations"
source install/dc-detect-version.sh
$dc up -d
echo "Making customizations"
cat <<EOT >sentry/enhance-image.sh
#!/bin/bash
touch /created-by-enhance-image
apt-get update
apt-get install -y gcc libsasl2-dev python-dev libldap2-dev libssl-dev
EOT
chmod +x sentry/enhance-image.sh
printf "python-ldap" >sentry/requirements.txt
chmod +x sentry/enhance-image.sh
printf "python-ldap" >sentry/requirements.txt

echo "Testing in-place upgrade and customizations"
./install.sh --minimize-downtime
_integration-test/run.sh
_integration-test/ensure-customizations-work.sh
_integration-test/ensure-backup-restore-works.sh
echo "Testing in-place upgrade and customizations"
./install.sh --minimize-downtime
_integration-test/run.sh
_integration-test/ensure-customizations-work.sh
_integration-test/ensure-backup-restore-works.sh
fi
Loading