Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow PR tests to run in CI #100

Merged
merged 5 commits into from
Dec 23, 2017
Merged

Allow PR tests to run in CI #100

merged 5 commits into from
Dec 23, 2017

Conversation

Ianfeather
Copy link
Contributor

Hey there

Following on from the conversation on #94, I took a look at the different options for running the tests in CI.

There are a few different ways we could take it:

  1. Rewrite the current tests to mock S3 completely.
    In this situation the tests would still look like integration tests but wouldn't actually be testing the storage backend.

  2. Create new unit tests which mock s3, keep the existing integration tests.
    note: the integration tests would still not be able to run on CI because they would require the decrypted secrets. If we believe there is value in keeping the integration tests (I do personally) this doesn't get us anywhere unless we introduce a dependency on the maintainers to test PRs.

The plus side of this is we'd get unit tests - but it would require a decent amount of work to add them and doesn't seem strictly related to this problem (would be nice to have regardless)

  1. Don't test S3 on pull requests, where secrets are unavailable, but do run all the other tests.
    In this scenario there would be a dependency on the maintainers to run the full suite of tests. This could be done locally or by moving the branch to jdxcode/npm-register where secrets would be decryptable by circle CI

The solution I've included for now is number 3. It's the simplest and unblocks the CI pipeline. Authors of PRs get some feedback on their PR and we are not in a less safe place than we are currently where none of the tests can run.

Would love to get your thoughts on the above @dgautsch @jdxcode

@codecov
Copy link

codecov bot commented Dec 18, 2017

Codecov Report

❗ No coverage uploaded for pull request base (master@ea4df04). Click here to learn what that means.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##             master     #100   +/-   ##
=========================================
  Coverage          ?   89.46%           
=========================================
  Files             ?       27           
  Lines             ?      427           
  Branches          ?        0           
=========================================
  Hits              ?      382           
  Misses            ?       45           
  Partials          ?        0

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea4df04...59aec6a. Read the comment docs.

@dgautsch
Copy link
Collaborator

Hey Ian, I like the approach you've taken and I agree we will need to figure out the S3 tests for integration in the future. Right now @jdxcode is the only who can run these. For now I think we should move forward. What do you think JD?

@Ianfeather
Copy link
Contributor Author

fwiw anyone can run the integration tests locally (if you set up your own s3 bucket). I've been running them to make sure i didn't break anything.

My assumption was also that they would work in CI on master because circleci will be able to decrypt the secrets there?

@Ianfeather
Copy link
Contributor Author

@jdxcode Any thoughts on this?

@jdx jdx merged commit 262e563 into jdx:master Dec 23, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants