Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ci] move data integration tests to civ2 #40658

Merged
merged 1 commit into from
Oct 27, 2023
Merged

[ci] move data integration tests to civ2 #40658

merged 1 commit into from
Oct 27, 2023

Conversation

can-anyscale
Copy link
Collaborator

  • Add java build option (one of the test required java build)
  • Move starting mongodb to mongo_test setup

Test:

  • CI

@@ -3,6 +3,8 @@ FROM $DOCKER_IMAGE_BASE_TEST

COPY . .

RUN apt-get install -y -qq maven openjdk-8-jre openjdk-8-jdk
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

need this to build java

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how much size will this add? I recall that the size of maven and jdk is non-trivial actually. should we consider split java required stuff into another base? (we can reuse this build file)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can make it so that we install this only for the data.mongodb build. There is only one data test requires java to be installed.

@can-anyscale
Copy link
Collaborator Author

CC: @c21 , @scottjlee , @raulchen if you can help review the changes to python/ray/data/tests/test_mongo.py i would appreciate, thankks

Copy link
Collaborator

@aslonnie aslonnie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

change looks ok high level, I worry that installing java makes all ml jobs slower as downloading the image now takes longer.

Signed-off-by: can <[email protected]>
@can-anyscale can-anyscale merged commit f2f1e63 into master Oct 27, 2023
43 of 47 checks passed
@can-anyscale can-anyscale deleted the can-data-01 branch October 27, 2023 02:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants