Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add google cloud storage support #284

Merged
merged 22 commits into from
May 4, 2021
Merged

Conversation

NikkyAI
Copy link
Collaborator

@NikkyAI NikkyAI commented Oct 28, 2020

fixes #283

rough first attempt, but it works

refactored and also added support for mavenLocal() and other file:https:// based repos

@NikkyAI NikkyAI force-pushed the feature/gcs branch 2 times, most recently from a84d0cd to 50c2022 Compare October 28, 2020 21:11
@NikkyAI NikkyAI changed the title add google cloud storage support fixes #283 add google cloud storage support Oct 28, 2020
@LouisCAD LouisCAD self-requested a review October 28, 2020 21:50
@LouisCAD LouisCAD self-assigned this Oct 28, 2020
Copy link
Member

@LouisCAD LouisCAD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@NikkyAI There's compilations error right now. I need them to be fixed in order to review the changes.

@jmfayard
Copy link
Member

@NikkyAI
Can you copy the maven repository from Ben Manes's versions plugin ?
This way we can have unit tests for the file:https:// part !
https://github.com/ben-manes/gradle-versions-plugin/tree/master/src/test/resources

Apart from that, I won't be able to help much, I'm moving to Paris this week-end

Copy link
Member

@LouisCAD LouisCAD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR breaks existing functionality: non existing files in a file repo (e.g. mavenLocal) would get an HttpException to be thrown with the 404 code via the OkHttp interceptor, but this change changes the types of the exceptions to be the raw ones (FileNotFoundException). Consequently, I'll need to perform other changes or reachitect the changes.

EDIT: I saw the code getting the exception was catching FileNotFoundException so it wasn't breaking it, but it was duplicating functionality.

@LouisCAD
Copy link
Member

Also using @ExperimentalStdlibApi is fine in end user apps, but it's not in plugins and libraries that need to keep binary compatibility whenever possible (like refreshVersions).

Copy link
Member

@LouisCAD LouisCAD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, answer the questions I asked, but don't push to the branch, I'll finish the work (already started changes).

This adds support for mavenLocal, improves handling of
various cases in Google Cloud Storage repos, and paves
the way for future integrations.
@jmfayard
Copy link
Member

jmfayard commented Nov 6, 2020

I'm sorry Nikky, I know it's frustrating, we need to improve our release process. Hopefully we can have a development version released soon

@LouisCAD
Copy link
Member

LouisCAD commented Nov 9, 2020

@jmfayard Maybe we can put the GCS support behind a flag (e.g. refreshVersions.preview.gcs), so we get a dev release for Nikky to use, and us to benefit from the other great changes of this PR, like the fix for mavenLocal repos?

@jmfayard
Copy link
Member

jmfayard commented Nov 9, 2020

I'm fine with that

@NikkyAI
Copy link
Collaborator Author

NikkyAI commented Nov 9, 2020

well gcs is not even used without a gcs repo being configured.. so i would just print a warning when it is used for now
it is not like anyone who has a working refreshVersions setup would start using gcs accidentally either.. right ?

@jmfayard jmfayard mentioned this pull request Nov 10, 2020
2 tasks
@jmfayard
Copy link
Member

jmfayard commented Nov 10, 2020

@NikkyAI @LouisCAD we can now use jitpack to have dev versions published faster

See #290

For this PR it would be configured this way in settings.gradle.kts

buildscript {
    val githubPR: Int? = 284
    repositories {
        gradlePluginPortal()
        if (githubPR != null) maven("https://jitpack.io")
    }
    val classpath = if (githubPR == null)
        "de.fayard.refreshVersions:refreshVersions:0.9.7" else
        "com.github.jmfayard:refreshVersions:PR${githubPR}-SNAPSHOT"
    dependencies.classpath(classpath)
}

@NikkyAI
Copy link
Collaborator Author

NikkyAI commented Nov 10, 2020

this dependency on jitpack seems to work for me: com.github.jmfayard:refreshVersions:feature~gcs-SNAPSHOTbut then https://github.com/jmfayard/refreshVersions/blob/9d862c71c4cf871d10770a836a25600f3737d68a/plugins/core/src/main/kotlin/de/fayard/refreshVersions/core/internal/CurrentlyUsedVersion.kt#L8 does not match

@NikkyAI NikkyAI force-pushed the feature/gcs branch 2 times, most recently from a335035 to 9d862c7 Compare November 10, 2020 16:27
@LouisCAD
Copy link
Member

@NikkyAI Why did you force-push a change instead of simply pushing a plain commit?

@NikkyAI
Copy link
Collaborator Author

NikkyAI commented Nov 10, 2020

i force pushed undid my changes, because i noticed i was still on the PR branch

@NikkyAI
Copy link
Collaborator Author

NikkyAI commented Nov 10, 2020

it currently seems to break android plugin because of dependencies
can we shadow and relocate gcs and dependencies ?

@LouisCAD
Copy link
Member

@NikkyAI Shadowing dependencies has big implications when it comes to plugin size, and we want to avoid delaying ephemeral CIs (such as GitHub Actions) further for projects using refreshVersions for a feature that many projects will not need.

There might be another way to avoid dependency version conflicts, we'll need to test it.

FYI, I'll get back to this early next year. We will test something with another optional module, and if it works, we might use that same technique for this feature.

@LouisCAD LouisCAD changed the base branch from develop to main December 13, 2020 21:05
@LouisCAD
Copy link
Member

Hello, just to let you know I added this PR into my priority list (currently number 3).

I think we can have an extra module that brings GCS support, and a clever way to enabled it very easily, in a discoverable manner. I'll experiment with that soon.

@LouisCAD LouisCAD assigned LouisCAD and unassigned jmfayard May 2, 2021
# Conflicts:
#	plugins/core/src/main/kotlin/de/fayard/refreshVersions/core/internal/CurrentlyUsedVersion.kt
#	plugins/versions.properties
@LouisCAD
Copy link
Member

LouisCAD commented May 2, 2021

it currently seems to break android plugin because of dependencies

@NikkyAI Please, can you give more details on the actual breakage, and the AGP version(s) you witnessed the issues with?

@LouisCAD
Copy link
Member

LouisCAD commented May 4, 2021

Thanks for your help on Kotlin's Slack in reproducing this and finding a solution.

I'm going to push the changes I made to avoid the issue from triggering.

Without that, the Google Cloud Storage client library would make
refreshVersions and projects using it use an "android" variant,
which would then lead to runtime errors with the Android Gradle Plugin
that relies on the "jre" variant and not the ont built to be embedded
into an Android library or application.
@LouisCAD LouisCAD merged commit c539671 into Splitties:main May 4, 2021
@LouisCAD
Copy link
Member

LouisCAD commented May 4, 2021

Again, thank you very much @NikkyAI for your contribution, your patience and your help!

I think GCS support is very good addition, especially considering the fact that Google is the most environment conscious and friendly companies of its scale in the cloud computing area, and it's been so for a long time (close to 2 decades back).

I think GCS support in refreshVersions has the potential to be very helpful for teams, developers and companies that have private artifacts to store.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Google cloud storage based maven repositories
3 participants