Repository settings not honored

[DanySK]

Describe the bug

Gradle repositories can be configured to expose only specific groups / artifacts, e.g., with:

repositories {
    mavenCentral()
    jcenter {
        content {
            includeGroup("org.jetbrains.kotlinx")
        }
    }
}

JCenter should only get used to retrieve artifacts with group org.jetbrains.kotlinx

Apparently, however, refreshVersions seems not to be honoring the setup. Try for instance:

dependencies {
    implementation("org.eclipse.mylyn.github:org.eclipse.egit.github.core:_")
}

Latest version on Central (at the moment of writing) is 2.1.5.

refreshVersions detects version 3.4.0.201406110918-r, which is not available.

version.org.eclipse.mylyn.github..org.eclipse.egit.github.core=2.1.5
##                                                 # available=3.4.0.201406110918-r

To Reproduce

An example is provided in this PR: DanySK/upgradle#8
The build fails, Gradle (legitimately) does not find the updated artifact:
https://travis-ci.com/github/DanySK/upgradle/jobs/301834976

Commenting out the jcenter repo declaration makes the plugin work as expected.

Expected behavior
No update is suggested, due to the repository configuration

Additional context
If needed, I can produce a minimal example starting from the repo in which I'm having issues, but the problem should be quite well isolated already. Let me know

[DanySK]

(side note, I'm preparing a bot that supports creating PRs out of the result of the refreshVersion execution, it's very early, but might become a useful complementary tool)

[LouisCAD]

Hi @DanySK, thanks manifesting!
I knew that it was a feature we don't currently support but was wondering if it'd actually bother anyone.

I'll revisit this issue after #153 is resolved, to see if after the behavior change, the issue is still important to address, and how we can address it best.

Regarding your bot, that's very cool, please open an issue or reach out on Kotlin's Slack (channel linked from a badge in README) when you have something to show! I know some people like @martinbonnin will be interested!

[LouisCAD]

@DanySK Are you aware of a way for a Gradle plugin to read the content from a repository? My research has been unsuccessful in finding a public API that allows to get this information so far.

[DanySK]

I don't know. I don't think you can fetch the entirety of the contents of a repository, as some format don't even have a summary file and you'd need to crawl it in its entirety.

How does the plugin get information about all available versions of a dependency right now?

[LouisCAD]

I'm not talking about fetching the entirety of a repository, I'm talking about the content declaration you mentioned initially. Did you check the context when replying?

About available versions, it's simply by reading the maven-metdata.xml files, which location can be inferred from artifact coordinates.

[DanySK]

Ah, I see. I don't know the gradle API that well. I think one possibility is to perform the same query the Gradle dependency resolver would -- but I never explored what the gradle API allows for in that regard.

[DanySK]

I took some time to take a deeper look to the Gradle API. We need to access content. The problem is, it is not exposed in the public API, and implementations such as AbstractArtifactRepository hide their RepositoryContentDescriptor.

I see two possibilities:

1. The cleaner is to make sure the repository is a ContentFilteringRepository, then use the getContentFilter and leverage the Action<in ArtifactResolutionDetails> by creating a custom ArtifactResolutionDetails, whose implementation of notFound works as a filter for the refreshVersions query. It's not trivial and the API is internal and not super clear.
2. This is dirtier but closer to what you ask @LouisCAD: call content, and from within the Action<in RepositoryContentDescriptor>bind the provided RepositoryContentDescriptor to an external variable, and use it directly. The problem with this approach is that it does not seem to be inspectionable. You'd need to cast it to something like DefaultRepositoryContentDescriptor and use the reflection anyway to go get private fields.

[LouisCAD]

I'll do my best to ensure we never rely on a Gradle internal API because these are internals that can change and have no API stability guarantees, and I don't want to break users in the future.

In the meantime, a feature request has been filed to Gradle, so, @DanySK, please, put your 👍 on this issue instead: gradle/gradle#13952

[DanySK]

Let's hope to receive some love from Gradle then. I bet repo querying could be of use beyond the scopes of refreshVersions by the way.