Can not run maven flyway in Kitodo-DataManagement module #6153

henning-gerhardt · 2024-07-29T06:40:42Z

Describe the bug
Running the validation or migration of FlyWay through mvn flyway:validate -Pflyway in the Kitodo-DataManagement module results in the error

[Kitodo-DataManagement/] $ mvn flyway:validate -Pflyway
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< org.kitodo:kitodo-data-management >------------------
[INFO] Building Kitodo - Data Management 3.7.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.783 s
[INFO] Finished at: 2024-07-29T08:32:44+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project kitodo-data-management: Could not resolve dependencies for project org.kitodo:kitodo-data-management:jar:3.7.0-SNAPSHOT: Failed to collect dependencies at org.kitodo:kitodo-api:jar:3.7.0-SNAPSHOT: Failed to read artifact descriptor for org.kitodo:kitodo-api:jar:3.7.0-SNAPSHOT: org.kitodo:kitodo-production:pom:${revision} was not found in https://repo1.maven.org/maven2 during a previous attempt. This failure was cached in the local repository and resolution is not reattempted until the update interval of central has elapsed or updates are forced -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] https://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

The error message points to wrong artifact descriptor ${revision} which is not available. This was introduced by PR #6127

To Reproduce

Change into the Kitodo-DataManagement module
run mvn flyway:validate -Pflywayor mvn flyway:migrate -Pflyway
See the error message

Expected behavior
Running the maven flyway for validation or migration in the Kitodo-DataManagement module should always be possible

Release
master branch (sha ee97b68)

The text was updated successfully, but these errors were encountered:

stweil · 2024-07-29T07:51:44Z

I tried it on a clean git master with reverted commit 144b93f and get a different error:

[ERROR] Failed to execute goal on project kitodo-data-management: Could not resolve dependencies for project org.kitodo:kitodo-data-management:jar:3.7.0-SNAPSHOT
[ERROR] dependency: org.kitodo:kitodo-api:jar:3.7.0-SNAPSHOT (provided)
[ERROR]   Could not find artifact org.kitodo:kitodo-api:jar:3.7.0-SNAPSHOT

stweil · 2024-07-29T07:53:04Z

mvn flyway:validate -Pflyway and mvn flyway:migrate -Pflyway work with git master when they are started from the root directory.

stweil · 2024-07-29T08:45:59Z

Other Maven commands (for example mvn install) fail, too, when running from a subdirectory. It looks like the root pom.xml is not found correctly. Adding -f ../pom.xml helps (although this should not be required).

henning-gerhardt · 2024-07-29T09:26:14Z

I tried it on a clean git master with reverted commit 144b93f and get a different error:

I did not get your error - instead the flyway validation / migration did work from the Kitodo-DataManagement module:

[Kitodo-DataManagement] $ mvn flyway:validate -Pflyway
[INFO] Scanning for projects...
[INFO] 
[INFO] -----------------< org.kitodo:kitodo-data-management >------------------
[INFO] Building Kitodo - Data Management 3.7.0-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- flyway-maven-plugin:8.4.4:validate (default-cli) @ kitodo-data-management ---
[INFO] Flyway Community Edition 8.4.4 by Redgate
[INFO] Database: jdbc:mariadb:https://localhost/kitodo_maven_37x (MariaDB 10.11)
[WARNING] Flyway upgrade recommended: MariaDB 10.11 is newer than this version of Flyway and support has not been tested. The latest supported version of MariaDB is 10.6.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.033 s
[INFO] Finished at: 2024-07-29T11:24:41+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:8.4.4:validate (default-cli) on project kitodo-data-management: org.flywaydb.core.api.exception.FlywayValidateException: Validate failed: Migrations have failed validation
[ERROR] Detected resolved migration not applied to database: 2.130. To fix this error, either run migrate, or set -ignorePendingMigrations=true.
[ERROR] Detected resolved migration not applied to database: 2.131. To fix this error, either run migrate, or set -ignorePendingMigrations=true.
[ERROR] Need more flexibility with validation rules? Learn more: https://rd.gt/3AbJUZE
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] https://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

The mentioned errors here are excepted as my local database was not already migrated.

mvn flyway:validate -Pflyway and mvn flyway:migrate -Pflyway work with git master when they are started from the root directory.

Not for me:

[kitodo-production] $ mvn flyway:validate -Pflyway
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Kitodo Production                                                  [pom]
[INFO] Kitodo - API                                                       [jar]
[INFO] Kitodo - Data Management                                           [jar]
[INFO] Kitodo - Query URL Import                                          [jar]
[INFO] Kitodo - Command                                                   [jar]
[INFO] Kitodo - File Management                                           [jar]
[INFO] Kitodo - Data Format                                               [jar]
[INFO] Kitodo - Data Editor                                               [jar]
[INFO] Kitodo - Image Management                                          [jar]
[INFO] Kitodo - Long Term Preservation Validation                         [jar]
[INFO] Kitodo - Persistent Identifier                                     [jar]
[INFO] Kitodo - Docket                                                    [jar]
[INFO] Kitodo - XML SchemaConverter                                       [jar]
[INFO] Kitodo - Validation                                                [jar]
[INFO] Kitodo - Core                                                      [war]
[INFO] 
[INFO] --------------------< org.kitodo:kitodo-production >--------------------
[INFO] Building Kitodo Production 3.7.0-SNAPSHOT                         [1/15]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- flyway-maven-plugin:8.4.4:validate (default-cli) @ kitodo-production ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Kitodo Production 3.7.0-SNAPSHOT:
[INFO] 
[INFO] Kitodo Production .................................. FAILURE [  0.422 s]
[INFO] Kitodo - API ....................................... SKIPPED
[INFO] Kitodo - Data Management ........................... SKIPPED
[INFO] Kitodo - Query URL Import .......................... SKIPPED
[INFO] Kitodo - Command ................................... SKIPPED
[INFO] Kitodo - File Management ........................... SKIPPED
[INFO] Kitodo - Data Format ............................... SKIPPED
[INFO] Kitodo - Data Editor ............................... SKIPPED
[INFO] Kitodo - Image Management .......................... SKIPPED
[INFO] Kitodo - Long Term Preservation Validation ......... SKIPPED
[INFO] Kitodo - Persistent Identifier ..................... SKIPPED
[INFO] Kitodo - Docket .................................... SKIPPED
[INFO] Kitodo - XML SchemaConverter ....................... SKIPPED
[INFO] Kitodo - Validation ................................ SKIPPED
[INFO] Kitodo - Core ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.763 s
[INFO] Finished at: 2024-07-29T10:29:04+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:8.4.4:validate (default-cli) on project kitodo-production: org.flywaydb.core.api.FlywayException: No database found to handle jdbc:mariadb:https://localhost/kitodo_maven?useSSL=false&amp;serverTimezone=Europe/Berlin -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] https://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Running the flyway migration from the source root directory did not work in the past and there was no changes which made running from the source root directory working.

henning-gerhardt · 2024-07-29T09:33:39Z

Adding -f ../pom.xml helps (although this should not be required).

No, this did not help. I got with this a similar error message like running on the source root directory:

[Kitodo-Datamanagement] $ mvn flyway:validate -Pflyway -f ../pom.xml
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Kitodo Production                                                  [pom]
[INFO] Kitodo - API                                                       [jar]
[INFO] Kitodo - Data Management                                           [jar]
[INFO] Kitodo - Query URL Import                                          [jar]
[INFO] Kitodo - Command                                                   [jar]
[INFO] Kitodo - File Management                                           [jar]
[INFO] Kitodo - Data Format                                               [jar]
[INFO] Kitodo - Data Editor                                               [jar]
[INFO] Kitodo - Image Management                                          [jar]
[INFO] Kitodo - Long Term Preservation Validation                         [jar]
[INFO] Kitodo - Persistent Identifier                                     [jar]
[INFO] Kitodo - Docket                                                    [jar]
[INFO] Kitodo - XML SchemaConverter                                       [jar]
[INFO] Kitodo - Validation                                                [jar]
[INFO] Kitodo - Core                                                      [war]
[INFO] 
[INFO] --------------------< org.kitodo:kitodo-production >--------------------
[INFO] Building Kitodo Production 3.7.0-SNAPSHOT                         [1/15]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- flyway-maven-plugin:8.4.4:validate (default-cli) @ kitodo-production ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Kitodo Production 3.7.0-SNAPSHOT:
[INFO] 
[INFO] Kitodo Production .................................. FAILURE [  0.781 s]
[INFO] Kitodo - API ....................................... SKIPPED
[INFO] Kitodo - Data Management ........................... SKIPPED
[INFO] Kitodo - Query URL Import .......................... SKIPPED
[INFO] Kitodo - Command ................................... SKIPPED
[INFO] Kitodo - File Management ........................... SKIPPED
[INFO] Kitodo - Data Format ............................... SKIPPED
[INFO] Kitodo - Data Editor ............................... SKIPPED
[INFO] Kitodo - Image Management .......................... SKIPPED
[INFO] Kitodo - Long Term Preservation Validation ......... SKIPPED
[INFO] Kitodo - Persistent Identifier ..................... SKIPPED
[INFO] Kitodo - Docket .................................... SKIPPED
[INFO] Kitodo - XML SchemaConverter ....................... SKIPPED
[INFO] Kitodo - Validation ................................ SKIPPED
[INFO] Kitodo - Core ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.380 s
[INFO] Finished at: 2024-07-29T11:30:53+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:8.4.4:validate (default-cli) on project kitodo-production: org.flywaydb.core.api.FlywayException: No database found to handle jdbc:mysql:https://localhost/kitodo?useSSL=false -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] https://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Only working way for me is to revert the changes of PR #6127.

stweil · 2024-07-29T10:36:11Z

Running the flyway migration from the source root directory did not work in the past and there was no changes which made running from the source root directory working.

Isn't flyway migration (started from the source root directory) part of the normal GitHub CI action? See latest build protocol.

stweil · 2024-07-29T10:46:07Z

It looks like you have different configurations for the database, and only the first one works:

jdbc:mariadb:https://localhost/kitodo_maven_37x
jdbc:mariadb:https://localhost/kitodo_maven?useSSL=false&serverTimezone=Europe/Berlin
jdbc:mysql:https://localhost/kitodo?useSSL=false

henning-gerhardt · 2024-07-29T10:51:18Z

Running the flyway migration from the source root directory did not work in the past and there was no changes which made running from the source root directory working.

Isn't flyway migration (started from the source root directory) part of the normal GitHub CI action? See latest build protocol.

Interesting to know that this could even work from source root directory but this did not work on my local setup.

It looks like you have different configurations for the database, and only the first one works:
1. jdbc:mariadb:https://localhost/kitodo_maven_37x
2. jdbc:mariadb:https://localhost/kitodo_maven?useSSL=false&serverTimezone=Europe/Berlin
3. jdbc:mysql:https://localhost/kitodo?useSSL=false

Because I have many databases with many different migration states? Even with the first mentioned entry jdbc:mariadb:https://localhost/kitodo_maven_37x it does not work from the source root directory.

stweil · 2024-07-29T10:58:43Z

Yes, but the log which you provided for the source root case failed because it used a different database. The -f ../pom.xml case also failed because of the database settings.

henning-gerhardt · 2024-07-29T11:02:50Z

Okay, here is the log

[kitodo-production] $ mvn flyway:validate -Pflyway
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Kitodo Production                                                  [pom]
[INFO] Kitodo - API                                                       [jar]
[INFO] Kitodo - Data Management                                           [jar]
[INFO] Kitodo - Query URL Import                                          [jar]
[INFO] Kitodo - Command                                                   [jar]
[INFO] Kitodo - File Management                                           [jar]
[INFO] Kitodo - Data Format                                               [jar]
[INFO] Kitodo - Data Editor                                               [jar]
[INFO] Kitodo - Image Management                                          [jar]
[INFO] Kitodo - Long Term Preservation Validation                         [jar]
[INFO] Kitodo - Persistent Identifier                                     [jar]
[INFO] Kitodo - Docket                                                    [jar]
[INFO] Kitodo - XML SchemaConverter                                       [jar]
[INFO] Kitodo - Validation                                                [jar]
[INFO] Kitodo - Core                                                      [war]
[INFO] 
[INFO] --------------------< org.kitodo:kitodo-production >--------------------
[INFO] Building Kitodo Production 3.7.0-SNAPSHOT                         [1/15]
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- flyway-maven-plugin:8.4.4:validate (default-cli) @ kitodo-production ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Kitodo Production 3.7.0-SNAPSHOT:
[INFO] 
[INFO] Kitodo Production .................................. FAILURE [  0.239 s]
[INFO] Kitodo - API ....................................... SKIPPED
[INFO] Kitodo - Data Management ........................... SKIPPED
[INFO] Kitodo - Query URL Import .......................... SKIPPED
[INFO] Kitodo - Command ................................... SKIPPED
[INFO] Kitodo - File Management ........................... SKIPPED
[INFO] Kitodo - Data Format ............................... SKIPPED
[INFO] Kitodo - Data Editor ............................... SKIPPED
[INFO] Kitodo - Image Management .......................... SKIPPED
[INFO] Kitodo - Long Term Preservation Validation ......... SKIPPED
[INFO] Kitodo - Persistent Identifier ..................... SKIPPED
[INFO] Kitodo - Docket .................................... SKIPPED
[INFO] Kitodo - XML SchemaConverter ....................... SKIPPED
[INFO] Kitodo - Validation ................................ SKIPPED
[INFO] Kitodo - Core ...................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.461 s
[INFO] Finished at: 2024-07-29T13:01:40+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:8.4.4:validate (default-cli) on project kitodo-production: org.flywaydb.core.api.FlywayException: No database found to handle jdbc:mariadb:https://localhost/kitodo_maven_37x -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] https://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

No difference.

henning-gerhardt · 2024-07-29T11:58:20Z

I tried all 4 variations: with and without PR #6127 and starting the validation / migration from source root or from Kitodo-DataManagement directory.
The only working way on my local setup is without the PR and starting from Kitodo-DataManagement directory. On all other cases I got only errors as already reported here.

This plugin is required for the Maven CI friendly multi module setup, see https://maven.apache.org/maven-ci-friendly.html#install-deploy. Fixes: 144b93f ("Use special property `revision` to set the version only once") Signed-off-by: Stefan Weil <[email protected]>

stweil · 2024-07-31T10:00:52Z

There exist now two pull requests which fix this issue:

PR Revert "Use special property revision to set the version only once" #6156 which restores the status before the commit which introduced the regression.
PR Add org.codehaus.mojo:flatten-maven-plugin (fixes #6153) #6158 which implements the recommended configuration for multi-module projects by adding the flatten-maven-plugin (see https://maven.apache.org/maven-ci-friendly.html#install-deploy).

Only the 2nd alternative allows new use cases like having several builds in the same source tree by using mvn -Drevision=MY_TEST_REVISION [...].

This plugin is required for the Maven CI friendly multi module setup, see https://maven.apache.org/maven-ci-friendly.html#install-deploy. Fixes: 144b93f ("Use special property `revision` to set the version only once") Signed-off-by: Stefan Weil <[email protected]>

henning-gerhardt added the bug label Jul 29, 2024

henning-gerhardt mentioned this issue Jul 30, 2024

Use special property revision to set the version only once #6127

Merged

stweil mentioned this issue Jul 30, 2024

Update version of flyway-maven-plugin (required for newer MariaDB) #6133

Open

solth mentioned this issue Aug 1, 2024

Add org.codehaus.mojo:flatten-maven-plugin (fixes #6153) #6158

Closed

solth added this to the Kitodo.Production 3.7.0 milestone Aug 1, 2024

henning-gerhardt mentioned this issue Aug 2, 2024

Revert "Use special property revision to set the version only once" #6156

Merged

solth closed this as completed in #6156 Aug 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Can not run maven flyway in Kitodo-DataManagement module #6153

Can not run maven flyway in Kitodo-DataManagement module #6153

henning-gerhardt commented Jul 29, 2024

stweil commented Jul 29, 2024

stweil commented Jul 29, 2024

stweil commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024 •

edited

Loading

stweil commented Jul 29, 2024

stweil commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024

stweil commented Jul 29, 2024 •

edited

Loading

henning-gerhardt commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024

stweil commented Jul 31, 2024

Can not run maven flyway in Kitodo-DataManagement module #6153

Can not run maven flyway in Kitodo-DataManagement module #6153

Comments

henning-gerhardt commented Jul 29, 2024

stweil commented Jul 29, 2024

stweil commented Jul 29, 2024

stweil commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024 • edited Loading

stweil commented Jul 29, 2024

stweil commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024

stweil commented Jul 29, 2024 • edited Loading

henning-gerhardt commented Jul 29, 2024

henning-gerhardt commented Jul 29, 2024

stweil commented Jul 31, 2024

henning-gerhardt commented Jul 29, 2024 •

edited

Loading

stweil commented Jul 29, 2024 •

edited

Loading