How to do an upgrade from 0.8.0 to 0.10.1 - Database issue - Migration checksum mismatch for migration version 1 #668
Comments
|
Did you try to migrate the database first? e.g. At Square, we apply schema changes in a separate system (not via Flyway), but this should still work. In general, we update Keywhiz quite often, so I might 100% what effect it would have to run migration first, while 0.8 code is still running -- would have to look closer at the list of migrations. |
|
Hi Mat, I have a local db running with a dump from the stg cluster. the idea is see what happen with migrate. url: jdbc:mysql:https://127.0.0.1/keywhiz_stg?useUnicode=true&characterEncoding=utf8 But when I run "migrate", I get the same error: $ java -cp /usr/safenet/lunaclient/jsp/lib/LunaProvider.jar:./keywhiz-server-0.10.1-shaded.jar -Djava.library.path=/usr/safenet/lunaclient/jsp/lib/ keywhiz.KeywhizService migrate ./keywhiz-stg.yml WARNING: An illegal reflective access operation has occurred Right now, I can only think to make this upgrade happen is creating a new DB; run the migrate and somehow restore all the records/data from the existing db (0.8) to the new database (0.10). Thank you again Mat for any tip or idea. |
|
What if you got an older version of Keywhiz? Could you check out earlier commit, maybe version 0.9.0 and try to run migration with that? |
|
Hi @mbyczkowski :) Yeah, I tried to build the 0.9.0 as well :) It is having an issue to build keywhiz-test... because the build can't find this file to download: See the log: [root@keywhiz keywhiz-0.9.0]# mvn clean install |
|
Okay, now I am trying migrate in a empty database (new DB, no table there yet) and I am getting this: $ java -jar ./keywhiz-server-0.10.1-shaded.jar migrate ./keywhiz-stg.yml $ java --version Do you know why I am getting this error? Thank you so much, Flavio. |
|
@FlavioImbertDomingos I talked to @jbpeirce who remembered some of migration issues that we came across. IIUC, there was some breaking change in Flyway (3.x -> 4.x but would need to double check) and we ended up manually changing checksum in our DB some time in early 2018. With regards to your latest comment, I see warnings in the logs but not actual error. Did the migration on the empty DB work as expected or did it stop half-way? |
|
Thanks @mbyczkowski Regarding my last comment... I was trying to do a test with an empty database. I have a new mysql database and I was trying to run "migrate" for the first time $ java --illegal-access=warn -jar ./keywhiz-server-0.10.1-shaded.jar migrate ./keywhiz-stg.yml WARNING: Illegal reflective access by com.fasterxml.jackson.module.afterburner.util.MyClassLoader (file:/opt/keywhiz-server-0.10.1-shaded.jar) to method java.lang.ClassLoader.findLoadedClass(java.lang.String) Am I missing anything? |
|
Thanks @mbyczkowski :) $ java -cp /usr/safenet/lunaclient/jsp/lib/LunaProvider.jar:./keywhiz-server-0.10.1-shaded.jar -Djava.library.path=/usr/safenet/lunaclient/jsp/lib/ keywhiz.KeywhizService migrate ./keywhiz-stg.yml WARNING: An illegal reflective access operation has occurred Thank you so much! Flavio. |
Hi Square Team,
We have a big deployment of Keywhiz (over 23 servers) running 0.8.0. I am trying to test the upgrade and it is not easy because the Java 11 change and the DAtabase changes as well.
How you do this upgrade? Do you have any tip or steps that I can do to make it possible?
Is my only option to drop the existing database, create a new db with the migrate and some how restore the data from the old database to the new one?
Here is what is happening when I try to start or migrate:
$ java -cp /usr/safenet/lunaclient/jsp/lib/LunaProvider.jar:./keywhiz-server-0.10.1-shaded.jar -Djava.library.path=/usr/safenet/lunaclient/jsp/lib/ keywhiz.KeywhizService server ./keywhiz-config.yml
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.fasterxml.jackson.module.afterburner.util.MyClassLoader (file:/opt/keywhiz-server-0.10.1-shaded.jar) to method java.lang.ClassLoader.findLoadedClass(java.lang.String)
WARNING: Please consider reporting this to the maintainers of com.fasterxml.jackson.module.afterburner.util.MyClassLoader
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
org.flywaydb.core.api.FlywayException: Validate failed: Migration checksum mismatch for migration version 1
-> Applied to database : -1043629835
-> Resolved locally : -118804294
at org.flywaydb.core.Flyway.doValidate(Flyway.java:1482)
at org.flywaydb.core.Flyway.access$100(Flyway.java:85)
at org.flywaydb.core.Flyway$2.execute(Flyway.java:1455)
at org.flywaydb.core.Flyway$2.execute(Flyway.java:1448)
at org.flywaydb.core.Flyway.execute(Flyway.java:1711)
at org.flywaydb.core.Flyway.validate(Flyway.java:1448)
at keywhiz.KeywhizService.validateDatabase(KeywhizService.java:171)
at keywhiz.KeywhizService.run(KeywhizService.java:159)
at keywhiz.KeywhizService.run(KeywhizService.java:71)
at io.dropwizard.cli.EnvironmentCommand.run(EnvironmentCommand.java:43)
at io.dropwizard.cli.ConfiguredCommand.run(ConfiguredCommand.java:87)
at io.dropwizard.cli.Cli.run(Cli.java:78)
at io.dropwizard.Application.run(Application.java:93)
at keywhiz.KeywhizService.main(KeywhizService.java:78
This is what I see in the schema_version of my 0.8 deployment...
[keywhiz_stg]> select * from schema_version;
+--------------+----------------+---------+----------------------------------------+------+--------------------------------------------------+-------------+--------------+---------------------+----------------+---------+
| version_rank | installed_rank | version | description | type | script | checksum | installed_by | installed_on | execution_time | success |
+--------------+----------------+---------+----------------------------------------+------+--------------------------------------------------+-------------+--------------+---------------------+----------------+---------+
| 1 | 1 | 1 | create tables | SQL | V1__create_tables.sql | -1043629835 | root | 2017-05-10 16:09:13 | 170 | 1 |
| 2 | 2 | 1.1 | alter secret content to add expiry | SQL | V1.1__alter_secret_content_to_add_expiry.sql | -657209746 | root | 2017-05-10 16:09:13 | 40 | 1 |
| 3 | 3 | 1.2 | alter secret content to remove version | SQL | V1.2__alter_secret_content_to_remove_version.sql | 1602621448 | root | 2017-05-10 16:09:13 | 37 | 1 |
| 4 | 4 | 1.3 | alter secrets to add current | SQL | V1.3__alter_secrets_to_add_current.sql | -533763817 | root | 2017-05-10 16:09:13 | 18 | 1 |
| 5 | 5 | 1.4 | add indexes | SQL | V1.4__add_indexes.sql | 1723007867 | root | 2017-05-10 16:09:13 | 32 | 1 |
| 6 | 6 | 1.5 | alter groups to add metadata | SQL | V1.5__alter_groups_to_add_metadata.sql | -327466309 | root | 2017-05-10 16:09:13 | 20 | 1 |
| 7 | 7 | 1.6 | add accessgrants index | SQL | V1.6__add_accessgrants_index.sql | 195441156 | root | 2017-05-10 16:09:13 | 13 | 1 |
| 8 | 8 | 1.7 | alter clients to add last seen | SQL | V1.7__alter_clients_to_add_last_seen.sql | 327844707 | root | 2017-05-10 16:09:14 | 261 | 1 |
| 9 | 9 | 4 | alter secret content to add hmac | SQL | V4__alter_secret_content_to_add_hmac.sql | 1486396107 | root | 2017-05-10 16:09:14 | 26 | 1 |
| 10 | 10 | 5 | add index current to secrets | SQL | V5__add_index_current_to_secrets.sql | 1338499923 | root | 2017-05-10 16:09:14 | 13 | 1 |
| 11 | 11 | 6 | drop unique constraint on secret name | SQL | V6__drop_unique_constraint_on_secret_name.sql | -1693049885 | gpskwstg | 2017-08-03 18:31:38 | 218 | 1 |
Any idea how I can perform this upgrade from 0.8.0 to 0.10.1?
Thank you so much,
Flavio.
The text was updated successfully, but these errors were encountered: