Migration system for sqlite

[fudanchii]

Hey @bradrydzewski
I tried to integrate your migrate package and write some sqlite specific helper, it turns out quite nice in my opinion. ;)

It has nice wrapper for the migration step, so for most of the time we don't have to write raw sql on our own, for example

some minor caveats:
It still lacks index operation support but it's rather easy to add, I believe.
Also it doesn't have any parameter sanitization, but I'm sure we're not going to write our migration script from the web, so I'll leave it at that for now.

The migration driver implements Operation interface, which is accessible from within migration script. And it's possible to create more drivers for another database just by implementing Operation interface and tell the main program to use that driver.

Anyway, I'm not sure if this is already being worked on, or if it's not being planned to be something like this. So it's okay if you reject this PR just in case. :)

Thanks!

[bradrydzewski]

thanks! this is really good work 👍 i'll get this merged shortly

[vito]

nice!

[bradrydzewski]

@fudanchii this is working really well, great job!

One last request. When a column is renamed (table is renamed, new table created, data copied, old table removed) we lose the indexes.

We might be able to retrieve the indexes prior to renaming the column, and then re-execute:

select * from sqlite_master where type = 'index' and tbl_name = 'repos'

thoughts?

[fudanchii]

Sure, I think I need a couple of steps to fetch all indices info before recreate it,
should check existing index by PRAGMA index_list(repos), filtering the autoindex for unique columns since it will automatically recreated. Then fetching indexed columns e.g. PRAGMA index_info(repo_user_ix). And finally recreate the index after matching it with the renamed/dropped columns just in case.

I'll shoot another PR later tonight (GMT+7 here) :D

Thanks!

[rgarcia]

Are there any docs for how to use this?

[fudanchii]

Not yet as of currently, it's a bit involved to document the components since we haven't document the overall architecture yet. But yes, I plan to setup some wiki article on this.
Thanks for the heads up. 👍

[floatdrop]

@fudanchii maybe pull migration out of the drone to separate package with docs?

[fudanchii]

@bradrydzewski what do you think?
I'm not planning to extract migration package out, since the core concept and some of the code is not my own.

I will fork drone/docs repo and put some documentation over there.

Thanks.

[bradrydzewski]

I actually think your migration package might make a great project that others would find useful :) It is up to you though