Reccommend using a submodule+fork for Unbias_LightGBM

[paddy74]

Due to Unbias_LightGBM effectively being a fork of LightGBM, it would be sensible to create a fork of LightGBM with the necessary changes and renamed to Unbais_LightGBM and add that fork as a submodule to this project. The new fork would initially be set to the relevant commit of LightGBM that Unbias_LightGBM is based upon.

This would allow updates and bug fixes to LightGBM to be easily incorporated into this project, as well as additional clarity that Unbias_LightGBM is LightGBM with modification.

[acbull]

That's a reasonable suggestion. So how to modify the current repository to be a fork of LightGBM?

[paddy74]

The current repository would not be a fork. It would contain a submodule (link to another git repository) to the fork.

Before doing any of the following I recommend creating a release. That way your paper is correlated with a specific release (i.e. "This paper is based on v1.0").

Do you recall what LightGBM commit you started at? If so:

• Create a fork of LightGBM
• Rename the fork to Unbias_LightGBM (or whatever desired name)
• Clone the fork to you local machine
• Rollback to the desired commit with

git reset --hard <old-commit-id>
git push -f

• Replace all the LightGBM files with your files currently in Unbias_LightGBM/
• Commit and push
• Remove the directory Unbiased_LambdaMart/Unbias_LightGBM
• Add the fork as a submodule with `git submodule add

Then anytime you clone the current repository you would run git submodule init && git submodule update.

If desired you can then attempt to merge in and test with the latest LightGBM changes. As submodules are created at a commit, you may then update the submodule to the latest commit in the fork with git submodule update --remote --merge and commit and push that.

If you do not recall the exact commit
Try and get as close as possible (prefer a later commit) then test to make sure it doesn't affect your results.

[upbit]

I tried to merge Unbiased_LambdaMart into the original and it is now ready to compile:
upbit/Unbiased_LambdaMart@0071cf2

The submodule method is not easy to track the new version.

[paddy74]

What is it you mean by:

The submodule method is not easy to track the new version.

?

The main project would reference a specific commit of the submodule (preferably a 'release' commit).

[upbit]

Sorry for my poor english. I mean

It is not easy to upgrade to the latest version of LightGBM (eg: v2.2.4).

If forked from microsoft/LightGBM, we can merge upstream changes at any time, just by initiating an merge request microsoft/LightGBM:master -> acbull:master

[paddy74]

Yes that would be the idea. acbull:Unbiased_LightGBM would be a fork of microsoft/LightGBM:release (where release is some release commit).

Then acbull:Unbiased_LightGBM:release would be added as a submodule to this project, acbull:Unbiased_LambdaMART.

Even if the decision is not to use the latest LightGBM, it still provides it as a possibility and links the fork acbull:Unbiased_LightGBM to a specific version of LightGBM. This also makes it significantly easier to distinguish what alterations were made to make it "unbiased".

[robert-howley-zocdoc]

An alternative that would greatly increase the impact of this methodology is to add this to lightgbm itself. Fork lightgbm, add your implementation, and open a pr to merge it back into the source repo.

This seems to be a potentially powerful approach to LTR, I suspect the lightgbm maintainers would be appreciative of the contribution.