Speed up refutation by parallelization

[astoeffelbauer]

Hi @amit-sharma,

I've added parallelization to the random cause and the placebo refuter using joblib.

Please let me know what you think of my implementation before I do the same with the rest. For example do you have a suggestion about how to avoid the inner function within refute_estimate which is not overly pretty imo.

I also added to the Getting Started Notebook. Hope this is a good start for the feature.

[amit-sharma]

the commits look good. thanks @astoeffelbauer. Can you also add docstrings for the new parameters that you have introduced and mention that None is the default?

[astoeffelbauer]

Thanks for the comments @amit-sharma. I've made those updates.

Since every refutation method can be sped up by itself, do you think it's really necessary to add the example with the parallel loop through multiple refutation methods at the end of the Getting Started Notebook.

If set to n_jobs=-1 for a single method, it uses all of the CPU power so I don't think that running multiple methods in parallel can be any faster that that. So I feel like that might just overcomplicate things.

[amit-sharma]

thanks @astoeffelbauer Left a few comments. We are getting there :) Happy to merge once these are in.

[astoeffelbauer]

Hi @amit-sharma, thanks again for the comments. I implemented the parallelization for the random_common_cause, placebo_treatment_refuter, and the data_subset_refuter now.

The bootstrap_refuter (raises an error) and the dummy_outcome_refuter (returns a list instead of the results) currently don't work fully or are inconsistent with the others I believe, and they are also not in the Getting Started Notebook, so I skipped them for now.

The other remaining ones are more complex and harder to parallelize I think, and I haven't had a chance to understand them fully yet.

Please let me know if this is fine for now and if you have any other comments. Once we've resolved any last comments or issues, do you think we can finalize and merge in the PR?

[amit-sharma]

Looks good, thank you @astoeffelbauer! We can take up how to parallelize the other refuters in a future PR.

[amit-sharma]

@all-contributors please add @astoeffelbauer for code.

[allcontributors]

@amit-sharma

I've put up a pull request to add @astoeffelbauer! 🎉

[astoeffelbauer]

Thank you @amit-sharma for all the comments and the help!! Hope this will be useful 😊