Install all the packages from requirments.txt

• Download train and test datasets manually or they will be automatically downloaded from torchvision datasets.
• Experiments are run on Mnist and Cifar.

We provide shell scripts to run all the approaches mentioned in our paper, except for GTG-Shapley. For the implementation of GTG-Shapley, please refer to https://github.com/liuzelei13/GTG-Shapley.

Before running the commands, navigate to the source code directory src/:

cd src

To calculate the actual Shapley value, run:

./RealShapley_Contribution_dist3.sh

For SPACE, run:

./KA_Contribution_dist3.sh

For SPACE(Avg), run:

./FedAvg_Contribution_dist3.sh

For DIG-FL, run:

./DIGFL_Contribution_dist3.sh

For TMC-Shapley, run:

./TMC_Contribution_dist3.sh

For Group Testing, run:

./GT_Contribution_dist3.sh

Note that the shell script runs all the local training sequentially. To run local training in parallel, add --RPC to the command.

To evaluate the Pearson Correlation Coefficient (PCC) between the estimated Shapley value and the actual Shapley value, run:

python3 plot_PCC_calculator.py --m1 gt+tmc+digfl+fedavg+ka

The result will be saved in save/Contribution_results/

For SPACE, run:

./KA_Contribution_dist3.sh

For SPACE (Average), run:

./FedAvg_Contribution_dist3.sh

For DIG-FL, run:

./DIGFL_Contribution_dist3.sh

To plot the reweighting result, run:

python3 plot_reweighting.py --m1 digfl+fedavg+KA

The result will be saved in save/Reweighting_results/

For the client selection task, run:

./KA_Selection.sh

Note that the default implementation amalgamates 100 clients simultaneously. If there is limited GPU memory capacity, you can change the --t_num parameter to adjust the number of teachers in knowledge amalgamation.

To plot the result, run:

python3 plots_paper.py

The results will be saved in src/clustered_sampling/plots/