I have a question about vertical federal learning

[lyh02]

Why does the server have gradients of works here？
Swapping intermediate variables doesn't seem to mean the other party knows the gradient

[lyh02]

In other words，in vertical federated learning, the gradients of works seems does not need to be exchanged with the server。
Even if the server is a worker, it seems impossible to obtain each other's gradient in the process of exchanging intermediate variables

[DeRafael]

Thanks for your question. Although it seems that uploading gradients to the server is unnecessary during the training process, the existence of server is still necessary. The most of the federated learning protocols, an authorized third party is required to ensure the training security. Otherwise, some malicious local workers may launch attack to block normal training. For example, a malicious local worker may provide incorrect intermediate results leading to abnormal model gradients.
Furthermore, the server is also responsible for synchronizing model parameters at each or several steps.

[lyh02]

Thanks for your question. Although it seems that uploading gradients to the server is unnecessary during the training process, the existence of server is still necessary. The most of the federated learning protocols, an authorized third party is required to ensure the training security. Otherwise, some malicious local workers may launch attack to block normal training. For example, a malicious local worker may provide incorrect intermediate results leading to abnormal model gradients. Furthermore, the server is also responsible for synchronizing model parameters at each or several steps.

Thank you for your response，I still have two questions. 1) since there is no need to upload gradient, how can the server launch this attack if there is no gradient? 2) Vertical Federation learning, why does the server need to synchronize model parameters？Synchronization parameters seem to be the only work required for horizontal Federation learning

[DeRafael]

Thank you for your question. For your first question, although there is no need to upload gradient, it is still necessary to synchronize the model parameters during the training process. I will explain why we need to do it later. Since the server has the access to the model parameters in each iteration, it can compute the change of the model parameters in each iteration which has the same meaning as the model gradients especially when they are using some simple optimization methods such as gradient descent.

For your second question, although each local worker can calculate the loss independently after the intermediate results exchange process. It is still insufficient for it to compute the gradients of loss w.r.t all model parameters. In that case, each worker only can optimize/ update part of the model parameter. At the result, model parameter synchronization is required.

Best,
Xiao Jin

[DeRafael]

You are welcomed to send me questions through the email [email protected] which I will response you quicker.

Best,
Xiao Jin