What will be the gradient and weight of the particula part of the network if coeffiecient of one of the losses contributed by that network is zero?

Suppose, we have a network consisting of two encoders (Enc1 and Enc2) and one decoder (Dec1) in sequential order.

The first encoder (Enc1) contributes as loss L1, the second encoder (Enc2) contributes as loss L2, and the decoder (Dec1) contributes as loss L3. So, the final loss L will be as follows:

L = coeff1 * L1 + coeff2 * L2 + coeff3 * L3

coeff1, coeff2, and coeff3 are weights for the different losses.

Suppose, we keep coeff2 = 0 for the second encoder (Enc2) then I have two doubts:

(i) Whether gradient will be zero corresponding to L2 or not?
(ii) Whether the weight of the second encoder (Enc2) will change or not as coeff2 is 0?

Let's assume that coeff2=0:

1. >Whether gradient will be zero corresponding to L2 or not?

   Then the L2 loss does not have any effect on the network and the gradient of the parameters with respect to this loss is equal to 0. In particular we have dL2=d(θ_Enc)=0 for all encoders Enc1, Enc2, and Enc3.

2. > Whether the weight of the second encoder Enc2 will change?

   • If L1 and L3 do not depend (mathematically speaking) on parameters from the second encoder (Enc2), then the gradients of those parameters will remain at zero. That's because d(L1+L3)/d(θ_Enc2)=0.

   • If however, these parameters where involved in computing either L1 or L3 then the gradient won't be null. In this case, we have |d(L1)/d(θ_Enc2)|>0 or |d(L3)/d(θ_Enc2)|>0