How does the training variable affects BatchNormalization in tensorflow and what it is its default value?

When unfreezeng layers with Efficientnet it is recommended to put the variable training in False (https://www.tensorflow.org/tutorials/images/transfer_learning?hl=es-419#important_note_about_batchnormalization_layers).

So i set it to false

model = efficientnet.EfficientNetB1(input_shape=input_1_shape, include_top=False)

model.trainable = not freeze

x = model_trained(input_1, training=self.training_batch)

But when changing the variable I look at the actual layer while debugging and I see no change in the BatchNormalization layers

In both cases it has 2 trainable_variables, 2 trainable_weights and 2 untrainable_variables, 2 untrainable_weights.

I am using Tensorflow 2.6.2

I'd like to know where I could see that my changes are applying or an explanation of what it is actually changing and what it is the actual default value of the variable.

The training variable in BatchNormalization affects based on below criteria whether or not the layer will normalize its inputs:

1. When the training variable is True, the layer will normalize its inputs.
2. When the training variable is False, the layer will not normalize its inputs.
3. The default value of the training variable is True.

The only built-in layer BatchNormalization has both trainable weights and non-trainable weights unlikely all other layer has only trainable weights.<br>
As mentioned in important notes about BN layers, The models which contain tf.keras.layers.BatchNormalization is a special case and precautions should be taken in the context of fine tuning.

When you set layer.trainable = False, The BatchNormalization layer will run in inference mode and will not update its trainable variables (to track mean and variance statistics), means value is no longer updated during training.

It uses non-trainable weights to keep track of the mean and variance of its inputs during training. Setting layer.trainable = False moves all the layer's weights from trainable to non-trainable.

You can check the updated variable weights using :

print(len(model.trainable_weights))
#print(model.trainable_weights)

print(len(model.non_trainable_weights))
#print(model.non_trainable_weights[:5])

Output:

2
260

You can refer to this link for more details.