Kubernetes scaledown ignoring PDB

We're getting consistent node scaledowns in GKE Autopilot that makes our application unavailable for a few seconds. We have two replicas and a PDB stating that at least one needs to be available. We haven't set up any anti affinity (I'll be doing that next) and both replicas end up on the same node.

According to https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/FAQ.md#does-ca-work-with-poddisruptionbudget-in-scale-down "Before starting to terminate a node, CA makes sure that PodDisruptionBudgets for pods scheduled there allow for removing at least one replica. Then it deletes all pods from a node through the pod eviction API" Do I understand correctly that if both replicas are on the same node this condition will be met because technically one replica can be removed? It just ignores the fact that both replicas will be gone in this case?

For reference here's our PDB status

  status:
    conditions:
    - lastTransitionTime: "2023-07-28T16:03:34Z"
      message: ""
      observedGeneration: 1
      reason: SufficientPods
      status: "True"
      type: DisruptionAllowed
    currentHealthy: 2
    desiredHealthy: 1
    disruptionsAllowed: 1
    expectedPods: 2
    observedGeneration: 1

In PDB, we are having a way to prevent the one replica from getting evicted when there are two replicas from the node by setting the target size which is called minimum availability for a particular type of pod. This indicates that atleast one replica should be running at any time. If the number of running replicas is below the target size, Kubernetes will prevent further disruptions to the remaining replicas until the target size is met.

So, in your case if you set the target size to 2 then one replica will be disrupted and another will be running which makes a node to prevent it from disruption. When a disruption occurs, Kubernetes will attempt to gracefully evict pods from the affected node(s) in order to maintain the desired number of replicas specified in the PDB.

Refer to this blog by Ink Insight which clearly explained about Kubernetes | Pod Disruption Budgets (PDB) from the blog here’s an example of a PDB that sets the target size to 2 for a deployment named “my-deployment” in the “my-namespace”.

> apiVersion: policy/v1beta1
> kind: PodDisruptionBudget
> metadata:
> name: my-pdb
> namespace: my-namespace
> spec:
> minAvailable: 2
> selector:
> matchLabels:
> app: my-deployment

You can also refer to the official doc on specifying a Disruption Budget for your Application and Considering Pod scheduling and disruption .

EDIT : After going through more on this, i understood this :

As you mentioned from the doc CA makes sure that PodDisruptionBudgets for pods scheduled there allow for removing at least one replica. Then it deletes all pods from a node through the pod eviction API"

Actually it doesn't delete all the pods, Here PDB will check its condition and allow one POD to be on the node as per the condition defined (min available : 1) this will prevent pod from eviction.

Then as mentioned in the same doc, If one of the eviction fails, the node is saved and it is not terminated then this way the node will save from disruption.

If we don't mention PDB condition properly then it will automatically delete all the pods and disrupt the node also. It didn't ignore the fact that both replicas will go.