Kubernetes Pod Deployment: Terminating, Logs show mkdir:permission denied

I want to deploy postgres using kubernetes

This is my postgres pod yaml file

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
  labels:
    app: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      securityContext:
        runAsUser: 70
        runAsGroup: 70
        fsGroup: 70
        fsGroupChangePolicy: "Always"
      containers:
        - image: docker.io/postgres:14.8-alpine3.18
          name: postgres
          resources: 
            limits:
              hugepages-2Mi: 512Mi
              memory: 2Gi
              cpu: "8"
            requests:
              memory: 128Mi
              cpu: "1"
          env:
            - name: POSTGRES_DB
              value: postgres_db_name
            - name: POSTGRES_USER
              value: postgres_db_user
            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: postgres-secrets
                  key: root_password_key
            - name: PGDATA
              value: /some/path/here
          ports:
            - containerPort: 5432
              name: postgres
          volumeMounts:
            - name: postgres-volume-name
              mountPath: /some/path/here
      volumes:
        - name: postgres-volume-name
          persistentVolumeClaim:
            claimName: postgres-pv-claim

After running

> kubectl get pods

I POD status is terminating, so I have checked logs and
It shows

> mkdir: can't create directory '/some/path/here': Permission denied

How can I solve this?
Thanks!

The error you are seeing is due to the file permission and you are not able to create a directory.

You can change the security context first place to override the permission & if not you can use the init container to change the file permission or create a directory.

dnsPolicy: ClusterFirst
  initContainers:
  - command:
    - sh
    - -c
    - chown -R 1000:1000 /usr/share/XYZ/data
    - mkdir /usr/share/XYZ
    - sysctl -w vm.max_map_count=262144
    - chmod 777 /usr/share/XYZ
    image: busybox:1.29.2
    imagePullPolicy: IfNotPresent
    name: set-dir-owner
    resources: {}
    securityContext:
      privileged: true
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /usr/share/XYZ
      name: data
  restartPolicy: Always

Try this stateful set if it's fine with you

apiVersion: apps/v1  
kind: StatefulSet
metadata:
  name: postgres
spec:
  serviceName: "postgres"
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
      - name: postgres
        image: postgres:9.5
        volumeMounts:
        - name: postgres-data
          mountPath: /var/lib/postgresql/data
          subPath: pgdata
        env:
        - name: POSTGRES_USER
          value: root
        - name: POSTGRES_PASSWORD
          value: password
        - name: POSTGRES_DB
          value: kong
        - name: PGDATA
          value: /var/lib/postgresql/data/pgdata
        ports:
        - containerPort: 5432
      terminationGracePeriodSeconds: 60
  volumeClaimTemplates:
  - metadata:
      name: postgres-data
    spec:
      accessModes:
      - "ReadWriteOnce"
      resources:
        requests:
          storage: 3Gi

As per official Kubernetes doc on Allow users to skip recursive permission changes on mount:

While inspecting the YAML used for the StatefulSet, noticed there's the use of a fsGroup inside the pod’s security context, which makes sure that the volume's content can be readable and writable by each new pod. One side-effect of setting fsGroup is that, each time a volume is mounted, Kubernetes must recursively change the owner and permission of all the files and directories inside the volume. This happens even if group ownership of the volume already matches the requested 'fsGroup', and can be pretty expensive for larger volumes with lots of small files, which causes pod startup to take a long time.

Solution : As per Configure volume permission and ownership change policy for Pods. Suggest setting 'fsGroupChangePolicy' to "OnRootMismatch" so if the root of the volume already has the correct permissions, the recursive permission change can be skipped.

> fsGroupChangePolicy - fsGroupChangePolicy defines behavior for
> changing ownership and permission of the volume before being exposed
> inside a Pod. This field only applies to volume types that support
> fsGroup controlled ownership and permissions. This field has two
> possible values:
>
> OnRootMismatch: Only change permissions and ownership if the
> permission and the ownership of root directory does not match with
> expected permissions of the volume. This could help shorten the time
> it takes to change ownership and permission of a volume.
>
> Always: Always change permission and ownership of the volume when
> volume is mounted.
>
> For example:
>
> securityContext:
>
> runAsUser: 1000
> runAsGroup: 3000
> fsGroup: 2000
> fsGroupChangePolicy: "OnRootMismatch"

*Also refer to the System Admin blog by LiveStream related to the Error, which may help to resolve your issue.