英文:
CrashLoopBackOff error while deploying postgresql to gke
问题
我正在尝试将 PostgreSQL 部署到 GKE，以下是我的 PersistentVolumeClaim 定义：
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-pvc
namespace: db
labels:
app: imgress-db
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 400Mi
以及部署/服务定义：
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: imgress-db
namespace: db
spec:
serviceName: imgress-db
replicas: 1
selector:
matchLabels:
app: imgress-db
template:
metadata:
labels:
app: imgress-db
spec:
containers:
- name: imgress-db
image: postgres
env:
- name: POSTGRES_HOST
valueFrom:
configMapKeyRef:
name: db-configmap
key: DATABASE_HOST
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_DB
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: POSTGRES_PASSWORD
ports:
- containerPort: 5432
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
volumes:
- name: postgres-data
persistentVolumeClaim:
claimName: postgres-pvc
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: imgress-db
namespace: db
spec:
selector:
app: imgress-db
ports:
- name: postgres
port: 5432
首先，我运行以下命令：
kubectl apply -f postgres-pvc.yaml
然后运行：
kubectl apply -f postgres-deployment.yaml
但当我运行 kubectl get pods -A 时，我遇到了以下错误：
NAMESPACE NAME READY STATUS RESTARTS AGE
db imgress-db-0 0/1 CrashLoopBackOff 6 (2m15s ago) 8m26s
对于 kubectl describe pvc postgres-pvc -n db，我得到以下结果：
Name: postgres-pvc
Namespace: db
StorageClass: standard
Status: Bound
Volume: pvc-c6369764-1106-4a7d-887c-0e4009968115
Labels: app=imgress-db
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
volume.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 1Gi
Access Modes: RWO
VolumeMode: Filesystem
Used By: imgress-db-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ExternalProvisioning 31m persistentvolume-controller waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by a system administrator
Normal Provisioning 31m pd.csi.storage.gke.io_gke-e0f710dc594c4eb5ac14-5c62-e039-vm_ca2409ad-83a8-4139-93b4-4fffbacbf44f External provisioner is provisioning a volume for claim "db/postgres-pvc"
Normal ProvisioningSucceeded 31m pd.csi.storage.gke.io_gke-e0f710dc594c4eb5ac14-5c62-e039-vm_ca2409ad-83a8-4139-93b4-4fffbacbf44f Successfully provisioned volume pvc-c6369764-1106-4a7d-887c-0e4009968115
对于 kubectl describe pod imgress-db-0 -n db，我得到以下结果（请注意最后一行的 Back-off restarting failed container）：
Name: imgress-db-0
Namespace: db
Priority: 0
Service Account: default
Node: gke-imgress-default-pool-e9bdef38-hjhv/10.156.0.5
Start Time: Fri, 24 Feb 2023 13:44:15 +0500
Labels: app=imgress-db
controller-revision-hash=imgress-db-7f557d4b88
statefulset.kubernetes.io/pod-name=imgress-db-0
Annotations: <none>
Status: Running
IP: 10.84.2.49
IPs:
IP: 10.84.2.49
Controlled By: StatefulSet/imgress-db
Containers:
imgress-db:
Container ID: containerd://96140ec0b0e369ca97822361a770abcb82e27b7924bc90e17111ab354e51d6aa
Image: postgres
Image ID: docker.io/library/postgres@sha256:901df890146ec46a5cab7a33f4ac84e81bac2fe92b2c9a14fd649502c4adf954
Port: 5432/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 24 Feb 2023 13:50:09 +0500
Finished: Fri, 24 Feb 2023 13:50:11 +0500
Ready: False
Restart Count: 6
Environment:
POSTGRES_HOST: <set to the key 'DATABASE_HOST' of config map 'db-configmap'> Optional: false
POSTGRES_DB: <set to the key 'POSTGRES_DB' of config map 'db-configmap'> Optional: false
POSTGRES_USER: <set to the key 'POSTGRES_USER' of config map 'db-configmap'> Optional: false
POSTGRES_PASSWORD: <set to the key 'POSTGRES_PASSWORD' in secret 'db-secret'>
<details>
<summary>英文:</summary>
I am trying to deploy PostgreSQL to GKE and here is my PersistentVolumeClaim definition:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: postgres-pvc
namespace: db
labels:
app: imgress-db
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 400Mi
and this is deployment/service definition:
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: imgress-db
namespace: db
spec:
serviceName: imgress-db
replicas: 1
selector:
matchLabels:
app: imgress-db
template:
metadata:
labels:
app: imgress-db
spec:
containers:
- name: imgress-db
image: postgres
env:
- name: POSTGRES_HOST
valueFrom:
configMapKeyRef:
name: db-configmap
key: DATABASE_HOST
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_DB
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: POSTGRES_PASSWORD
ports:
- containerPort: 5432
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
volumes:
- name: postgres-data
persistentVolumeClaim:
claimName: postgres-pvc
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: imgress-db
namespace: db
spec:
selector:
app: imgress-db
ports:
- name: postgres
port: 5432
First I run:
kubectl apply -f postgres-pvc.yaml
and then:
kubectl apply -f postgres-deployment.yaml
but I get this notorious error when I run `kubectl get pods -A`:
NAMESPACE NAME READY STATUS RESTARTS AGE
db imgress-db-0 0/1 CrashLoopBackOff 6 (2m15s ago) 8m26s
For `kubectl describe pvc postgres-pvc -n db` I get this result:
Name: postgres-pvc
Namespace: db
StorageClass: standard
Status: Bound
Volume: pvc-c6369764-1106-4a7d-887c-0e4009968115
Labels: app=imgress-db
Annotations: pv.kubernetes.io/bind-completed: yes
pv.kubernetes.io/bound-by-controller: yes
volume.beta.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
volume.kubernetes.io/storage-provisioner: pd.csi.storage.gke.io
Finalizers: [kubernetes.io/pvc-protection]
Capacity: 1Gi
Access Modes: RWO
VolumeMode: Filesystem
Used By: imgress-db-0
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal ExternalProvisioning 31m persistentvolume-controller waiting for a volume to be created, either by external provisioner "pd.csi.storage.gke.io" or manually created by system administrator
Normal Provisioning 31m pd.csi.storage.gke.io_gke-e0f710dc594c4eb5ac14-5c62-e039-vm_ca2409ad-83a8-4139-93b4-4fffbacbf44f External provisioner is provisioning volume for claim "db/postgres-pvc"
Normal ProvisioningSucceeded 31m pd.csi.storage.gke.io_gke-e0f710dc594c4eb5ac14-5c62-e039-vm_ca2409ad-83a8-4139-93b4-4fffbacbf44f Successfully provisioned volume pvc-c6369764-1106-4a7d-887c-0e4009968115
and for `kubectl describe pod imgress-db-0 -n db` I get this result (please pay attention to `Back-off restarting failed container` on the last line):
Name: imgress-db-0
Namespace: db
Priority: 0
Service Account: default
Node: gke-imgress-default-pool-e9bdef38-hjhv/10.156.0.5
Start Time: Fri, 24 Feb 2023 13:44:15 +0500
Labels: app=imgress-db
controller-revision-hash=imgress-db-7f557d4b88
statefulset.kubernetes.io/pod-name=imgress-db-0
Annotations: <none>
Status: Running
IP: 10.84.2.49
IPs:
IP: 10.84.2.49
Controlled By: StatefulSet/imgress-db
Containers:
imgress-db:
Container ID: containerd://96140ec0b0e369ca97822361a770abcb82e27b7924bc90e17111ab354e51d6aa
Image: postgres
Image ID: docker.io/library/postgres@sha256:901df890146ec46a5cab7a33f4ac84e81bac2fe92b2c9a14fd649502c4adf954
Port: 5432/TCP
Host Port: 0/TCP
State: Waiting
Reason: CrashLoopBackOff
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Fri, 24 Feb 2023 13:50:09 +0500
Finished: Fri, 24 Feb 2023 13:50:11 +0500
Ready: False
Restart Count: 6
Environment:
POSTGRES_HOST: <set to the key 'DATABASE_HOST' of config map 'db-configmap'> Optional: false
POSTGRES_DB: <set to the key 'POSTGRES_DB' of config map 'db-configmap'> Optional: false
POSTGRES_USER: <set to the key 'POSTGRES_USER' of config map 'db-configmap'> Optional: false
POSTGRES_PASSWORD: <set to the key 'POSTGRES_PASSWORD' in secret 'db-secret'> Optional: false
Mounts:
/var/lib/postgresql/data from postgres-data (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-tfsf9 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
postgres-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: postgres-pvc
ReadOnly: false
kube-api-access-tfsf9:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 6m51s default-scheduler Successfully assigned db/imgress-db-0 to gke-imgress-default-pool-e9bdef38-hjhv
Normal SuccessfulAttachVolume 6m46s attachdetach-controller AttachVolume.Attach succeeded for volume "pvc-c6369764-1106-4a7d-887c-0e4009968115"
Normal Pulled 6m42s kubelet Successfully pulled image "postgres" in 365.355391ms
Normal Pulled 6m39s kubelet Successfully pulled image "postgres" in 236.532115ms
Normal Pulled 6m26s kubelet Successfully pulled image "postgres" in 278.12995ms
Normal Created 5m59s (x4 over 6m42s) kubelet Created container imgress-db
Normal Started 5m59s (x4 over 6m41s) kubelet Started container imgress-db
Normal Pulled 5m59s kubelet Successfully pulled image "postgres" in 348.694043ms
Normal Pulling 5m10s (x5 over 6m42s) kubelet Pulling image "postgres"
Normal Pulled 5m9s kubelet Successfully pulled image "postgres" in 288.848351ms
Warning BackOff 95s (x25 over 6m38s) kubelet Back-off restarting failed container
Result of `kubectl logs imgress-db-0 -n db`:
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
initdb: error: directory "/var/lib/postgresql/data" exists but is not empty
initdb: detail: It contains a lost+found directory, perhaps due to it being a mount point.
initdb: hint: Using a mount point directly as the data directory is not recommended.
Create a subdirectory under the mount point.
I've already seen some related questions asked on SO but I guess it can be caused by different problems.
**What I tried**:
1. adding user and password as an env variables
2. deleting everything in the namespace and recreating
3. deleting livenessprobe and readinessprobe from the description
</details>
# 答案1
**得分**: 2
我通过在描述中添加两个内容来解决了这个问题：
1. 在容器内部添加`PGDATA`环境变量：
```yaml
env:
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
- 在
volumeMounts下面添加subPath: postgres：
现在我的部署文件看起来是这样的：
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: imgress-db
namespace: db
spec:
serviceName: imgress-db
replicas: 1
selector:
matchLabels:
app: imgress-db
template:
metadata:
labels:
app: imgress-db
spec:
containers:
- name: imgress-db
image: postgres
env:
- name: POSTGRES_HOST
valueFrom:
configMapKeyRef:
name: db-configmap
key: DATABASE_HOST
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_DB
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: POSTGRES_PASSWORD
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
ports:
- containerPort: 5432
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
subPath: postgres
volumes:
- name: postgres-data
persistentVolumeClaim:
claimName: postgres-pvc
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: imgress-db
namespace: db
spec:
selector:
app: imgress-db
ports:
- name: postgres
port: 5432
英文:
I was able to solve the issue by adding two things to description:
-
PGDATAenv variable inside containers:env: - name: PGDATA value: /var/lib/postgresql/data/pgdata -
subPath: postgresundervolumeMounts
Now my deployment file looks like this:
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: imgress-db
namespace: db
spec:
serviceName: imgress-db
replicas: 1
selector:
matchLabels:
app: imgress-db
template:
metadata:
labels:
app: imgress-db
spec:
containers:
- name: imgress-db
image: postgres
env:
- name: POSTGRES_HOST
valueFrom:
configMapKeyRef:
name: db-configmap
key: DATABASE_HOST
- name: POSTGRES_DB
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_DB
- name: POSTGRES_USER
valueFrom:
configMapKeyRef:
name: db-configmap
key: POSTGRES_USER
- name: POSTGRES_PASSWORD
valueFrom:
secretKeyRef:
name: db-secret
key: POSTGRES_PASSWORD
- name: PGDATA
value: /var/lib/postgresql/data/pgdata
ports:
- containerPort: 5432
volumeMounts:
- name: postgres-data
mountPath: /var/lib/postgresql/data
subPath: postgres
volumes:
- name: postgres-data
persistentVolumeClaim:
claimName: postgres-pvc
restartPolicy: Always
---
apiVersion: v1
kind: Service
metadata:
name: imgress-db
namespace: db
spec:
selector:
app: imgress-db
ports:
- name: postgres
port: 5432
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论