Helm在GKE Ingress中添加TLS失败。

huangapple
go评论104阅读模式
英文:

Helm fails to add TLS in the GKE ingress

问题

我正在尝试部署包含Ingress详细信息的应用程序在GKE中。

values.yaml

  1. # Default values for test-frontend.
  2. # This is a YAML-formatted file.
  3. # Declare variables to be passed into your templates.
  5. global:
  6.   namespace: isp
  8. replicaCount: 2
  10. image:
  11.   repository: test.code.com/test/test-frontend
  12.   pullPolicy: Always
  13.   # Overrides the image tag whose default is the chart appVersion.
  14.   tag: ""
  16. imagePullSecrets:
  17.   - name: docker-cfg        
  18. nameOverride: ""
  19. fullnameOverride: "test-frontend"
  21. serviceAccount:
  22.   # Specifies whether a service account should be created
  23.   create: false
  24.   # Annotations to add to the service account
  25.   annotations: {}
  26.   # The name of the service account to use.
  27.   # If not set and create is true, a name is generated using the fullname template
  28.   name: "cloudsql-sa"
  30. podAnnotations: {}
  32. podSecurityContext: {}
  34. securityContext: {}
  36. service:
  37.   name: isp-stabsuite-frontend-service
  38.   annotations:
  39.     cloud.google.com/neg: '{"ingress": true}'
  40.   type: NodePort
  41.   port: 80
  43. resources:
  44.   limits:
  45.     cpu: 350m
  46.     memory: 1000Mi
  47.   requests:
  48.     cpu: 250m
  49.     memory: 750Mi
  51. autoscaling:
  52.   enabled: false
  53.   minReplicas: 1
  54.   maxReplicas: 100
  55.   targetCPUUtilizationPercentage: 80
  57. nodeSelector: {}
  59. tolerations: []
  61. affinity: {}
  63. #########Required Modifications###############
  65. configmap:
  66.   name: test-frontend-configmap
  67.   env:
  68.     GET_HOSTS_FROM: "dns"
  69.     REACT_APP_BACKEND_API: "/api/v1"
  70.     REACT_APP_GOOGLE_CLIENT_ID: ""
  71.     APP_DB_CONNECTION_STRING: ""
  72.     NODE_ENV: "production"
  73. ingress:
  74.   enabled: true
  75.   annotations:
  76.     kubernetes.io/ingress.class: "gce-internal"
  77.     kubernetes.io/ingress.regional-static-ip-name: "ingress-internal-static-ip"
  78.     kubernetes.io/ingress.allow-http: "false"
  79.   hosts:
  80.     - host: test-sandbox.test.com
  81.       paths:
  82.       - path: "/"
  83.         servicename: test-frontend-service
  84.         serviceport: 80
  85.       - path: "/api/"
  86.         servicename: test-backend-service
  87.         serviceport: 80
  88.   tls:
  89.   - hosts:
  90.     - test-sandbox.test.com
  91.     secretName: ingress-tls-credential-file 
  92.     type: kubernetes.io/tls
  94. #########Required Subchart Modifications###
  95. isp-stabsuite-backend:
  96.   cloudsqlConnection: ""
  97.   cloudsqlproxySa: ""
  98.   service:
  99.     name: test-backend-service
  101.   bigquerysecret:
  102.     name: gcp-bigquery-credential-file
  103.     type: Opaque
  104.     content:
  105.       credentials: test

ingress-template file

  1. {{- if .Values.ingress.enabled -}}
  2. apiVersion: networking.k8s.io/v1
  3. kind: Ingress
  4. metadata:
  5.   name: {{ .Values.global.namespace }}-ingress
  6.   namespace: {{ .Values.global.namespace }}
  7.   labels:
  8.     {{- include "test-frontend.labels" . | nindent 4 }}
  9.   annotations:
  10.     kubernetes.io/ingress.class: "gce-internal"
  11.     kubernetes.io/ingress.allow-http: "false"
  12.     kubernetes.io/ingress.global-static-ip-name: "ingress-internal-static-ip" 
  13.     cloud.google.com/neg: '{"ingress": true}'
  14. spec:
  15.   {{- if .Values.ingress.tls }}
  16.   tls:
  17.      {{- range .Values.ingress.tls }}
  18.      - hosts:
  19.          {{- range .hosts }}
  20.          - {{ . | quote }}
  21.          {{- end }}
  22.        secretName: {{ .secretName }}
  23.      {{- end }}
  24.   {{- end }} 
  25.   rules:
  26.     {{- range .Values.ingress.hosts }}
  27.     - host: {{ .host | quote }}
  28.       http:
  29.         paths:
  30.           {{- range .paths }}
  31.           - path: {{ .path }}
  32.             pathType: Prefix
  33.             backend:
  34.               service:
  35.                 name: {{ .servicename }}
  36.                 port:
  37.                   number: {{ .serviceport }}
  38.           {{- end }}
  39.     {{- end }}
  40.   {{- end }}

当尝试创建Ingress时，它会抛出以下错误：

  1. Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic

即使我清理Ingress并尝试安装新的Ingress，它仍然会抛出相同的错误，具有相同的LoadBalancer ID（loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i）。

  1. kubectl describe ingress test-ingress -n test
  2. Name:             test-ingress
  3. Labels:           app.kubernetes.io/managed-by=Helm
  4. app.kubernetes.io/name=test-frontend
  5. app.kubernetes.io/type=frontend
  6. app.kubernetes.io/version=1.16.0
  7. helm/chart=test-frontend-0.1.0
  8. Namespace:        isp
  9. Address:
  10. Ingress Class:    <none>
  11. Default backend:  <default>
  12. Rules:
  13. Host                         Path  Backends
  14. ----                         ----  --------
  15. test-sandbox.test.com
  16. /       test-frontend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
  17. /api/   test-backend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
  18. Annotations:                   cloud.google.com/neg: {"ingress": true}
  19. kubernetes.io/ingress.allow-http: false
  20. kubernetes.io/ingress.class: gce-internal
  21. kubernetes.io/ingress.regional-static-ip-name: ingress-internal-static-ip
  22. meta.helm.sh/release-name: test-frontend
  23. meta.helm.sh/release-namespace: default
  24. Events:
  25. Type     Reason  Age                 From                     Message
  26. ----     ------  ----                ----                     -------
  27. Normal   Sync    113s (x4 over 21m)  loadbalancer-controller  Scheduled for sync
  28. Warning  Sync    24s (x19 over 18m)  loadbalancer-controller  Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic
  29. Helm Chart在ingress.yaml中未能添加TLS详细信息。
  30. <details>
  31. <summary>英文:</summary>
  32. I&#39;m trying to deploy the application in the GKE which contains ingress details
  33. **values.yaml**
  34. # Default values for test-frontend.
  35. # This is a YAML-formatted file.
  36. # Declare variables to be passed into your templates.
  37. global:
  38. namespace: isp
  39. replicaCount: 2
  40. image:
  41. repository: test.code.com/test/test-frontend
  42. pullPolicy: Always
  43. # Overrides the image tag whose default is the chart appVersion.
  44. tag: &quot;&quot;
  45. imagePullSecrets:
  46. - name: docker-cfg        
  47. nameOverride: &quot;&quot;
  48. fullnameOverride: &quot;test-frontend&quot;
  49. serviceAccount:
  50. # Specifies whether a service account should be created
  51. create: false
  52. # Annotations to add to the service account
  53. annotations: {}
  54. # The name of the service account to use.
  55. # If not set and create is true, a name is generated using the fullname template
  56. name: &quot;cloudsql-sa&quot;
  57. podAnnotations: {}
  58. podSecurityContext: {}
  59. # fsGroup: 2000
  60. securityContext: {}
  61. # capabilities:
  62. #   drop:
  63. #   - ALL
  64. # readOnlyRootFilesystem: true
  65. # runAsNonRoot: true
  66. # runAsUser: 1000
  67. service:
  68. name: isp-stabsuite-frontend-service
  69. annotations:
  70. cloud.google.com/neg: &#39;{&quot;ingress&quot;: true}&#39;
  71. type: NodePort
  72. port: 80
  73. resources:
  74. limits:
  75. cpu: 350m
  76. memory: 1000Mi
  77. requests:
  78. cpu: 250m
  79. memory: 750Mi
  80. # We usually recommend not to specify default resources and to leave this as a conscious
  81. # choice for the user. This also increases chances charts run on environments with little
  82. # resources, such as Minikube. If you do want to specify resources, uncomment the following
  83. # lines, adjust them as necessary, and remove the curly braces after &#39;resources:&#39;.
  84. # limits:
  85. #   cpu: 350m
  86. #   memory: 1000Mi
  87. # requests:
  88. #   cpu: 250m
  89. #   memory: 750Mi
  90. autoscaling:
  91. enabled: false
  92. minReplicas: 1
  93. maxReplicas: 100
  94. targetCPUUtilizationPercentage: 80
  95. # targetMemoryUtilizationPercentage: 80
  96. nodeSelector: {}
  97. tolerations: []
  98. affinity: {}
  99. #########Required Modifications###############
  100. configmap:
  101. name: test-frontend-configmap
  102. env:
  103. GET_HOSTS_FROM: &quot;dns&quot;
  104. REACT_APP_BACKEND_API: &quot;/api/v1&quot;
  105. REACT_APP_GOOGLE_CLIENT_ID: &quot;&quot;
  106. APP_DB_CONNECTION_STRING: &quot;&quot;
  107. NODE_ENV: &quot;production&quot;
  108. ingress:
  109. enabled: true
  110. annotations:
  111. kubernetes.io/ingress.class: &quot;gce-internal&quot;
  112. kubernetes.io/ingress.regional-static-ip-name : &quot;ingress-internal-static-ip&quot;
  113. kubernetes.io/ingress.allow-http: &quot;false&quot;
  114. hosts:
  115. - host: test-sandbox.test.com
  116. paths:
  117. - path: &quot;/&quot;
  118. servicename: test-frontend-service
  119. serviceport: 80
  120. - path: &quot;/api/&quot;
  121. servicename: test-backend-service
  122. serviceport: 80
  123. tls:
  124. - hosts:
  125. - test-sandbox.test.com
  126. secretName: ingress-tls-credential-file 
  127. type: kubernetes.io/tls
  128. #########Required Subchart Modifications###
  129. isp-stabsuite-backend:
  130. cloudsqlConnection: &quot;&quot;
  131. cloudsqlproxySa: &quot;&quot;
  132. service:
  133. name: test-backend-service
  134. bigquerysecret:
  135. name: gcp-bigquery-credential-file
  136. type: Opaque
  137. content:
  138. credentials: test
  139. **ingress-template file**
  140. {{- if .Values.ingress.enabled -}}
  141. apiVersion: networking.k8s.io/v1
  142. kind: Ingress
  143. metadata:
  144. name: {{ .Values.global.namespace }}-ingress
  145. namespace: {{ .Values.global.namespace }}
  146. labels:
  147. {{- include &quot;test-frontend.labels&quot; . | nindent 4 }}
  148. annotations:
  149. kubernetes.io/ingress.class: &quot;gce-internal&quot;
  150. kubernetes.io/ingress.allow-http: &quot;false&quot;
  151. kubernetes.io/ingress.global-static-ip-name : &quot;ingress-internal-static-ip&quot; 
  152. cloud.google.com/neg: &#39;{&quot;ingress&quot;: true}&#39;
  153. spec:
  154. {{- if .Values.ingress.tls }}
  155. tls:
  156. {{- range .Values.ingress.tls }}
  157. - hosts:
  158. {{- range .hosts }}
  159. - {{ . | quote }}
  160. {{- end }}
  161. secretName: {{ .secretName }}
  162. {{- end }}
  163. {{- end }} 
  164. rules:
  165. {{- range .Values.ingress.hosts }}
  166. - host: {{ .host | quote }}
  167. http:
  168. paths:
  169. {{- range .paths }}
  170. - path: {{ .path }}
  171. pathType: Prefix
  172. backend:
  173. service:
  174. name: {{ .servicename }}
  175. port:
  176. number: {{ .serviceport }}
  177. {{- end }}
  178. {{- end }}
  179. {{- end }}
  180. When it tries to create ingress, it thows below error
  181. &gt; Error syncing to GCP: error running load balancer syncing routine:
  182. &gt; loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid
  183. &gt; configuration: both HTTP and HTTPS are disabled
  184. &gt; (kubernetes.io/ingress.allow-http is false and there is no valid TLS
  185. &gt; configuration); your Ingress will not be able to serve any traffic
  186. Even I cleanup the ingress and try to install fresh ingress it throws same error with same LoadBalancer ID (loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i)
  187. kubectl describe ingress test-ingress -n test
  188. Name:             test-ingress
  189. Labels:           app.kubernetes.io/managed-by=Helm
  190. app.kubernetes.io/name=test-frontend
  191. app.kubernetes.io/type=frontend
  192. app.kubernetes.io/version=1.16.0
  193. helm/chart=test-frontend-0.1.0
  194. Namespace:        isp
  195. Address:
  196. Ingress Class:    &lt;none&gt;
  197. Default backend:  &lt;default&gt;
  198. Rules:
  199. Host                         Path  Backends
  200. ----                         ----  --------
  201. test-sandbox.test.com
  202. /       test-frontend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
  203. /api/   test-backend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
  204. Annotations:                   cloud.google.com/neg: {&quot;ingress&quot;: true}
  205. kubernetes.io/ingress.allow-http: false
  206. kubernetes.io/ingress.class: gce-internal
  207. kubernetes.io/ingress.regional-static-ip-name: ingress-internal-static-ip
  208. meta.helm.sh/release-name: test-frontend
  209. meta.helm.sh/release-namespace: default
  210. Events:
  211. Type     Reason  Age                 From                     Message
  212. ----     ------  ----                ----                     -------
  213. Normal   Sync    113s (x4 over 21m)  loadbalancer-controller  Scheduled for sync
  214. Warning  Sync    24s (x19 over 18m)  loadbalancer-controller  Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic
  215. Helm Chart fails to add the TLS details in the ingress.yaml. Do I miss something?
  216. </details>
  217. # 答案1
  218. **得分**: 1
  219. 从外观上看，您的 `values.yaml` 错误。 `tls` 块的缩进不正确。它应该与 `enabled`、`annotations` 和 `hosts` 处于相同的级别，即在 `ingress` 之后的第一级。
  220. <details>
  221. <summary>英文:</summary>
  222. From the looks of it, your `values.yaml` is wrong. The `tls` block starts at the wrong indentation. It should be at the same level as the `enabled`, `annotations` and `hosts` i.e. at the first level after `ingress`.
  223. </details>
  224. # 答案2
  225. **得分**: 0
  226. 我看到一个自定义的k8s命名空间，名为`isp`，这个命名空间将无法访问任何其他命名空间中的TLS证书，包括`default`命名空间。您必须在与您的Ingress部署到的相同的k8s命名空间中创建TLS证书。
  227. <details>
  228. <summary>英文:</summary>
  229. I see a custom k8s namespace of `isp`, this namespace will not have access to the TLS cert in any other namespace including the `default` one. You must create the TLS cert in the same k8s namespace your ingress is deployed into.
  230. </details>

huangapple
  • 本文由 发表于 2023年6月29日 00:29:23
  • 转载请务必保留本文链接：https://go.coder-hub.com/76575091.html
  • google-compute-engine
  • google-kubernetes-engine
  • ingress-controller
  • kubernetes-helm
  • kubernetes-ingress
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定