Helm在GKE Ingress中添加TLS失败。

huangapple
go评论75阅读模式
英文:

Helm fails to add TLS in the GKE ingress

问题

我正在尝试部署包含Ingress详细信息的应用程序在GKE中。

values.yaml

# Default values for test-frontend.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

global:
  namespace: isp

replicaCount: 2

image:
  repository: test.code.com/test/test-frontend
  pullPolicy: Always
  # Overrides the image tag whose default is the chart appVersion.
  tag: ""

imagePullSecrets:
  - name: docker-cfg        
nameOverride: ""
fullnameOverride: "test-frontend"

serviceAccount:
  # Specifies whether a service account should be created
  create: false
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: "cloudsql-sa"

podAnnotations: {}

podSecurityContext: {}

securityContext: {}

service:
  name: isp-stabsuite-frontend-service
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
  type: NodePort
  port: 80

resources:
  limits:
    cpu: 350m
    memory: 1000Mi
  requests:
    cpu: 250m
    memory: 750Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80

nodeSelector: {}

tolerations: []

affinity: {}

#########Required Modifications###############

configmap:
  name: test-frontend-configmap
  env:
    GET_HOSTS_FROM: "dns"
    REACT_APP_BACKEND_API: "/api/v1"
    REACT_APP_GOOGLE_CLIENT_ID: ""
    APP_DB_CONNECTION_STRING: ""
    NODE_ENV: "production"
ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
    kubernetes.io/ingress.regional-static-ip-name: "ingress-internal-static-ip"
    kubernetes.io/ingress.allow-http: "false"
  hosts:
    - host: test-sandbox.test.com
      paths:
      - path: "/"
        servicename: test-frontend-service
        serviceport: 80
      - path: "/api/"
        servicename: test-backend-service
        serviceport: 80
  tls:
  - hosts:
    - test-sandbox.test.com
    secretName: ingress-tls-credential-file 
    type: kubernetes.io/tls

#########Required Subchart Modifications###
isp-stabsuite-backend:
  cloudsqlConnection: ""
  cloudsqlproxySa: ""
  service:
    name: test-backend-service

  bigquerysecret:
    name: gcp-bigquery-credential-file
    type: Opaque
    content:
      credentials: test

ingress-template file

{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ .Values.global.namespace }}-ingress
  namespace: {{ .Values.global.namespace }}
  labels:
    {{- include "test-frontend.labels" . | nindent 4 }}
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.global-static-ip-name: "ingress-internal-static-ip" 
    cloud.google.com/neg: '{"ingress": true}'
spec:
  {{- if .Values.ingress.tls }}
  tls:
     {{- range .Values.ingress.tls }}
     - hosts:
         {{- range .hosts }}
         - {{ . | quote }}
         {{- end }}
       secretName: {{ .secretName }}
     {{- end }}
  {{- end }} 
  rules:
    {{- range .Values.ingress.hosts }}
    - host: {{ .host | quote }}
      http:
        paths:
          {{- range .paths }}
          - path: {{ .path }}
            pathType: Prefix
            backend:
              service:
                name: {{ .servicename }}
                port:
                  number: {{ .serviceport }}
          {{- end }}
    {{- end }}
  {{- end }}

当尝试创建Ingress时，它会抛出以下错误：

Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic

即使我清理Ingress并尝试安装新的Ingress，它仍然会抛出相同的错误，具有相同的LoadBalancer ID（loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i）。

kubectl describe ingress test-ingress -n test
Name:             test-ingress
Labels:           app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=test-frontend
app.kubernetes.io/type=frontend
app.kubernetes.io/version=1.16.0
helm/chart=test-frontend-0.1.0
Namespace:        isp
Address:
Ingress Class:    <none>
Default backend:  <default>
Rules:
Host                         Path  Backends
----                         ----  --------
test-sandbox.test.com
/       test-frontend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
/api/   test-backend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
Annotations:                   cloud.google.com/neg: {"ingress": true}
kubernetes.io/ingress.allow-http: false
kubernetes.io/ingress.class: gce-internal
kubernetes.io/ingress.regional-static-ip-name: ingress-internal-static-ip
meta.helm.sh/release-name: test-frontend
meta.helm.sh/release-namespace: default
Events:
Type     Reason  Age                 From                     Message
----     ------  ----                ----                     -------
Normal   Sync    113s (x4 over 21m)  loadbalancer-controller  Scheduled for sync
Warning  Sync    24s (x19 over 18m)  loadbalancer-controller  Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic
Helm Chart在ingress.yaml中未能添加TLS详细信息。
<details>
<summary>英文:</summary>
I&#39;m trying to deploy the application in the GKE which contains ingress details
**values.yaml**
# Default values for test-frontend.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.
global:
namespace: isp
replicaCount: 2
image:
repository: test.code.com/test/test-frontend
pullPolicy: Always
# Overrides the image tag whose default is the chart appVersion.
tag: &quot;&quot;
imagePullSecrets:
- name: docker-cfg        
nameOverride: &quot;&quot;
fullnameOverride: &quot;test-frontend&quot;
serviceAccount:
# Specifies whether a service account should be created
create: false
# Annotations to add to the service account
annotations: {}
# The name of the service account to use.
# If not set and create is true, a name is generated using the fullname template
name: &quot;cloudsql-sa&quot;
podAnnotations: {}
podSecurityContext: {}
# fsGroup: 2000
securityContext: {}
# capabilities:
#   drop:
#   - ALL
# readOnlyRootFilesystem: true
# runAsNonRoot: true
# runAsUser: 1000
service:
name: isp-stabsuite-frontend-service
annotations:
cloud.google.com/neg: &#39;{&quot;ingress&quot;: true}&#39;
type: NodePort
port: 80
resources:
limits:
cpu: 350m
memory: 1000Mi
requests:
cpu: 250m
memory: 750Mi
# We usually recommend not to specify default resources and to leave this as a conscious
# choice for the user. This also increases chances charts run on environments with little
# resources, such as Minikube. If you do want to specify resources, uncomment the following
# lines, adjust them as necessary, and remove the curly braces after &#39;resources:&#39;.
# limits:
#   cpu: 350m
#   memory: 1000Mi
# requests:
#   cpu: 250m
#   memory: 750Mi
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 100
targetCPUUtilizationPercentage: 80
# targetMemoryUtilizationPercentage: 80
nodeSelector: {}
tolerations: []
affinity: {}
#########Required Modifications###############
configmap:
name: test-frontend-configmap
env:
GET_HOSTS_FROM: &quot;dns&quot;
REACT_APP_BACKEND_API: &quot;/api/v1&quot;
REACT_APP_GOOGLE_CLIENT_ID: &quot;&quot;
APP_DB_CONNECTION_STRING: &quot;&quot;
NODE_ENV: &quot;production&quot;
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: &quot;gce-internal&quot;
kubernetes.io/ingress.regional-static-ip-name : &quot;ingress-internal-static-ip&quot;
kubernetes.io/ingress.allow-http: &quot;false&quot;
hosts:
- host: test-sandbox.test.com
paths:
- path: &quot;/&quot;
servicename: test-frontend-service
serviceport: 80
- path: &quot;/api/&quot;
servicename: test-backend-service
serviceport: 80
tls:
- hosts:
- test-sandbox.test.com
secretName: ingress-tls-credential-file 
type: kubernetes.io/tls
#########Required Subchart Modifications###
isp-stabsuite-backend:
cloudsqlConnection: &quot;&quot;
cloudsqlproxySa: &quot;&quot;
service:
name: test-backend-service
bigquerysecret:
name: gcp-bigquery-credential-file
type: Opaque
content:
credentials: test
**ingress-template file**
{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: {{ .Values.global.namespace }}-ingress
namespace: {{ .Values.global.namespace }}
labels:
{{- include &quot;test-frontend.labels&quot; . | nindent 4 }}
annotations:
kubernetes.io/ingress.class: &quot;gce-internal&quot;
kubernetes.io/ingress.allow-http: &quot;false&quot;
kubernetes.io/ingress.global-static-ip-name : &quot;ingress-internal-static-ip&quot; 
cloud.google.com/neg: &#39;{&quot;ingress&quot;: true}&#39;
spec:
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }} 
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
pathType: Prefix
backend:
service:
name: {{ .servicename }}
port:
number: {{ .serviceport }}
{{- end }}
{{- end }}
{{- end }}
When it tries to create ingress, it thows below error
&gt; Error syncing to GCP: error running load balancer syncing routine:
&gt; loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid
&gt; configuration: both HTTP and HTTPS are disabled
&gt; (kubernetes.io/ingress.allow-http is false and there is no valid TLS
&gt; configuration); your Ingress will not be able to serve any traffic
Even I cleanup the ingress and try to install fresh ingress it throws same error with same LoadBalancer ID (loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i)
kubectl describe ingress test-ingress -n test
Name:             test-ingress
Labels:           app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=test-frontend
app.kubernetes.io/type=frontend
app.kubernetes.io/version=1.16.0
helm/chart=test-frontend-0.1.0
Namespace:        isp
Address:
Ingress Class:    &lt;none&gt;
Default backend:  &lt;default&gt;
Rules:
Host                         Path  Backends
----                         ----  --------
test-sandbox.test.com
/       test-frontend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
/api/   test-backend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
Annotations:                   cloud.google.com/neg: {&quot;ingress&quot;: true}
kubernetes.io/ingress.allow-http: false
kubernetes.io/ingress.class: gce-internal
kubernetes.io/ingress.regional-static-ip-name: ingress-internal-static-ip
meta.helm.sh/release-name: test-frontend
meta.helm.sh/release-namespace: default
Events:
Type     Reason  Age                 From                     Message
----     ------  ----                ----                     -------
Normal   Sync    113s (x4 over 21m)  loadbalancer-controller  Scheduled for sync
Warning  Sync    24s (x19 over 18m)  loadbalancer-controller  Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic
Helm Chart fails to add the TLS details in the ingress.yaml. Do I miss something?
</details>
# 答案1
**得分**: 1
从外观上看，您的 `values.yaml` 错误。 `tls` 块的缩进不正确。它应该与 `enabled`、`annotations` 和 `hosts` 处于相同的级别，即在 `ingress` 之后的第一级。
<details>
<summary>英文:</summary>
From the looks of it, your `values.yaml` is wrong. The `tls` block starts at the wrong indentation. It should be at the same level as the `enabled`, `annotations` and `hosts` i.e. at the first level after `ingress`.
</details>
# 答案2
**得分**: 0
我看到一个自定义的k8s命名空间，名为`isp`，这个命名空间将无法访问任何其他命名空间中的TLS证书，包括`default`命名空间。您必须在与您的Ingress部署到的相同的k8s命名空间中创建TLS证书。
<details>
<summary>英文:</summary>
I see a custom k8s namespace of `isp`, this namespace will not have access to the TLS cert in any other namespace including the `default` one. You must create the TLS cert in the same k8s namespace your ingress is deployed into.
</details>

huangapple
  • 本文由 发表于 2023年6月29日 00:29:23
  • 转载请务必保留本文链接：https://go.coder-hub.com/76575091.html
  • google-compute-engine
  • google-kubernetes-engine
  • ingress-controller
  • kubernetes-helm
  • kubernetes-ingress
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定