传递一个 ARN 从 ConfigMap 到 Kubernetes 中的 TargetGroupBinding。

huangapple go评论70阅读模式
英文:

Passing an ARN from ConfigMap tp TargetGroupBinding in Kubernetes

问题

我明白你的需求,以下是翻译好的部分:

  1. 我需要在AWS中执行目标组绑定。
  2. 我几乎使用Terraform创建所有资源(EKS集群、节点、其他AWS服务)。
  3. 我还有一个Kubernetes集群,当然还有使用K8s yaml编写的代码。
  4. 我正在创建一个类似于nginx代理的服务,用于执行代理工作。
  5. 我想要实现的目标是将代理服务与Terraform中创建的目标组绑定。
  6. 有一个名为TargetGroupBinding的CRD,我可以用它来实现这个目的,但我需要将TargetGroup ARN传递给它。
  7. 起初,我认为可以使用ConfigMap,但是TargetGroupBinding不理解ConfigMap。
  8. 然后我考虑使用Kustomize,但我无法弄清楚如何(如果可能的话)从包含该值的ConfigMap中传递值,
  9. 然后Kustomize可以使用它并替换TargetGroupBinding中的适当字段。

希望这些翻译对你有所帮助。如果你需要更多信息或有其他问题,请随时提出。

英文:

I need to do a Target Group Binding in AWS.
I am creating almost all resources using Terraform (EKS cluster, nodes, other AWS services).
On top of it I am having a Kubernetes cluster of course and code written in K8s yamls.

I am creating a service like f.e. nginx proxy which is meant to be dong its proxying job.

What I want to achieve is to bind the proxy service

kind: Service
metadata:
  name: nginx-proxy-service
  namespace: nginx-proxy
spec:
  selector:
    app: nginx-proxy
  ports:
    - protocol: TCP
      port: 443
      targetPort: 443

with a target group created in Terraform

resource "aws_lb_target_group" "nginx-proxy" {
  name        = "${var.environment}-proxy-tg"
  port        = 443
  protocol    = "HTTPS"
  vpc_id      = var.vpc_id
  target_type = "ip"
}

There is a CRD TargetGroupBinding that I can use for this purpose, but I need to pass the TargetGroup ARN to it

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: nginx-proxy-tgb
  namespace: nginx-proxy
spec:
  serviceRef:
    name: nginx-proxy-service
    port: 443
  targetGroupARN: $(TARGETGROUP_ARN)

Firstly I thought that I can use a ConfigMap, but that TargetGroupBinding doesn't understand ConfigMaps.
Then I thought that I can use Kustomize, but I cannot figure out how (if it is possible) to pass a value from a ConfigMap that contains that value,

$ kubectl describe configmap proxy-cm
Name:         proxy-cm
Namespace:    nginx-proxy
Labels:       <none>
Annotations:  <none>

Data
====
targetgroup_arn:
----
arn:aws:elasticloadbalancing:eu-west-1:<somevaluehere>:targetgroup/beta-proxy-tg/<somevaluethere>

BinaryData
====

Events:  <none>

to a Kustomize, and then Kustomize can use it and replace proper fields in my TargetGroupBinding.

Do you have any ideas how can I get marry those two things?
Seems pretty common pattern, but as a newbie in Terraform and K8s I cannot figure it out.

答案1

得分: 1

如果我理解问题正确的话,需要在集群中应用一个CRD,其中一个参数的值应该由另一个创建的资源提供。为此,可以使用hashicorp/kubernetes提供程序,特别是其kubernetes_manifest资源。根据问题,配置可能如下所示:

resource "aws_lb_target_group" "nginx-proxy" {
  name        = "${var.environment}-proxy-tg"
  port        = 443
  protocol    = "HTTPS"
  vpc_id      = var.vpc_id
  target_type = "ip"
}

resource "kubernetes_manifest" "target_group_binding_crd" {
  manifest = {
    apiVersion = "elbv2.k8s.aws/v1beta1"
    kind       = "TargetGroupBinding"

    metadata = {
      name      = "nginx-proxy-tgb"
      namespace = "nginx-proxy"
    }

    spec = {
      serviceRef = {
        name = "nginx-proxy-service"
        port = 443
      }
      targetGroupARN = aws_lb_target_group.nginx-proxy.arn
    }
  }
}

其中目标组ARN将通过隐式依赖传递给kubernetes_manifest资源,即 targetGroupARN = aws_lb_target_group.nginx-proxy.arn

英文:

If I understood the question, there is a need to apply a CRD to the cluster where one of the arguments should be populated by the value provided from another resource that gets created. For this purpose, the hashicorp/kubernetes provider can be used, namely its kubernetes_manifest resource. Based on the question it could look like the following:

resource "aws_lb_target_group" "nginx-proxy" {
  name        = "${var.environment}-proxy-tg"
  port        = 443
  protocol    = "HTTPS"
  vpc_id      = var.vpc_id
  target_type = "ip"
}

resource "kubernetes_manifest" "target_group_binding_crd" {
  manifest = {
    apiVersion = "elbv2.k8s.aws/v1beta1"
    kind       = "TargetGroupBinding"

    metadata = {
      name      = "nginx-proxy-tgb"
      namespace = "nginx-proxy"
    }

    spec = {
      serviceRef = {
        name = "nginx-proxy-service"
        port = 443
      }
      targetGroupARN = aws_lb_target_group.nginx-proxy.arn
    }
  }
}

Where the target group ARN would be passed to the kubernetes_manifest resource using implicit dependency, i.e., targetGroupARN = aws_lb_target_group.nginx-proxy.arn.

答案2

得分: 0

I want to use Terraform to create infrastructure obcjets (EKS clusters, nodes, Load Balancers, Target groups and so on).
我想使用 Terraform 来创建基础设施对象(EKS 集群、节点、负载均衡器、目标组等)。

I want to use kubernetes/Kustomize/Helm code to create K8s objects like deployments, services, statefulsets ad so on.
我想使用 Kubernetes/Kustomize/Helm 代码来创建 K8s 对象,比如部署、服务、有状态副本集等等。

I've almost done that using Helm chart approach.
我几乎完成了使用 Helm 图表方法的工作。

Terraform object creation:
Terraform 对象创建:

resource "aws_lb_target_group" "nginx-proxy" {
name = "${var.environment}-proxy-tg"
port = 443
protocol = "HTTPS"
vpc_id = var.vpc_id
target_type = "ip"
}

resource "kubernetes_config_map" "proxy_configmap" {
metadata {
name = "proxy-cm"
namespace = "nginx-proxy"
}
data = {
targetgroup_arn = "${aws_lb_target_group.nginx-proxy.arn}"
lb-name = "${aws_lb.eks_alb.name}"
}
depends_on = [kubernetes_namespace.nginx-proxy-namespace]
}

_helpers.tpl
_helpers.tpl:

{{- define "awsLBTargetGroupArn" -}}
{{- (lookup "v1" "ConfigMap" "nginx-proxy" "proxy-cm").data.targetgroup_arn }}
{{- end -}}

Helm templates:
Helm 模板:

apiVersion: v1
kind: Service
metadata:
name: nginx-proxy-service
namespace: nginx-proxy
spec:
selector:
app: nginx-proxy
ports:
- protocol: TCP
port: 443
targetPort: 443

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: nginx-proxy-tgb
namespace: nginx-proxy
spec:
serviceRef:
name: nginx-proxy-service
port: 443
targetGroupARN: {{ include "awsLBTargetGroupArn" . }}

I can see that kubectl shows me that it is being binded.
我可以看到 kubectl 告诉我它正在绑定。

✗ kubectl describe TargetGroupBinding nginx-proxy-tgb
Name: nginx-proxy-tgb
Namespace: nginx-proxy
Labels: app.kubernetes.io/managed-by=Helm
Annotations: meta.helm.sh/release-name: nginx-proxy
meta.helm.sh/release-namespace: nginx-proxy
API Version: elbv2.k8s.aws/v1beta1
Kind: TargetGroupBinding
(...)

Ip Address Type: ipv4
Service Ref:
Name: nginx-proxy-service
Port: 443
Target Group ARN: arn:aws:elasticloadbalancing:eu-west-1::targetgroup/beta-proxy-tg/bba0b9519459370e
Target Type: ip
Status:
Observed Generation: 1
Events:
Type Reason Age From Message


Normal SuccessfullyReconciled 48m (x3 over 83m) targetGroupBinding Successfully reconciled

But I can't see the nodes being attached in fact (solved and described down the post.).
但事实上我看不到节点被附加(问题已解决并在帖子中描述)。

✗ aws elbv2 describe-target-health --target-group-arn arn:aws:elasticloadbalancing:eu-west-1::targetgroup/beta-proxy-tg/bba0b9519459370e

|DescribeTargetHealth|
+--------------------+

英文:

I want to use Terraform to create infrastructure obcjets (EKS clusters, nodes, Load Balancers, Target groups and so on).
I want to use kubernetes/Kustomize/Helm code to create K8s objects like deployments, services, statefulsets ad so on.
I've almost done that using Helm chart approach

Terraform object creation:

resource "aws_lb_target_group" "nginx-proxy" {
name        = "${var.environment}-proxy-tg"
port        = 443
protocol    = "HTTPS"
vpc_id      = var.vpc_id
target_type = "ip"
}
resource "kubernetes_config_map" "proxy_configmap" {
metadata {
name      = "proxy-cm"
namespace = "nginx-proxy"
}
data = {
targetgroup_arn  = "${aws_lb_target_group.nginx-proxy.arn}"
lb-name = "${aws_lb.eks_alb.name}"
}
depends_on = [kubernetes_namespace.nginx-proxy-namespace]
}

_helpers.tpl

{{- define "awsLBTargetGroupArn" -}}
{{- (lookup "v1" "ConfigMap" "nginx-proxy" "proxy-cm").data.targetgroup_arn }}
{{- end -}}

Helm templates:

apiVersion: v1
kind: Service
metadata:
name: nginx-proxy-service
namespace: nginx-proxy
spec:
selector:
app: nginx-proxy
ports:
- protocol: TCP
port: 443
targetPort: 443
apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
name: nginx-proxy-tgb
namespace: nginx-proxy
spec:
serviceRef:
name: nginx-proxy-service
port: 443
targetGroupARN: {{ include "awsLBTargetGroupArn" . }}

I can see that kubectl shows me that it is being binded

✗ kubectl describe TargetGroupBinding nginx-proxy-tgb
Name:         nginx-proxy-tgb
Namespace:    nginx-proxy
Labels:       app.kubernetes.io/managed-by=Helm
Annotations:  meta.helm.sh/release-name: nginx-proxy
meta.helm.sh/release-namespace: nginx-proxy
API Version:  elbv2.k8s.aws/v1beta1
Kind:         TargetGroupBinding
(...)
Ip Address Type:  ipv4
Service Ref:
Name:            nginx-proxy-service
Port:            443
Target Group ARN:  arn:aws:elasticloadbalancing:eu-west-1:<somenumbershere>:targetgroup/beta-proxy-tg/bba0b9519459370e
Target Type:       ip
Status:
Observed Generation:  1
Events:
Type    Reason                  Age                From                Message
----    ------                  ----               ----                -------
Normal  SuccessfullyReconciled  48m (x3 over 83m)  targetGroupBinding  Successfully reconciled

But I can't see the nodes being attached in fact (solved and described down the post.)

✗ aws elbv2 describe-target-health  --target-group-arn arn:aws:elasticloadbalancing:eu-west-1:<somenumbershere>:targetgroup/beta-proxy-tg/bba0b9519459370e
----------------------
|DescribeTargetHealth|
+--------------------+

答案3

得分: 0

以下是翻译好的内容:

我无法在AWS中找到附件的原因是Pod处于糟糕的状态。当我修复了该Pod后,服务能够连接到它,然后所有的绑定都进行得很顺利。因此,我们可以将上述视为答案。

英文:

the reason why I could not find that attachment in AWS, was the fact that pod was in a bad state. When I fixed that pod, service was able to connect to it and then all the binding went fine. So we can consider above as the answer.

huangapple
  • 本文由 发表于 2023年5月25日 20:06:08
  • 转载请务必保留本文链接:https://go.coder-hub.com/76332086.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定