为使用jenkinsci kubernetes-plugin创建的Jenkins从节点配置一个旁路容器。

huangapple
go评论98阅读模式
英文:

Configure a side car container for Jenkins slave created with jenkinsci kubernetes-plugin

问题

我有这个YAML文件，用于配置我的Jenkins从节点容器在K8s上通过插件：

  1. clouds:
  2.   - kubernetes:
  3.       ...
  4.       name: "kubernetes"
  5.       templates:
  6.       - name: jenkins-slave
  7.         containers:
  8.           - image: "..."
  9.             name: "jnlp"
  10.             resourceLimitCpu: "1000m"
  11.             resourceLimitMemory: "4000Mi"
  12.             ...

如何配置另一个Sidecar容器？我想添加buildkit。像这样添加它不起作用，插件会忽略这个configMap配置：

  1. clouds:
  2.   - kubernetes:
  3.       ...
  4.       name: "kubernetes"
  5.       templates:
  6.         - name: jenkins-slave
  7.           containers:
  8.             - image: "..."
  9.               name: "jnlp"
  10.               resourceLimitCpu: "1000m"
  11.               resourceLimitMemory: "4000Mi"
  12.               ...
  13.             - image: "moby/buildkit"
  14.               name: "buildkit"

我看到在文档中，我可以配置类似additionalContainers的东西，但我不知道如何设置它。

英文:

I have this yaml file that configures my Jenkins slave container on k8s via the plugin:

  1. clouds:
  2.   - kubernetes:
  3.       ...
  4.       name: "kubernetes"
  5.       templates:
  6.       - name: jenkins-slave
  7.         containers:
  8.           - image: "..."
  9.             name: "jnlp"
  10.             resourceLimitCpu: "1000m"
  11.             resourceLimitMemory: "4000Mi"
  12.             ...

How can I configure another sidecar container? I want to add buildkit. Adding it like this doesn't work, the plugin just ignores this configMap configuration:

  1. clouds:
  2.   - kubernetes:
  3.       ...
  4.       name: "kubernetes"
  5.       templates:
  6.       - name: jenkins-slave
  7.         containers:
  8.         - image: "..."
  9.           name: "jnlp"
  10.           resourceLimitCpu: "1000m"
  11.           resourceLimitMemory: "4000Mi"
  12.           ...
  13.         - image: "moby/buildkit"
  14.           name: "buildkit"

I saw that in the docs I can configure something like additionalContainers but I can't understand how I can set it.

答案1

得分: 0

如果您使用CASC插件配置您的代理，

此示例可以帮助您：

  1. jenkins:
  2.   clouds:
  3.     - kubernetes:
  4.         name: "advanced-k8s-config"
  5.         serverUrl: "https://advanced-k8s-config:443"
  6.         serverCertificate: "serverCertificate"
  7.         skipTlsVerify: true
  8.         credentialsId: "advanced-k8s-credentials"
  9.         namespace: "default"
  10.         jenkinsUrl: "http://jenkins/"
  11.         jenkinsTunnel: "jenkinsTunnel"
  12.         containerCapStr: 42
  13.         maxRequestsPerHostStr: 64
  14.         retentionTimeout: 5
  15.         connectTimeout: 10
  16.         readTimeout: 20
  18.         templates:
  19.           - name: "test"
  20.             serviceAccount: "serviceAccount"
  21.             instanceCap: 1234
  22.             idleMinutes: 0
  23.             label: "label"
  24.             # 是否在每个构建日志中显示POD Yaml，默认为`true`。
  25.             showRawYaml: true
  26.             
  27.             volumes:
  28.               - hostPathVolume:
  29.                   mountPath: "mountPath"
  30.                   hostPath: "hostPath"
  32.             containers:
  33.               - name: "name"
  34.                 image: "image"
  35.                 privileged: true
  36.                 alwaysPullImage: true
  37.                 command: "command"
  38.                 args: "args"
  39.                 workingDir: "workingDir"
  40.                 ttyEnabled: true
  41.                 resourceRequestCpu: "resourceRequestCpu"
  42.                 resourceRequestMemory: "resourceRequestMemory"
  43.                 resourceLimitCpu: "resourceLimitCpu"
  44.                 resourceLimitMemory: "resourceLimitMemory"
  45.             imagePullSecrets:
  46.               - name: "imagePullSecrets"
  48.             envVars:
  49.               - envVar:
  50.                   key: "FOO"
  51.                   value: "BAR"
  53.           - name: "k8s-agent"
  54.             namespace: "default"
  55.             label: "linux-x86_64"
  56.             nodeUsageMode: EXCLUSIVE
  57.             containers:
  58.               - name: "jnlp"
  59.                 image: "jenkins/inbound-agent:latest"
  60.                 alwaysPullImage: true
  61.                 workingDir: "/home/jenkins"
  62.                 ttyEnabled: true
  63.                 resourceRequestCpu: "500m"
  64.                 resourceLimitCpu: "1000m"
  65.                 resourceRequestMemory: "1Gi"
  66.                 resourceLimitMemory: "2Gi"
  67.             volumes:
  68.               - emptyDirVolume:
  69.                   memory: false
  70.                   mountPath: "/tmp"
  71.               # 使用ConfigMap `configmap-name` 的数据 `config` 挂载内容。
  72.               - configMapVolume:
  73.                   configMapName: configmap-name
  74.                   mountPath: /home/jenkins/.aws/config
  75.                   subPath: config
  76.             idleMinutes: "1"
  77.             activeDeadlineSeconds: "120"
  78.             slaveConnectTimeout: "1000"
  80. 也可以在此处查看文档：[https://plugins.jenkins.io/kubernetes/#plugin-content-pod-template](https://plugins.jenkins.io/kubernetes/#plugin-content-pod-template)
  82. PS：某些名称与K8s的名称不同，插件会封装所有内容，并且所有代理都应该有 "jnlp" 容器，它由Jenkins自身使用并且其中有Java客户端。您可以运行任何您想要的代理并在流水线中切换容器。
  84. 在模板示例中有2个容器：
  86. ```yaml
  87. jenkins:
  88.   clouds:
  89.     - kubernetes:
  90.         name: "advanced-k8s-config"
  91.         serverUrl: "https://advanced-k8s-config:443"
  92.         serverCertificate: "serverCertificate"
  93.         skipTlsVerify: true
  94.         credentialsId: "advanced-k8s-credentials"
  95.         namespace: "default"
  96.         jenkinsUrl: "http://jenkins/"
  97.         jenkinsTunnel: "jenkinsTunnel"
  98.         containerCapStr: 42
  99.         maxRequestsPerHostStr: 64
  100.         retentionTimeout: 5
  101.         connectTimeout: 10
  102.         readTimeout: 20
  103.         templates:
  104.           - name: "templatename"
  105.             label: "label"
  106.             showRawYaml: true
  107.             containers:
  108.               - name: "name1"
  109.                 command: "cat"
  110.                 args: ''
  111.                 alwaysPullImage: true
  112.                 image: "anyrandomimage"
  113.                 workingDir: '/home/jenkins/agent'
  114.                 ttyEnabled: true
  115.               - name: "name2"
  116.                 command: "cat"
  117.                 args: ''
  118.                 alwaysPullImage: true
  119.                 image: "anyrandomimage2"
  120.                 workingDir: '/home/jenkins/agent'
  121.                 ttyEnabled: true

流水线示例：

  1. pipeline {
  2.   agent {
  3.     kubernetes {
  4.       cloud 'YOUR PREDEFINED CLOUD NAME FROM PLUGIN'
  5.       defaultContainer 'jnlp'
  6.       yaml """apiVersion: v1
  7. kind: Pod
  8. metadata:
  9.   labels:
  10.     app: myapp
  11. spec:
  12.   serviceAccountName: k8s-agent
  13.   containers:
  14.     - name: maven
  15.       image: maven:latest
  16.       command:
  17.       - cat
  18.       tty: true
  19.     - name: docker
  20.       image: docker:latest
  21.       command:
  22.       - cat
  23.       tty: true
  24.       volumeMounts:
  25.         - mountPath: /var/run/docker.sock
  26.           name: docker-sock
  27.   volumes:
  28.     - name: docker-sock
  29.       hostPath:
  30.         path: /var/run/docker.sock
  31. """
  32.     }
  33.   }
  34.   stages {
  36.     stage ('build java') {
  37.       steps {
  38.         container('maven') {
  39.           sh 'mvn clean install'
  40.         }
  41.       }
  42.     }
  43.     stage ('build docker image'){
  44.       steps {
  45.         container('docker') {
  46.           sh 'docker build -t image:v1 .'
  47.         }
  48.       }
  49.     }
  50.   }
  51. }

文档链接

英文:

if you use CASC plugin to configure you agents

this example can help you

  1. jenkins:
  2. clouds:
  3. - kubernetes:
  4. name: "advanced-k8s-config"
  5. serverUrl: "https://advanced-k8s-config:443"
  6. serverCertificate: "serverCertificate"
  7. skipTlsVerify: true
  8. credentialsId: "advanced-k8s-credentials"
  9. namespace: "default"
  10. jenkinsUrl: "http://jenkins/"
  11. jenkinsTunnel: "jenkinsTunnel"
  12. containerCapStr: 42
  13. maxRequestsPerHostStr: 64
  14. retentionTimeout: 5
  15. connectTimeout: 10
  16. readTimeout: 20
  17. templates:
  18. - name: "test"
  19. serviceAccount: "serviceAccount"
  20. instanceCap: 1234
  21. idleMinutes: 0
  22. label: "label"
  23. # Enable whether the POD Yaml is displayed in each build log or not, `true` by default.
  24. showRawYaml: true
  25. volumes:
  26. - hostPathVolume:
  27. mountPath: "mountPath"
  28. hostPath: "hostPath"
  29. containers:
  30. - name: "name"
  31. image: "image"
  32. privileged: true
  33. alwaysPullImage: true
  34. command: "command"
  35. args: "args"
  36. workingDir: "workingDir"
  37. ttyEnabled: true
  38. resourceRequestCpu: "resourceRequestCpu"
  39. resourceRequestMemory: "resourceRequestMemory"
  40. resourceLimitCpu: "resourceLimitCpu"
  41. resourceLimitMemory: "resourceLimitMemory"
  42. imagePullSecrets:
  43. - name: "imagePullSecrets"
  44. envVars:
  45. - envVar:
  46. key: "FOO"
  47. value: "BAR"
  48. - name: "k8s-agent"
  49. namespace: "default"
  50. label: "linux-x86_64"
  51. nodeUsageMode: EXCLUSIVE
  52. containers:
  53. - name: "jnlp"
  54. image: "jenkins/inbound-agent:latest"
  55. alwaysPullImage: true
  56. workingDir: "/home/jenkins"
  57. ttyEnabled: true
  58. resourceRequestCpu: "500m"
  59. resourceLimitCpu: "1000m"
  60. resourceRequestMemory: "1Gi"
  61. resourceLimitMemory: "2Gi"
  62. volumes:
  63. - emptyDirVolume:
  64. memory: false
  65. mountPath: "/tmp"
  66. # Mount the content of the ConfigMap `configmap-name` with the data `config`.
  67. - configMapVolume:
  68. configMapName: configmap-name
  69. mountPath: /home/jenkins/.aws/config
  70. subPath: config
  71. idleMinutes: "1"
  72. activeDeadlineSeconds: "120"
  73. slaveConnectTimeout: "1000"

also you can check documentation here https://plugins.jenkins.io/kubernetes/#plugin-content-pod-template

ps some names differs to what K8s has, plugin wraps everything

and all agent should have "jnlp" container - it used by jenkins itself and has java client on it

you can run any agent you want and switch container in pipeline documentation

  1. container('your-second-container-name') {
  2. sh 'hostname'
  3. }

2 containers in template example:

  1. jenkins:
  2. clouds:
  3. - kubernetes:
  4. name: "advanced-k8s-config"
  5. serverUrl: "https://advanced-k8s-config:443"
  6. serverCertificate: "serverCertificate"
  7. skipTlsVerify: true
  8. credentialsId: "advanced-k8s-credentials"
  9. namespace: "default"
  10. jenkinsUrl: "http://jenkins/"
  11. jenkinsTunnel: "jenkinsTunnel"
  12. containerCapStr: 42
  13. maxRequestsPerHostStr: 64
  14. retentionTimeout: 5
  15. connectTimeout: 10
  16. readTimeout: 20
  17. templates:
  18. - name: "templatename"
  19. label: "label"
  20. showRawYaml: true
  21. containers:
  22. - name: "name1"
  23. command: "cat"
  24. args: ''
  25. alwaysPullImage: true
  26. image: "anyrandomimage"
  27. workingDir: '/home/jenkins/agent'
  28. ttyEnabled: true
  29. - name: "name2"
  30. command: "cat"
  31. args: ''
  32. alwaysPullImage: true
  33. image: "anyrandomimage2"
  34. workingDir: '/home/jenkins/agent'
  35. ttyEnabled: true

pipeline example:

  1. pipeline {
  2. agent {
  3. kubernetes {
  4. cloud 'YOUR PREDEFINED CLOUD NAME FROM PLUGIN'
  5. defaultContainer 'jnlp'
  6. yaml """
  7. apiVersion: v1
  8. kind: Pod
  9. metadata:
  10. labels:
  11. app: myapp
  12. spec:
  13. serviceAccountName: k8s-agent
  14. containers:
  15. - name: maven
  16. image: maven:latest
  17. command:
  18. - cat
  19. tty: true
  20. - name: docker
  21. image: docker:latest
  22. command:
  23. - cat
  24. tty: true
  25. volumeMounts:
  26. - mountPath: /var/run/docker.sock
  27. name: docker-sock
  28. volumes:
  29. - name: docker-sock
  30. hostPath:
  31. path: /var/run/docker.sock
  32. """
  33. }
  34. }
  35. stages {
  36. stage ('build java') {
  37. steps {
  38. container('maven') {
  39. sh 'mvn clean install'
  40. }
  41. }
  42. }
  43. stage ('build docker image'){
  44. steps {
  45. container('docker') {
  46. sh 'docker build -t image:v1 .'
  47. }
  48. }
  49. }
  50. }
  51. }

huangapple
  • 本文由 发表于 2023年7月27日 21:38:23
  • 转载请务必保留本文链接：https://go.coder-hub.com/76780338.html
  • jenkins
  • jenkins-plugins
  • kubernetes
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定