如何使用Golang仅在一段时间内并行收集Kubernetes（k8s）的Pod日志

huangapple
go评论128阅读模式
英文:

How to collect k8s pods logs in parallel using golang only for a duration of time

问题

我是你的中文翻译助手，以下是翻译好的内容：

我是golang的新手，我有一个任务是收集应用程序日志，该应用程序作为部署在k8s集群中运行，总共有4个pod。

作为测试自动化的一部分，我需要在执行一些操作并将日志写入文件的同时，以并行方式收集应用程序日志（仅在我的测试操作期间）。然后进行下一个操作并执行相同的操作。

最后，我逐个迭代日志文件，并根据我的操作筛选出特定关键字，并验证日志是否正确。

我考虑直接使用kubectl命令获取pod日志，而不是使用go-sdk，因为我在尝试多次后仍然无法解决丢失的日志条目。

  1. kubectl logs -f -l app=my-app -n dev > /usr/mylogs.txt

我找到了一种使用exec.Command运行此命令的方法

  1. command := exec.Command("/bin/sh", "-c", "kubectl logs -f -l app=my-app -n dev > /usr/mylogs.txt")
  2. err := command.Start()

现在我需要在golang中实现这个功能

  1. func myTest(t *testing.T){
  2.   go collectApplicationLogs("test1")
  3.   
  4.   // 执行应用程序操作
  5.   // 运行测试
  6.   
  7.   stopLogsCollection ()   -------> 如何实现这一点？
  8. }
  10. func collectApplicationLogs(fileName string){
  11.    // 收集日志到文件的命令
  12.   // kubectl logs -f -l app=my-app -n dev > /usr/{fileName}
  13. }
英文:

I am new to golang, I have a task to collect the application logs, application is running as deployment in k8s cluster, there are 4 pods in total.
As part of test automation, I need to collect the application logs (only for the duration of my test operation) in parallel while I perform some operations on the application and write the logs to a file, and move to the next operation and do the same.

Finally I iterate through the log files one-by-one and filter for certain keywords corresponding to my operation and validate the logs are correct/not.

I am thinking to get the pod logs using kubectl command directly, instead of using the go-sdk as I am facing missing log entries which I couldn't triage with many attempts.

  1. kubectl logs -f -l app=my-app -n dev > /usr/mylogs.txt

I found a way to run this command using exec.Command

  1. command := exec.Command("/bin/sh", "-c", "kubectl logs -f -l app=my-app -n dev > /usr/mylogs.txt")
  2. 	err := command.Start()

Now I need to do this in golang,

  1. func myTest(t *testing.T){
  2.   go collectApplicationLogs("test1")
  3.   
  4.   // do application operation 
  5.    // run test
  6.   
  7.   stopLogsCollection ()   -------> how to achieve this?
  8. }
  10. func collectApplicationLogs(fileName string){
  11.    // command to collect the logs to a file
  12.   // kubectl logs -f -l app=my-app -n dev > /usr/{fileName}
  13. }

答案1

得分: 4

你可以使用Kubernetes go-client从Pod中获取日志。首先从Kubernetes配置中创建clientset。你可以使用InclusterConfig或Out of cluster config。这里我使用的是Out of cluster config。

  1. const (
  2. 	// 设置命名空间和标签
  3. 	namespace = "dev"
  4. 	label     = "app=my-app"
  5. )
  7. func main() {
  8. 	// 解析.kubeconfig文件
  9. 	var kubeconfig *string
  10. 	if home := homedir.HomeDir(); home != "" {
  11. 		kubeconfig = flag.String("kubeconfig", filepath.Join(home, ".kube", "config"), "(optional) absolute path to the kubeconfig file")
  12. 	} else {
  13. 		kubeconfig = flag.String("kubeconfig", "", "absolute path to the kubeconfig file")
  14. 	}
  15. 	flag.Parse()
  17. 	// 使用当前上下文的kubeconfig
  18. 	ctx := context.TODO()
  19. 	config, err := clientcmd.BuildConfigFromFlags("", *kubeconfig)
  20. 	if err != nil {
  21. 		log.Println(err, "Failed to build config from flags")
  22. 		return
  23. 	}
  25. 	err = collectApplicationLogs(ctx, config, "/usr/mylogs.txt")
  26. 	if err != nil {
  27. 		log.Println(err, "Failed to collect logs")
  28. 	}
  30. }

要收集日志，首先需要列出Pods。然后并发地从每个Pod获取日志，并将其追加到文件中。

  1. func collectApplicationLogs(ctx context.Context, config *rest.Config, filename string) error {
  2. 	// 创建clientset
  3. 	clientSet, err := kubernetes.NewForConfig(config)
  4. 	if err != nil {
  5. 		log.Println("Failed to create clientset from the given config")
  6. 		return err
  7. 	}
  8. 	// 获取Pods作为ListItems
  9. 	pods, err := clientSet.CoreV1().Pods(namespace).List(ctx, metav1.ListOptions{
  10. 		LabelSelector: label,
  11. 	})
  12. 	if err != nil {
  13. 		log.Println(err, "Failed to get pods")
  14. 		return err
  15. 	}
  16. 	// 如果文件不存在，则创建文件或追加到文件
  17. 	file, err := os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0666)
  18. 	if err != nil {
  19. 		return err
  20. 	}
  21. 	defer file.Close()
  22. 	// 首先获取Pod列表
  23. 	// 然后从每个Pod获取PodLogs
  24. 	// 并发地写入文件
  25. 	// 使用通道进行阻塞
  26. 	ch := make(chan bool)
  27. 	podItems := pods.Items
  28. 	for i := 0; i < len(podItems); i++ {
  29. 		podLogs, err := clientSet.CoreV1().Pods(namespace).GetLogs(podItems[i].Name, &v1.PodLogOptions{
  30. 			Follow: true,
  31. 		}).Stream(ctx)
  32. 		if err != nil {
  33. 			return err
  34. 		}
  35. 		buffer := bufio.NewReader(podLogs)
  36. 		go writeLogs(buffer, file, ch)
  37. 	}
  38. 	<-ch
  39. 	return nil
  40. }
  42. func writeLogs(buffer *bufio.Reader, file *os.File, ch chan bool) {
  43. 	defer func() {
  44. 		ch <- true
  45. 	}()
  46. 	for {
  47. 		str, readErr := buffer.ReadString('\n')
  48. 		if readErr == io.EOF {
  49. 			break
  50. 		}
  51. 		_, err := file.Write([]byte(str))
  52. 		if err != nil {
  53. 			return
  54. 		}
  55. 	}
  56. }
英文:

You can use Kubernetes go-client to get logs from the pods. At first create the clientset from kubenetes config. You can use InclusterConfig or Out of cluster config. I have used out of cluster config here.

  1. const (
  2. // set namespace and label
  3. namespace = &quot;dev&quot;
  4. label     = &quot;app=my-app&quot;
  5. )
  6. func main() {
  7. // parse the .kubeconfig file
  8. var kubeconfig *string
  9. if home := homedir.HomeDir(); home != &quot;&quot; {
  10. kubeconfig = flag.String(&quot;kubeconfig&quot;, filepath.Join(home, &quot;.kube&quot;, &quot;config&quot;), &quot;(optional) absolute path to the kubeconfig file&quot;)
  11. } else {
  12. kubeconfig = flag.String(&quot;kubeconfig&quot;, &quot;&quot;, &quot;absolute path to the kubeconfig file&quot;)
  13. }
  14. flag.Parse()
  15. // use the current context in kubeconfig
  16. ctx := context.TODO()
  17. config, err := clientcmd.BuildConfigFromFlags(&quot;&quot;, *kubeconfig)
  18. if err != nil {
  19. log.Println(err, &quot;Failed to build config from flags&quot;)
  20. return
  21. }
  22. err = collectApplicationLogs(ctx, config, &quot;/usr/mylogs.txt&quot;)
  23. if err != nil {
  24. log.Println(err, &quot;Failed to collect logs&quot;)
  25. }
  26. }

To collect logs, you need to list the gods first. Then get logs from each of those pods and append them to the file concurrently.

  1. func collectApplicationLogs(ctx context.Context, config *rest.Config, filename string) error {
  2. // create the clientset
  3. clientSet, err := kubernetes.NewForConfig(config)
  4. if err != nil {
  5. log.Println(&quot;Failed to create clientset from the given config&quot;)
  6. return err
  7. }
  8. // get the pods as ListItems
  9. pods, err := clientSet.CoreV1().Pods(namespace).List(ctx, metav1.ListOptions{
  10. LabelSelector: label,
  11. })
  12. if err != nil {
  13. log.Println(err, &quot;Failed to get pods&quot;)
  14. return err
  15. }
  16. // If the file doesn&#39;t exist, create it or append to the file
  17. file, err := os.OpenFile(filename, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0666)
  18. if err != nil {
  19. return err
  20. }
  21. defer file.Close()
  22. // get the pod lists first
  23. // then get the podLogs from each of the pods
  24. // write to files concurrently
  25. // use channel for blocking reasons
  26. ch := make(chan bool)
  27. podItems := pods.Items
  28. for i := 0; i &lt; len(podItems); i++ {
  29. podLogs, err := clientSet.CoreV1().Pods(namespace).GetLogs(podItems[i].Name, &amp;v1.PodLogOptions{
  30. Follow: true,
  31. }).Stream(ctx)
  32. if err != nil {
  33. return err
  34. }
  35. buffer := bufio.NewReader(podLogs)
  36. go writeLogs(buffer, file, ch)
  37. }
  38. &lt;-ch
  39. return nil
  40. }
  41. func writeLogs(buffer *bufio.Reader, file *os.File, ch chan bool) {
  42. defer func() {
  43. ch &lt;- true
  44. }()
  45. for {
  46. str, readErr := buffer.ReadString(&#39;\n&#39;)
  47. if readErr == io.EOF {
  48. break
  49. }
  50. _, err := file.Write([]byte(str))
  51. if err != nil {
  52. return
  53. }
  54. }
  55. }

huangapple
  • 本文由 发表于 2022年11月18日 13:43:26
  • 转载请务必保留本文链接：https://go.coder-hub.com/74485370.html
  • automation
  • go
  • go-testing
  • goroutine
  • testing
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定