为什么 golang 的 CronJob 不能在 goroutine 中运行?

huangapple go评论133阅读模式
英文:

Why golang CronJob cannot run within goroutine?

问题

我每天使用CronJob来启动我的任务,我的任务有几个子任务,我计划使用goroutine来运行。然而,事情并不顺利。

文件框架

  1. |-gpool
  2. | -pool.go
  3. |-main.go

main.go

  1. import (
  2. "code.byted.org/i18n_web/content_import_tool_cronJob/gpool"
  3. "fmt"
  4. "github.com/robfig/cron/v3"
  5. "log"
  6. "os"
  7. "runtime"
  8. "time"
  9. )
  10. func SeedJob(it string, pool *gpool.Pool){
  11. fmt.Println("Name item: ", it)
  12. println(runtime.NumGoroutine())
  13. pool.Done()
  14. }
  15. type delayJob struct {
  16. PagePatternNameList []string
  17. }
  18. func (j *delayJob) GetPagePatternNameList() {
  19. //j.PagePatternNameList = dal.GetPagePatternName()
  20. j.PagePatternNameList = []string{"atama_posts","cchan_posts", "cookdoor_posts", "cookpad_posts",
  21. "cookpad_recipe_seed", "kurashiru_posts", "lips_all_posts", "lips_product", "lips_product_sku_seed",
  22. "lips_rank", "press_posts", "voce_all_posts", "zozo_posts_women"}
  23. }
  24. func (j *delayJob)Run(){
  25. log.Println("delay Job RUN")
  26. //time.Sleep(2 * time.Second)
  27. // startSeedJob
  28. pool := gpool.New(10)
  29. println(runtime.NumGoroutine())
  30. for _, it := range j.PagePatternNameList {
  31. pool.Add(1)
  32. go SeedJob(it, pool)
  33. }
  34. pool.Wait()
  35. println(runtime.NumGoroutine())
  36. }
  37. func main() {
  38. c := cron.New(
  39. cron.WithLogger(
  40. cron.VerbosePrintfLogger(log.New(os.Stdout, "cron: ", log.LstdFlags))))
  41. _, err := c.AddJob("CRON_TZ=America/New_York @every 2m", cron.NewChain(cron.DelayIfStillRunning(cron.DefaultLogger)).Then(&delayJob{}))
  42. if err != nil {
  43. fmt.Println("Cron Job err!")
  44. return
  45. }
  46. fmt.Println("it started")
  47. c.Start()
  48. defer c.Stop()
  49. time.Sleep(time.Second * 5)
  50. }

pool.go

  1. package gpool
  2. import (
  3. "sync"
  4. )
  5. type Pool struct {
  6. queue chan int
  7. wg *sync.WaitGroup
  8. }
  9. func New(size int) *Pool {
  10. if size <= 0 {
  11. size = 1
  12. }
  13. return &Pool{
  14. queue: make(chan int, size),
  15. wg: &sync.WaitGroup{},
  16. }
  17. }
  18. func (p *Pool) Add(delta int) {
  19. for i := 0; i < delta; i++ {
  20. p.queue <- 1
  21. }
  22. for i := 0; i > delta; i-- {
  23. <-p.queue
  24. }
  25. p.wg.Add(delta)
  26. }
  27. func (p *Pool) Done() {
  28. <-p.queue
  29. p.wg.Done()
  30. }
  31. func (p *Pool) Wait() {
  32. p.wg.Wait()
  33. }

当我运行我的代码时,它没有打印出我期望的任何信息,只有

  1. it started
  2. cron: 2021/06/10 13:41:21 start
  3. cron: 2021/06/10 13:41:21 schedule, now=2021-06-10T13:41:21+08:00, entry=1, next=2021-06-10T13:43:21+08:00
  4. Exiting.
  5. Debugger finished with the exit code 0

看起来它没有触发错误,但没有正确运行。我该如何解决这个问题?

英文:

I use CronJob for starting my task everyday, and my task has several sub-tasks which I plan to use goroutine to run. However, things do not go smoothly.

File Framework

  1. |-gpool
  2. | -pool.go
  3. |-main.go

main.go

  1. import (
  2. &quot;code.byted.org/i18n_web/content_import_tool_cronJob/gpool&quot;
  3. &quot;fmt&quot;
  4. &quot;github.com/robfig/cron/v3&quot;
  5. &quot;log&quot;
  6. &quot;os&quot;
  7. &quot;runtime&quot;
  8. &quot;time&quot;
  9. )
  10. func SeedJob(it string, pool *gpool.Pool){
  11. fmt.Println(&quot;Name item: &quot;, it)
  12. println(runtime.NumGoroutine())
  13. pool.Done()
  14. }
  15. type delayJob struct {
  16. PagePatternNameList []string
  17. }
  18. func (j *delayJob) GetPagePatternNameList() {
  19. //j.PagePatternNameList = dal.GetPagePatternName()
  20. j.PagePatternNameList = []string{&quot;atama_posts&quot;,&quot;cchan_posts&quot;, &quot;cookdoor_posts&quot;, &quot;cookpad_posts&quot;,
  21. &quot;cookpad_recipe_seed&quot;, &quot;kurashiru_posts&quot;, &quot;lips_all_posts&quot;, &quot;lips_product&quot;, &quot;lips_product_sku_seed&quot;,
  22. &quot;lips_rank&quot;, &quot;press_posts&quot;, &quot;voce_all_posts&quot;, &quot;zozo_posts_women&quot;}
  23. }
  24. func (j *delayJob)Run(){
  25. log.Println(&quot;delay Job RUN&quot;)
  26. //time.Sleep(2 * time.Second)
  27. // startSeedJob
  28. pool := gpool.New(10)
  29. println(runtime.NumGoroutine())
  30. for _, it := range j.PagePatternNameList {
  31. pool.Add(1)
  32. go SeedJob(it, pool)
  33. }
  34. pool.Wait()
  35. println(runtime.NumGoroutine())
  36. }
  37. func main() {
  38. c := cron.New(
  39. cron.WithLogger(
  40. cron.VerbosePrintfLogger(log.New(os.Stdout, &quot;cron: &quot;, log.LstdFlags))))
  41. _, err := c.AddJob(&quot;CRON_TZ=America/New_York @every 2m&quot;, cron.NewChain(cron.DelayIfStillRunning(cron.DefaultLogger)).Then(&amp;delayJob{}))
  42. if err != nil {
  43. fmt.Println(&quot;Cron Job err!&quot;)
  44. return
  45. }
  46. fmt.Println(&quot;it started&quot;)
  47. c.Start()
  48. defer c.Stop()
  49. time.Sleep(time.Second * 5)
  50. }

pool.go

  1. package gpool
  2. import (
  3. &quot;sync&quot;
  4. )
  5. type Pool struct {
  6. queue chan int
  7. wg *sync.WaitGroup
  8. }
  9. func New(size int) *Pool {
  10. if size &lt;= 0 {
  11. size = 1
  12. }
  13. return &amp;Pool{
  14. queue: make(chan int, size),
  15. wg: &amp;sync.WaitGroup{},
  16. }
  17. }
  18. func (p *Pool) Add(delta int) {
  19. for i := 0; i &lt; delta; i++ {
  20. p.queue &lt;- 1
  21. }
  22. for i := 0; i &gt; delta; i-- {
  23. &lt;-p.queue
  24. }
  25. p.wg.Add(delta)
  26. }
  27. func (p *Pool) Done() {
  28. &lt;-p.queue
  29. p.wg.Done()
  30. }
  31. func (p *Pool) Wait() {
  32. p.wg.Wait()
  33. }

When I run my code, it did not print out any information I expected, just

  1. it started
  2. cron: 2021/06/10 13:41:21 start
  3. cron: 2021/06/10 13:41:21 schedule, now=2021-06-10T13:41:21+08:00, entry=1, next=2021-06-10T13:43:21+08:00
  4. Exiting.
  5. Debugger finished with the exit code 0

It seems like it did not trigger an err, but not run correctly. How can I solve this?

答案1

得分: 2

它没有运行,因为你将作业安排为每2分钟运行一次,然后只等待了5秒钟,然后从main()返回,这导致你的程序退出。

如果你想在后台启动这个cron作业,你必须让程序运行足够长的时间,以便它能够运行和完成,至少在这个例子中要超过2分钟。

如果在main()中没有其他事情要做,你可以考虑使用c.Run()而不是c.Start()

英文:

It did not run because you scheduled the job to run every 2 minutes and then waited only 5 seconds, then you returned from main() which causes your program to exit.

If you want to start this cron in the background you must keep the program running long enough for it to run and complete, at least more than 2 minutes in this example.

If you have nothing else to do in main() then you might consider using c.Run() rather than c.Start().

huangapple
  • 本文由 发表于 2021年6月10日 13:56:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/67915507.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定