在Go中安排轮询任务

huangapple go评论119阅读模式
英文:

Scheduled polling task in Go

问题

我已经编写了一些代码,可以同时轮询URL,每30分钟一次:

func (obj * MyObj) Poll() {
    for ;; {
        for _, url := range obj.UrlList {
            //下载URL的当前内容并对其进行处理
        }
        time.Sleep(30 * time.Minute)
}

//在其他函数中启动goroutine
go obj.Poll()

那么我该如何在代码的其他地方添加到obj.UrlList,并确保下次轮询URL时,Poll goroutine中的UrlList也已经更新,并且将轮询新的URL?

我了解在Go中通过通信来共享内存,而不是相反的方式,并且我已经研究了通道,但是我不确定如何在这个示例中实现它们。

英文:

I have written some code that will concurrently poll URLs every 30 minutes:

func (obj * MyObj) Poll() {
    for ;; {
        for _, url := range obj.UrlList {
            //Download the current contents of the URL and do something with it
        }
        time.Sleep(30 * time.Minute)
}

//Start the routine in another function
go obj.Poll()

How would I then add to obj.UrlList elsewhere in the code and ensure that the next time the URLs are polled that the UrlList in the Poll goroutine as also been updated and as such will also poll the new URL?

I understand that memory is shared through communicating rather than vice versa in Go and I've investigated channels however I'm not sure how to implement them in this example.

答案1

得分: 16

这是一个未经测试但安全的模型,用于定期获取一些URL,并能够安全地动态添加新的URL到URL列表中。如果您想要删除一个URL,读者应该明白需要做什么。

type harvester struct {
	ticker *time.Ticker // 定期的计时器
	add    chan string  // 新URL通道
	urls   []string     // 当前的URLs
}

func newHarvester() *harvester {
	rv := &harvester{
		ticker: time.NewTicker(time.Minute * 30),
		add:    make(chan string),
	}
	go rv.run()
	return rv
}

func (h *harvester) run() {
	for {
		select {
		case <-h.ticker.C:
			// 当计时器触发时,是收割的时候
			for _, u := range h.urls {
				harvest(u)
			}
		case u := <-h.add:
			// 在任何时候(除了我们正在收割的时候),我们可以处理添加新URL的请求
			h.urls = append(h.urls, u)
		}
	}
}

func (h *harvester) AddURL(u string) {
	// 添加新URL就像将其放入通道中一样简单。
	h.add <- u
}
英文:

Here's an untested, but safe model for periodically fetching some URLs with the ability to dynamically add new URLs to the list of URLs safely. It should be obvious to the reader what would be required if you wanted to remove a URL as well.

type harvester struct {
	ticker *time.Ticker // periodic ticker
	add    chan string  // new URL channel
	urls   []string     // current URLs
}

func newHarvester() *harvester {
	rv := &harvester{
		ticker: time.NewTicker(time.Minute * 30),
		add:    make(chan string),
	}
	go rv.run()
	return rv
}

func (h *harvester) run() {
	for {
		select {
		case <-h.ticker.C:
			// When the ticker fires, it's time to harvest
			for _, u := range h.urls {
				harvest(u)
			}
		case u := <-h.add:
			// At any time (other than when we're harvesting),
			// we can process a request to add a new URL
			h.urls = append(h.urls, u)
		}
	}
}

func (h *harvester) AddURL(u string) {
	// Adding a new URL is as simple as tossing it onto a channel.
	h.add <- u
}

答案2

得分: 6

如果您需要定期按照固定时间间隔进行轮询,不应该使用time.Sleep,而应该使用time.Ticker(或类似的time.After)来实现。原因是time.Sleep只是一个简单的休眠,不考虑由于循环中实际工作而导致的时间漂移。相反,time.Ticker有一个单独的goroutine和一个通道,它们能够定期发送事件,从而引发一些有用的操作。

下面是一个类似于您的示例的例子。我添加了一个随机的抖动来说明使用time.Ticker的好处。

package main

import (
    "fmt"
    "time"
    "math/rand"
)

func Poll() {
    r := rand.New(rand.NewSource(99))
    c := time.Tick(10 * time.Second)
    for _ = range c {
        //Download the current contents of the URL and do something with it
        fmt.Printf("Grab at %s\n", time.Now())
        // add a bit of jitter
        jitter := time.Duration(r.Int31n(5000)) * time.Millisecond 
        time.Sleep(jitter)
    }
}

func main() {
    //go obj.Poll()
    Poll()
}

当我运行这个程序时,我发现它尽管有抖动,但仍然严格按照10秒的周期运行。

英文:

If you need to poll at regular periodic intervals, you should not use time.Sleep but a time.Ticker instead (or relative like time.After). The reason is that a sleep is just a sleep and takes no account of drift due to the real work you did in your loop. Conversely, a Ticker has a separate goroutine and a channel, which together are able to send you regular events and thereby cause something useful to happen.

Here's an example that's similar to yours. I put in a random jitter to illustrate the benefit of using a Ticker.

package main

import (
    "fmt"
    "time"
    "math/rand"
)

func Poll() {
    r := rand.New(rand.NewSource(99))
    c := time.Tick(10 * time.Second)
    for _ = range c {
        //Download the current contents of the URL and do something with it
        fmt.Printf("Grab at %s\n", time.Now())
        // add a bit of jitter
        jitter := time.Duration(r.Int31n(5000)) * time.Millisecond 
        time.Sleep(jitter)
    }
}

func main() {
    //go obj.Poll()
    Poll()
}

When I ran this, I found that it kept to a strict 10-second cycle in spite of the jitter.

答案3

得分: 1

// 通过通道进行队列类型。
type MyType struct {
queue chan []*net.URL
}

func (t *MyType) poll() {
for urls := range t.queue {
...
time.Sleep(30 * time.Minute)
}
}

// 使用缓冲队列创建实例。
t := MyType{make(chan []*net.URL, 25)}

go t.Poll()

英文:
// Type with queue through a channel.
type MyType struct {
    queue chan []*net.URL
}

func (t *MyType) poll() {
    for urls := range t.queue {
        ...
        time.Sleep(30 * time.Minute)
    }
}

// Create instance with buffered queue.
t := MyType{make(chan []*net.URL, 25)}

go t.Poll()

huangapple
  • 本文由 发表于 2013年6月4日 02:25:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/16903348.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定