HTTP批量从多个API获取数据并保存到结构体中

huangapple go评论81阅读模式
英文:

HTTP Batch Get from multiple APIs and save to struct

问题

我有以下函数,用于获取一个URL并将数据返回给接口(例如结构体/整数/其他):

var httpClient = &http.Client{Timeout: 10 * time.Second}

func getURLToTarget(url string, target interface{}) error {
    req, err := httpClient.Get(url)
    if err != nil {
        return err
    }
    defer req.Body.Close()
    return json.NewDecoder(req.Body).Decode(target)
}

然后我有几个类似这样的函数:

func GetCustomerByID(APIKey, cusID string) {
    cus := new(Customer)
    getURLToTarget(fmt.Sprintf("http://someurl.com/%s/customerbyid/:%s", APIKey, cusID), &cus)
}

在这种情况下,它将把JSON响应保存到以下结构体中:

type Customer struct {
    Name    string
    Email   string
    Address string
}

现在我的问题是,当我运行以下代码时,如何使所有这些HTTP请求同时执行:

func main() {
    apikey := "some api key"
    GetCustomerByID(apikey, "43279843")
    GetCustomerDiscounts(apikey, "43279843")
    GetProductByID(apikey, "32124")
}

我相当确定我需要使用通道,但我无法弄清楚如何使用,请帮忙解答。

英文:

I have the following function that I use to get a URL and return the data to an interface (e.g. struct/int/whatever):

var httpClient = &http.Client{Timeout: 10 * time.Second}

func getURLToTarget(url string, target interface{}) error {
	req, err := httpClient.Get(url)
	if err != nil {
		return err
	}
	defer req.Body.Close()
	return json.NewDecoder(req.Body).Decode(target)
}

then I have several functions that look like this:

func GetCustomerByID(APIKey, cusID string) {
  cus := new(Customer)
  getURLToTarget(fmt.Sprintf("http://someurl.com/%s/customerbyid/:%s", APIKey, cusID), &cus)
}

which in this case will save the json response into a struct like this:

type Customer struct {
  Name string
  Email string
  Address string
}

Now my question is, how can I make all these http requests perform simultaneously when I run:

func main() {
  apikey := "some api key"
  GetCustomerByID(apikey, "43279843")
  GetCustomerDiscounts(apikey, "43279843")
  GetProductByID(apikey, "32124")
}

I am pretty sure I need to use channels but I can't figure out how.. any help would be much appreciated

答案1

得分: 1

有很多方法可以实现这个,具体取决于你需要的操作。

基本的方法是使用goroutines和wg.WaitGroup来并行/并发进行HTTP调用,并在程序退出之前等待所有调用完成。例如:

func main() {
  apikey := "some api key"

  var wg sync.WaitGroup
  wg.Add(3)

  go func() {
    GetCustomerByID(apikey, "43279843")
    wg.Done()
  }()
  
  go func() {
    GetCustomerDiscounts(apikey, "43279843")
    wg.Done()
  }()

  go func() {
    GetProductByID(apikey, "32124")
    wg.Done()
  }()

  wg.Wait()
}

另一种方法是使用go通道(channel),如果你想检查每个HTTP调用的结果。例如:

func GetCustomerByID(APIKey, cusID string) Customer {
  cus := new(Customer)
  getURLToTarget(fmt.Sprintf("http://someurl.com/%s/customerbyid/:%s", APIKey, cusID), &cus)
  return cus
}

func main() {
  apikey := "some api key"

  c := make(chan Customer, 3)

  go func() {
    cus := GetCustomerByID(apikey, "43279843")
    c <- cus
  }()
  
  go func() {
    cus := GetCustomerDiscounts(apikey, "43279843")
    c <- cus
  }()

  go func() {
    cus := GetProductByID(apikey, "32124")
    c <- cus
  }()

  // 打印结果
  var i int
  for cus := range c {
    fmt.Printf("%#v\n", cus)
    i++

    if i == 3 {
      break
    }
  }
}
英文:

There are many ways to achieve this and it's based on what you need to happen.

The basic one is to use goroutines and wg.WaitGroup to do http call in parallel/concurrent and wait for all of it to finish before exiting the program. For example:

func main() {
  apikey := &quot;some api key&quot;

  var wg sync.WaitGroup
  wg.Add(3)

  go func() {
    GetCustomerByID(apikey, &quot;43279843&quot;)
    wg.Done()
  }()
  
  go func() {
    GetCustomerDiscounts(apikey, &quot;43279843&quot;)
    wg.Done()
  }()

  go func() {
    GetProductByID(apikey, &quot;32124&quot;)
    wg.Done()
  }()

  wg.Wait()
}

Another approach is to use go channel if you want to inspect the result of each http call. For example:

func GetCustomerByID(APIKey, cusID string) Customer {
  cus := new(Customer)
  getURLToTarget(fmt.Sprintf(&quot;http://someurl.com/%s/customerbyid/:%s&quot;, APIKey, cusID), &amp;cus)
  return cus
}

func main() {
  apikey := &quot;some api key&quot;

  c := make(chan Customer, 3)

  go func() {
    cus := GetCustomerByID(apikey, &quot;43279843&quot;)
    c &lt;- cus
  }()
  
  go func() {
    cus := GetCustomerDiscounts(apikey, &quot;43279843&quot;)
    c &lt;- cus
  }()

  go func() {
    cus := GetProductByID(apikey, &quot;32124&quot;)
    c &lt;- cus
  }()

  // Print the result
  var i int
  for cus := range c {
    fmt.Printf(&quot;%#v\n&quot;, cus)
    i++

    if i == 3 {
      break
    }
  }
}

huangapple
  • 本文由 发表于 2017年5月17日 15:10:03
  • 转载请务必保留本文链接:https://go.coder-hub.com/44017927.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定