GoLang的FanIn函数不起作用

huangapple go评论92阅读模式
英文:

GoLang FanIn function not working

问题

所以我正在尝试使用Rob Pike的fanin函数编写一个网络爬虫。

这是我的代码 -

package main
import (
    "net/http"
    "encoding/json"
    "fmt"
    "io/ioutil"
)


func main() {
   fanIn(getDuckDuckGo("food"), getGitHub("defunkt"))

}

type DuckDuckGoResponse struct {
    RelatedTopics []struct {
        Result string `json:"Result"`
        FirstUrl string `json:"FirstURL"`
        Text string `json:"Text"`
    } `json:"RelatedTopics"`
}

type GitHubResponse struct {
    Login string `json:"login"`
    Email string `json:"email"`
    Name string `json:"name"`
}

func fanIn(input1 <-chan DuckDuckGoResponse, input2 <-chan GitHubResponse) <-chan string {
    c := make(chan string)
    go func() {
        for {
            select {
            case s := <-input1:
                fmt.Println(s)
            case s := <-input2:
                fmt.Println(s)
            }
        }
    }()
    return c
}


func getDuckDuckGo(k string) <-chan DuckDuckGoResponse {
    resp, err := http.Get("http://api.duckduckgo.com/?q=" + k + "&format=json&pretty=1")
    if err != nil {
        return nil
    }
    c := make(chan DuckDuckGoResponse)
    var duckDuckParsed DuckDuckGoResponse
    jsonDataFromHttp, jsonErr := ioutil.ReadAll(resp.Body)

    if jsonErr != nil {
        fmt.Println("Json error!")
    }
    defer resp.Body.Close()


    if err:= json.Unmarshal(jsonDataFromHttp, &duckDuckParsed); err != nil {
        panic(err)
    }
    return c
}

func getGitHub(k string) <-chan GitHubResponse {
    resp, err := http.Get("https://api.github.com/users/?q=" + k)
    if err != nil {
        return nil
    }
    c := make(chan GitHubResponse)

    var githubParsed GitHubResponse
    jsonDataFromHttp, jsonErr := ioutil.ReadAll(resp.Body)

    if jsonErr != nil {
        fmt.Println("Json error!")
    }

    defer resp.Body.Close()

    if err:= json.Unmarshal(jsonDataFromHttp, &githubParsed); err != nil {
        panic(err)
    }
    return c
}

我运行这个程序,但没有任何输出。

为什么?

谢谢

英文:

So I'm trying to write a webcrawler using Rob Pike's fanin function.

This is my code -

package main
import (
	&quot;net/http&quot;
	&quot;encoding/json&quot;
    &quot;fmt&quot;
    &quot;io/ioutil&quot;
)


func main() {
   fanIn(getDuckDuckGo(&quot;food&quot;), getGitHub(&quot;defunkt&quot;))

}

type DuckDuckGoResponse struct {
    RelatedTopics []struct {
        Result string `json:&quot;Result&quot;`
        FirstUrl string `json:&quot;FirstURL&quot;`
        Text string `json:&quot;Text&quot;`
    } `json:&quot;RelatedTopics&quot;`
}

type GitHubResponse struct {
    Login string `json:&quot;login&quot;`
    Email string `json:&quot;email&quot;`
    Name string `json:&quot;name&quot;`
}

func fanIn(input1 &lt;-chan DuckDuckGoResponse, input2 &lt;-chan GitHubResponse) &lt;-chan string {
    c := make(chan string)
    go func() {
        for {
            select {
            case s := &lt;-input1:
                fmt.Println(s)
            case s := &lt;-input2:
                fmt.Println(s)
            }
        }
    }()
    return c
}


func getDuckDuckGo(k string) &lt;-chan DuckDuckGoResponse {
    resp, err := http.Get(&quot;http://api.duckduckgo.com/?q=&quot; + k + &quot;&amp;format=json&amp;pretty=1&quot;)
    if err != nil {
        return nil
    }
    c := make(chan DuckDuckGoResponse)
    var duckDuckParsed DuckDuckGoResponse
    jsonDataFromHttp, jsonErr := ioutil.ReadAll(resp.Body)

    if jsonErr != nil {
        fmt.Println(&quot;Json error!&quot;)
    }
    defer resp.Body.Close()


    if err:= json.Unmarshal(jsonDataFromHttp, &amp;duckDuckParsed); err != nil {
        panic(err)
    }
    return c
}

func getGitHub(k string) &lt;-chan GitHubResponse {
    resp, err := http.Get(&quot;https://api.github.com/users/?q=&quot; + k)
    if err != nil {
        return nil
    }
    c := make(chan GitHubResponse)

    var githubParsed GitHubResponse
    jsonDataFromHttp, jsonErr := ioutil.ReadAll(resp.Body)

    if jsonErr != nil {
        fmt.Println(&quot;Json error!&quot;)
    }

    defer resp.Body.Close()

    if err:= json.Unmarshal(jsonDataFromHttp, &amp;githubParsed); err != nil {
        panic(err)
    }
    return c
}

I run this program, and nothing prints.

Why?

Thanks

答案1

得分: 0

乍一看,fanIn 函数返回的通道在你的主循环中没有被读取。所以,是的,你调用了返回通道的 fanIn 函数,但是没有任何东西从该通道中读取。为了使通道有用,需要有一个消费者从通道中消费,同时在另一端需要有一个生产者在该通道上生产。换句话说,除非另一端准备好接收,否则在通道上发送无法取得进展。

接下来,你的 getGitHubgetDuckDuckGo 返回通道,但它们实际上没有在返回的通道上发送任何内容。此外,你真正需要的是一种调用这些函数的方式,让它们返回一个通道并继续执行你的工作。你需要使用额外的 goroutine 来使 http.Get 调用完成它们的工作。

最后,你的 fanIn 函数也创建并返回一个通道,但它实际上没有将来自 input1input2 的结果“汇聚”起来。由于 fanIn 返回的是一个字符串类型的通道,你需要将一个字符串写入其中,这可以是 DuckDuckGoResponseGitHubResponse 的一个字段。

我建议你看一下这个简化的示例,了解你想要实现的内容:https://talks.golang.org/2012/go-docs/faninboring.go

最后观察到,你检查了 jsonErr != nil 并将其打印出来,但你可能也希望返回 nil,以防止代码继续执行。

希望这能给你足够的见解,让你的代码能够正常工作。祝你好运!

英文:

At first glance, the fanIn function returns a channel that is not being read from in your main loop. So yes, you are invoking the fanIn function which returns a channel, but there is nothing reading off of that channel. For a channel to be useful there needs to be a consumer consuming from the channel while on the other end there needs to be a producer producing on that channel. In other words, sending on a channel can't make progress unless someone on the other end is ready to receive on it.

Next, your getGitHub and getDuckDuckGo return channels, but they don't actually send anything on those channels that they return. Also, what you really need is a way to invoke those functions, have them return a channel and still execute your work. You need to use additional goroutines in order be able to have the http.Get calls do their work.

Lastly, your fanIn function also creates a channel and returns it, however it doesn't actually "fan-in" the results from input1 or input2. Since the fanIn returns a channel of type string, you'll need to write a string into them which could be a field off of DuckDuckGoResponse and GitHubResponse.

I urge you too look at this streamlined example of what you are trying to accomplish: https://talks.golang.org/2012/go-docs/faninboring.go

Last observation you are checking that jsonErr != nil and printing it out but you probably want to return nil as well to prevent the code from continuing on.

I hope this gives you just enough insight to get your code working. Good luck!

huangapple
  • 本文由 发表于 2015年1月16日 12:19:53
  • 转载请务必保留本文链接:https://go.coder-hub.com/27977242.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定