New a array or map of json encoder type in Golang

huangapple go评论84阅读模式
英文:

New a array or map of json encoder type in Golang

问题

我在Golang中还是比较新手,遇到了以下问题。

// XXX a bit inefficient. could open r files and run over list once
for r := 0; r < nreduce; r++ {
    file, err = os.Create(ReduceName(fileName, JobNumber, r))
    if err != nil {
        log.Fatal("DoMap: create ", err)
    }
    enc := json.NewEncoder(file)
    for e := res.Front(); e != nil; e = e.Next() {
        kv := e.Value.(KeyValue)
        if ihash(kv.Key)%uint32(nreduce) == uint32(r) {
            err := enc.Encode(&kv)
            if err != nil {
                log.Fatal("DoMap: marshall ", err)
            }
        }
    }
    file.Close()
}

基本上,这段代码在每次循环迭代中创建一个文件,然后打开一个文件来编码属于该文件的内容,根据(key, value)对。然而,这段代码效率低下,因为它对该文件进行了太多次扫描。更高效的方法是打开r个文件,并在此列表文件上运行一次。所以我想这样写(但我不知道如何做):

enc_map := make(map[int]*json.Encoder)
for r := 0; r < nreduce; r++ {
    file, err := os.Create(ReduceName(fileName, JobNumber, r))
    if err != nil {
        log.Fatal("DoMap: create ", err)
    }
    enc := json.NewEncoder(file)
    enc_map[r] = enc
    for e := res.Front(); e != nil; e = e.Next() {
        kv := e.Value.(KeyValue)
        r := ihash(kv.Key) & uint32(nreduce)
        err := enc_map[r].Encode(&kv)
        if err != nil {
            log.Fatal("DoMap: marshall ", err)
        }
    }
}

这段代码首先创建了一个保存json.Encoder对象的映射(enc_map),然后对该文件进行一次迭代。我查阅了Go文档,它说json.Encoder的类型名是*Encoder。但是下面这行代码:

enc_map := make(map[int]*Encode)

是错误的,编译器给出了以下错误:

../mapreduce/mapreduce.go:228: undefined: Encode
../mapreduce/mapreduce.go:230: file.err undefined (type *os.File has no field or method err)
../mapreduce/mapreduce.go:230: multiple-value os.Create() in single-value context

那么正确的做法是什么呢?

英文:

I am pretty new in Golang and encountered the following problem.

// XXX a bit inefficient. could open r files and run over list once
for r := 0; r &lt; nreduce; r++ {
	file, err = os.Create(ReduceName(fileName, JobNumber, r))
	if err != nil {
		log.Fatal(&quot;DoMap: create &quot;, err)
	}
	enc := json.NewEncoder(file)
	for e := res.Front(); e != nil; e = e.Next() {
		kv := e.Value.(KeyValue)
		if ihash(kv.Key)%uint32(nreduce) == uint32(r) {
			err := enc.Encode(&amp;kv)
			if err != nil {
				log.Fatal(&quot;DoMap: marshall &quot;, err)
			}
		}
	}
	file.Close()
}

Basically this code snippet creates a file in each for-loop iteration and then open a file to encode the stuff which belongs to this file, according to a (key, value) pair. However, this code is inefficient since it scan this file for too many times. The more efficient way to do so is to open r files and run over this list file once. So I would like to write like this (but I don't know how to do so):

enc_map := make(map[int]*Encode)
for r := 0; r &lt; nreduce; r++ {
file. err = os.Create(ReduceName(fileName, JobNumber, r))
if err != nil {
	log.Fatal(&quot;DoMap: create &quot;, err)
}
enc := json.NewEncoder(file)
enc_map[r] = enc
for e := res.Front(); e != nil; e = e.Next() {
	kv := e.Value.(KeyValue)
	r := ihash(kv.Key)&amp;uint32(nreduce) 
	err := enc_map[r].Encode(&amp;kv)
	if err != nil {
		log.Fatal(&quot;DoMap: marshall &quot;, err)
	}
}

This code snippets first creates a map that saves the json.Encoder objects and then iterate over this file once. I looked up the go documentation and it says the type name of json.Encoder is *Encode. But the line

enc_map := make(map[int]*Encode)

is wrong and the compiler gives me the following error:

../mapreduce/mapreduce.go:228: undefined: Encode
../mapreduce/mapreduce.go:230: file.err undefined (type *os.File has no field or method err)
../mapreduce/mapreduce.go:230: multiple-value os.Create() in single-value context

So what's the right way to do things?

答案1

得分: 1

json.NewEncoder(file) 返回一个类型为 *Encoder 的对象。因此,尝试编辑以下行:

enc_map := make(map[int]*Encode)

改为:

enc_map := make(map[int]*json.Encoder)

以消除错误:

../mapreduce/mapreduce.go:228: undefined: Encode

为了消除其他两个错误,即:

../mapreduce/mapreduce.go:230: file.err undefined (type *os.File has no field or method err)
../mapreduce/mapreduce.go:230: multiple-value os.Create() in single-value context

只需在代码中将行中的 file.err 编辑为 file, err

希望这解决了你遇到的问题。

英文:

json.NewEncoder(file) returns an object of type *Encoder. Therefore, try editing the line:

enc_map := make(map[int]*Encode)

to:

enc_map := make(map[int]* json.Encoder)

to eliminate the error:

../mapreduce/mapreduce.go:228: undefined: Encode

To get rid of the other 2 errors, i.e.,

../mapreduce/mapreduce.go:230: file.err undefined (type *os.File has no field or method err)
../mapreduce/mapreduce.go:230: multiple-value os.Create() in single-value context

just edit file .err in line:

file. err = os.Create(ReduceName(fileName, JobNumber, r))

to file, err in your code.

I hope this solves the issue you're facing.

huangapple
  • 本文由 发表于 2015年12月28日 16:14:59
  • 转载请务必保留本文链接:https://go.coder-hub.com/34489925.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定