在Go中实现并发安全的模板:我该如何做?

huangapple go评论78阅读模式
英文:

Concurrency-safe templates in Go: How do I do it?

问题

以下是您的翻译:

我有以下调用

```go
import (
  "text/template"
)

//...

template.New(filepath.Base(name)).Funcs(templateFunctions).Parse(string(asset))

这个调用在多个 Go 协程中同时进行,导致以下恐慌:

致命错误同时进行的映射迭代和映射写入

以下是回溯信息:

协程 140 [运行中]
text/template.addValueFuncs(0xc00188e000?, 0xc00188e000?)
        [...]/go/src/text/template/funcs.go:88 +0x76
[...]/modules/template.loadEmbeddedTemplates({0x38ff6cb?, 0xc001cf8060?})
        [...]/src/modules/template/configBased.go:163 +0x749

上面引用的行在 src/modules/template/configBased.go:163,这是 template.New(...)

周围的函数是从多个协程同时调用的。

如果有帮助,以下是位于 go/src/text/template/funcs.go:88 的代码:

// addValueFuncs 向值中添加 funcs 中的函数,将它们转换为 reflect.Values。
func addValueFuncs(out map[string]reflect.Value, in FuncMap) {
    for name, fn := range in {
        if !goodName(name) {
            panic(fmt.Errorf("函数名 %q 不是有效的标识符", name))
        }
        v := reflect.ValueOf(fn)
        if v.Kind() != reflect.Func {
            panic("值 " + name + " 不是函数")
        }
        if !goodFunc(v.Type()) {
            panic(fmt.Errorf("无法安装方法/函数 %q,带有 %d 个结果", name, v.Type().NumOut()))
        }
        out[name] = v
    }
}

如果 template.New 是并发安全的,为什么这行代码会引发此恐慌,我应该如何正确处理它?

更新。

可疑函数 loadEmbeddedTemplates 的代码如下:

func loadEmbeddedTemplates(templateFile string) (*template.Template, error) {
    var t *template.Template

    templateFile = filepath.Join("share", "templates", filepath.Base(templateFile))
    dir := filepath.Dir(templateFile)
    names := assets.GetAssetNames()

    // 除了最后一个之外的所有模板
    filteredNames := []string{}

    for _, name := range names {
        if !strings.HasPrefix(name, dir+"/") || !strings.HasSuffix(name, ".tmpl") {
            continue
        }

        if name != templateFile {
            filteredNames = append(filteredNames, name)
        }
    }

    filteredNames = append(filteredNames, templateFile)

    for _, name := range filteredNames {
        asset, err := assets.GetAsset(name)
        if err != nil {
            return nil, err
        }

        if t == nil {
            t, err = template.New(filepath.Base(name)).Funcs(templateFunctions).Parse(string(asset))
        } else {
            t, err = t.New(filepath.Base(name)).Parse(string(asset))
        }

        if err != nil {
            return nil, err
        }
    }

    return t, nil
}

该函数简单地顺序加载 share/templates/ 中的所有模板。

英文:

I have the following call:

import (
  "text/template"
)

//...

template.New(filepath.Base(name)).Funcs(templateFunctions).Parse(string(asset))

which is called in several Go routines concurrently,
which in turn causes the following panic:

fatal error: concurrent map iteration and map write

Here is the backtrace:

goroutine 140 [running]:
text/template.addValueFuncs(0xc00188e000?, 0xc00188e000?)
        [...]/go/src/text/template/funcs.go:88 +0x76
[...]/modules/template.loadEmbeddedTemplates({0x38ff6cb?, 0xc001cf8060?})
        [...]/src/modules/template/configBased.go:163 +0x749

The line on src/modules/template/configBased.go:163
quoted above. It is template.New(...).

The surrounding function is called from goroutines concurrently.

That is the code from go/src/text/template/funcs.go:88 is it helps:

// addValueFuncs adds to values the functions in funcs, converting them to reflect.Values.
func addValueFuncs(out map[string]reflect.Value, in FuncMap) {
    for name, fn := range in {
        if !goodName(name) {
            panic(fmt.Errorf("function name %q is not a valid identifier", name))
        }
        v := reflect.ValueOf(fn)
        if v.Kind() != reflect.Func {
            panic("value for " + name + " not a function")
        }
        if !goodFunc(v.Type()) {
            panic(fmt.Errorf("can't install method/function %q with %d results", name, v.Type().NumOut()))
        }
        out[name] = v
    }
}

If template.New is concurrent-safe, why this line produces this panic, and how am I supposed to handle it properly?

Update.

The code of the nasty function loadEmbeddedTemplates:

func loadEmbeddedTemplates(templateFile string) (*template.Template, error) {
    var t *template.Template

    templateFile = filepath.Join("share", "templates", filepath.Base(templateFile))
    dir := filepath.Dir(templateFile)
    names := assets.GetAssetNames()

    // All templates except + the last one at the end
    filteredNames := []string{}

    for _, name := range names {
        if !strings.HasPrefix(name, dir+"/") || !strings.HasSuffix(name, ".tmpl") {
            continue
        }

        if name != templateFile {
            filteredNames = append(filteredNames, name)
        }
    }

    filteredNames = append(filteredNames, templateFile)

    for _, name := range filteredNames {
        asset, err := assets.GetAsset(name)
        if err != nil {
            return nil, err
        }

        if t == nil {
            t, err = template.New(filepath.Base(name)).Funcs(templateFunctions).Parse(string(asset))
        } else {
            t, err = t.New(filepath.Base(name)).Parse(string(asset))
        }

        if err != nil {
            return nil, err
        }
    }

    return t, nil
}

The function simply loads all templates from share/templates/ one after another

答案1

得分: 2

你的 loadEmbeddedTemplates() 函数访问了 templateFunctions 变量,将其传递给 Template.Funcs(),显然会读取它(会迭代它)。

而且你很可能在另一个 goroutine 中同时填充它。因此出现了并发映射写入错误。必须同步访问它。

如果可能的话,先填充它,然后才开始使用它(将其传递给 Template.Funcs())。这样就不需要额外的同步或锁定(并发只读始终是可以的)。

英文:

Your loadEmbeddedTemplates() function accesses the templateFunctions variable, passes it to Template.Funcs() which will obviously read it (will iterate over it).

And you are likely populating it concurrently, in another goroutine. Hence the concurrent map write error. Access to it must be synchronized.

If possible, populate it first, and only after that start using it (passing it to Template.Funcs()). That way no additional synchronization or locking will be required (concurrent read only is always OK).

huangapple
  • 本文由 发表于 2023年4月17日 21:29:52
  • 转载请务必保留本文链接:https://go.coder-hub.com/76035714.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定