如何一次性插入多个数据

huangapple go评论96阅读模式
英文:

How to insert multiple data at once

问题

我知道一次性插入多个数据更高效:

INSERT INTO test(n1, n2, n3) 
VALUES(v1, v2, v3),(v4, v5, v6),(v7, v8, v9);

在 Golang 中如何实现这个功能?

data := []map[string]string{
   {"v1":"1", "v2":"1", "v3":"1"},
   {"v1":"2", "v2":"2", "v3":"2"},
   {"v1":"3", "v2":"3", "v3":"3"},
}
// 我不想这样做
for _, v := range data {
    sqlStr := "INSERT INTO test(n1, n2, n3) VALUES(?, ?, ?)"
    stmt, _ := db.Prepare(sqlStr)
    res, _ := stmt.Exec(v["v1"], v["v2"], v["v3"])
}

使用字符串拼接的方式,但这不是一个好方法。db.Prepare 更安全,对吗?

sqlStr := "INSERT INTO test(n1, n2, n3) VALUES"
for k, v := range data {
    if k == 0 {
        sqlStr += fmt.Sprintf("(%v, %v, %v)", v["v1"], v["v2"], v["v3"])
    } else {
        sqlStr += fmt.Sprintf(",(%v, %v, %v)", v["v1"], v["v2"], v["v3"])
    } 
}
res, _ := db.Exec(sqlStr)

我需要一个更安全和高效的函数来一次性插入多个数据。

英文:

I know that Insert multiple data at once more efficiency:

INSERT INTO test(n1, n2, n3) 
VALUES(v1, v2, v3),(v4, v5, v6),(v7, v8, v9);

How to do that in golang?

data := []map[string]string{
   {"v1":"1", "v2":"1", "v3":"1"},
   {"v1":"2", "v2":"2", "v3":"2"},
   {"v1":"3", "v2":"3", "v3":"3"},
}
//I do not want to do it
for _, v := range data {
    sqlStr := "INSERT INTO test(n1, n2, n3) VALUES(?, ?, ?)"
    stmt, _ := db.Prepare(sqlStr)
    res, _ := stmt.Exec(v["v1"], v["v2"], v["v3"])
}

Use string splice, but it's not good. db.Prepare more safer, right?

sqlStr := "INSERT INTO test(n1, n2, n3) VALUES"
for k, v := range data {
    if k == 0 {
        sqlStr += fmt.Sprintf("(%v, %v, %v)", v["v1"], v["v2"], v["v3"])
    } else {
        sqlStr += fmt.Sprintf(",(%v, %v, %v)", v["v1"], v["v2"], v["v3"])
    } 
}
res, _ := db.Exec(sqlStr)

I need a function safer and efficient insert mulitple data at once.

答案1

得分: 106

为什么不像这样写呢?(在这里写,没有测试,所以可能会有语法错误):

sqlStr := "INSERT INTO test(n1, n2, n3) VALUES "
vals := []interface{}{}

for _, row := range data {
    sqlStr += "(?, ?, ?),"
    vals = append(vals, row["v1"], row["v2"], row["v3"])
}
// 去掉最后一个逗号
sqlStr = sqlStr[0:len(sqlStr)-1]
// 准备语句
stmt, _ := db.Prepare(sqlStr)

// 一次性格式化所有的值
res, _ := stmt.Exec(vals...)

这段代码的作用是将数据批量插入到数据库表中。它使用了预处理语句和参数化查询,以提高性能和安全性。首先,它构建了一个包含占位符的 SQL 语句,并将每一行的值添加到一个值数组中。然后,它去掉了 SQL 语句末尾的逗号,并准备了该语句。最后,它使用 Exec 方法执行了该语句,并传入了值数组作为参数。

英文:

why not something like this? (writing here without testing so there might be syntax errors):

sqlStr := "INSERT INTO test(n1, n2, n3) VALUES "
vals := []interface{}{}

for _, row := range data {
	sqlStr += "(?, ?, ?),"
	vals = append(vals, row["v1"], row["v2"], row["v3"])
}
//trim the last ,
sqlStr = sqlStr[0:len(sqlStr)-1]
//prepare the statement
stmt, _ := db.Prepare(sqlStr)

//format all vals at once
res, _ := stmt.Exec(vals...)

答案2

得分: 13

对于Postgres,lib pq支持批量插入:https://godoc.org/github.com/lib/pq#hdr-Bulk_imports

但是通过下面的代码也可以实现相同的效果,它在尝试执行批量条件更新时非常有用(根据需要修改查询)。

要执行类似的Postgres批量插入,可以使用以下函数。

// ReplaceSQL将任何字符串模式的实例替换为递增的$ n序列
func ReplaceSQL(oldsearchPattern string) string {
   tmpCount := strings.Count(oldsearchPattern)
   for m := 1; m <= tmpCount; m++ {
      old = strings.Replace(oldsearchPattern"$"+strconv.Itoa(m)1)
   }
   return old
}

因此,上面的示例变为:

sqlStr := "INSERT INTO test(n1,n2,n3) VALUES "
vals := []interface{}{}

for _row := range data {
   sqlStr += "(?,?,?),"
   vals = append(valsrow["v1"]row["v2"]row["v3"])
}

//去掉最后一个逗号
sqlStr = strings.TrimSuffix(sqlStr",")

//将?替换为$n以适应Postgres
sqlStr = ReplaceSQL(sqlStr"?")

//准备语句
stmt_ := db.Prepare(sqlStr)

//一次性格式化所有vals
res_ := stmt.Exec(vals...)
英文:

For Postgres lib pq supports bulk inserts: https://godoc.org/github.com/lib/pq#hdr-Bulk_imports

But same can be achieved through below code but where it is really helpful is when one tries to perform bulk conditional update (change the query accordingly).

For performing similar bulk inserts for Postgres, you can use the following function.

// ReplaceSQL replaces the instance occurrence of any string pattern with an increasing $n based sequence
func ReplaceSQL(old, searchPattern string) string {
   tmpCount := strings.Count(old, searchPattern)
   for m := 1; m &lt;= tmpCount; m++ {
	  old = strings.Replace(old, searchPattern, &quot;$&quot;+strconv.Itoa(m), 1)
   }
   return old
}

So above sample becomes

sqlStr := &quot;INSERT INTO test(n1, n2, n3) VALUES &quot;
vals := []interface{}{}

for _, row := range data {
   sqlStr += &quot;(?, ?, ?),&quot;
   vals = append(vals, row[&quot;v1&quot;], row[&quot;v2&quot;], row[&quot;v3&quot;])
}

//trim the last ,
sqlStr = strings.TrimSuffix(sqlStr, &quot;,&quot;)

//Replacing ? with $n for postgres
sqlStr = ReplaceSQL(sqlStr, &quot;?&quot;)

//prepare the statement
stmt, _ := db.Prepare(sqlStr)

//format all vals at once
res, _ := stmt.Exec(vals...)

答案3

得分: 4

Gorm V2(于2020年8月30日发布)现在支持批量插入查询。

// 将切片数据传递给Create方法,GORM将生成一个单独的SQL语句
// 以插入所有数据并回填主键值,
// 钩子方法也将被调用。

var users = []User{{Name: "jinzhu1"}, {Name: "jinzhu2"}, {Name: "jinzhu3"}}
DB.Create(&users)

for _, user := range users {
  user.ID // 1,2,3
}

有关更多详细信息,请参阅官方文档:https://gorm.io/docs/create.html。

英文:

Gorm V2 (released on 30th August 2020) now supports batch insert query.

// Pass slice data to method Create, GORM will generate a single SQL statement
// to insert all the data and backfill primary key values,
// hook methods will be invoked too.

var users = []User{{Name: &quot;jinzhu1&quot;}, {Name: &quot;jinzhu2&quot;}, {Name: &quot;jinzhu3&quot;}}
DB.Create(&amp;users)

for _, user := range users {
  user.ID // 1,2,3
}

For more details refer to the official documentation here: https://gorm.io/docs/create.html.

答案4

得分: 1

如果您启用了多语句功能,那么您可以一次执行多个语句。
通过这样做,您应该能够处理多个插入操作。

https://github.com/go-sql-driver/mysql#multistatements

英文:

If you enable multi statements , then you can execute multiple statement at once.
With that , you should be able to handle multiple inserts.

https://github.com/go-sql-driver/mysql#multistatements

答案5

得分: 0

经过广泛的研究,以下代码对我有效:

var values []interface{}
for _, scope := range scopes {
    values = append(values, scope.ID, scope.Code, scope.Description)
}
sqlStr := `INSERT INTO scopes (application_id, scope, description) VALUES %s`
sqlStr = setupBindVars(sqlStr, "(?, ?, ?)", len(scopes))

_, err = s.db.ExecContext(ctx, sqlStr, values...)

// 辅助函数,用于将?替换为正确数量的绑定变量集合

func setupBindVars(stmt, bindVars string, len int) string {
    bindVars += ","
    stmt = fmt.Sprintf(stmt, strings.Repeat(bindVars, len))
    return strings.TrimSuffix(stmt, ",")
}

请注意,这是一段Go语言代码,用于将一组数据插入到数据库表中。它使用了绑定变量来防止SQL注入攻击。

英文:

After extensive research this worked for me:

var values []interface{}
for _, scope := range scopes {
	values = append(values, scope.ID, scope.Code, scope.Description)
}
sqlStr := `INSERT INTO scopes (application_id, scope, description) VALUES %s`
sqlStr = setupBindVars(sqlStr, &quot;(?, ?, ?)&quot;, len(scopes))

_, err = s.db.ExecContext(ctx, sqlStr, values...)

// helper function to replace ? with the right number of sets of bind vars

func setupBindVars(stmt, bindVars string, len int) string {
	bindVars += &quot;,&quot;
	stmt = fmt.Sprintf(stmt, strings.Repeat(bindVars, len))
	return strings.TrimSuffix(stmt, &quot;,&quot;)
}

答案6

得分: -1

https://gorm.io/docs/create.html#Batch-Insert

代码示例:

var users = []User{{Name: "jinzhu1"}, {Name: "jinzhu2"}, {Name: "jinzhu3"}}
DB.Create(&users)
英文:

From https://gorm.io/docs/create.html#Batch-Insert

Code sample:

var users = []User{{Name: &quot;jinzhu1&quot;}, {Name: &quot;jinzhu2&quot;}, {Name: &quot;jinzhu3&quot;}}
DB.Create(&amp;users)

答案7

得分: -1

这是一种高效的过渡方式,它只在提交后进行网络调用。

func insert(requestObj []models.User) (bool, error) {
    tx := db.Begin()
    defer func() {
        if r := recover(); r != nil {
            tx.Rollback()
        }
    }()

    for _, obj := range requestObj {
        if err := tx.Create(&obj).Error; err != nil {
            logging.AppLogger.Errorf("Failed to create user")
            tx.Rollback()
            return false, err
        }
    }
    err := tx.Commit().Error
    if err != nil {
        return false, err
    }
    return true, nil
}
英文:

this is an efficient way to do transition which will do network call only after commit.

func insert(requestObj []models.User) (bool, error) {
	tx := db.Begin()
	defer func() {
		if r := recover(); r != nil {
			tx.Rollback()
		}
	}()

	for _, obj := range requestObj {
		if err := tx.Create(&amp;obj).Error; err != nil {
			logging.AppLogger.Errorf(&quot;Failed to create user&quot;)
			tx.Rollback()
			return false, err
		}
	}
	err := tx.Commit().Error
	if err != nil {
		return false, err
	}
	return true, nil
}

答案8

得分: -1

在整合了发布的答案的反馈后,我得到了以下代码:

const insertQuery := "INSERT INTO test(n1, n2, n3) VALUES "
const row = "(?, ?, ?)"

var inserts []string 
var vals []interface{}

for _, row := range data {
    inserts = append(inserts, row)
    vals = append(vals, row["v1"], row["v2"], row["v3"])
}
sqlStr := insertQuery + strings.Join(inserts, ",")

// 准备语句
stmt, _ := db.Prepare(sqlStr)

// 使用后关闭 stmt
defer stmt.Close()

// 一次性格式化所有 vals
res, _ := stmt.Exec(vals...)

请注意,我只翻译了代码部分,其他内容不包括在内。

英文:

I ended up with this, after combining the feedback on posted answers:

const insertQuery := &quot;INSERT INTO test(n1, n2, n3) VALUES &quot;
const row = &quot;(?, ?, ?)&quot;

var inserts []string 
vars vals []interface{}

for _, row := range data {
    inserts = append(inserts, row)
    vals = append(vals, row[&quot;v1&quot;], row[&quot;v2&quot;], row[&quot;v3&quot;])
}
sqlStr := insertQuery + strings.Join(inserts, &quot;,&quot;)

//prepare the statement
stmt, _ := db.Prepare(sqlStr)

//close stmt after use
defer stmt.Close()

//format all vals at once
res, _ := stmt.Exec(vals...)

huangapple
  • 本文由 发表于 2014年1月14日 15:32:08
  • 转载请务必保留本文链接:https://go.coder-hub.com/21108084.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定