英文:
golang: CSV file to MS SQL does not work properly without placing fmt.Printf() towards end of for loop
问题
我正在使用Go语言读取CSV文件,并使用go-odbc将记录保存在MS SQL数据库中。它工作得很好,但是有一个问题,就是有些记录(大约10条记录)无法保存。这是一个随机问题,有时候有3条记录无法保存,有时候有2条等等。唯一能够保存所有记录的方法是在for循环的末尾加上fmt.Printf(" ")
。请注意,它必须打印一个空格,不能只是fmt.Printf("")
。我不确定我做错了什么。欢迎任何建议。另外,没有产生错误,程序正常终止。
我在下面附上了相关问题的代码,如果需要我贴出整个代码,请告诉我。
Go版本:go1.1 windows/amd64
for {
record, err := c.Read()
if err == io.EOF {
break
} else if err != nil {
fmt.Printf("Error while reading %s: %s\n", filename, err)
} else {
//替换字符串开头和结尾的单引号
re, err := regexp.Compile("^'|'$")
params := make([]interface{}, 0, numElements)
valueHolders := make([]string, 0, numElements)
tmpFields := make([]string, 0, numElements)
count := 0
for i:=1;i<=numElements;i++ {
tmp := re.ReplaceAllString(record[i],"")
//只插入非空值
if len(tmp) > 0 {
params = params[0:count+1]
params[count] = tmp
valueHolders = valueHolders[0:count+1]
valueHolders[count] = "?"
tmpFields = tmpFields[0:count+1]
tmpFields[count] = fieldNames[i-1]
count++
}
}
query := "insert into [l2test].[dbo]."+tablename+" (" + strings.Join(tmpFields, ",") + ") values (" + strings.Join(valueHolders, ",") + ")"
stmt, err := dest.Prepare(query)
if stmt == nil {
fmt.Printf("Error preparing statment: %s\nQuery: %s\n%v\n\n", err, query, params)
} else {
stmt.Execute(params...)
stmt.Close()
}
}
fmt.Printf(" ")
}
英文:
I am using Go to read CSV file and save the records in a MS SQL database using go-odbc. It works great, but I have an issue where some records (about 10 records) do not get stored. This a random problem, sometimes 3 do not get saved, other times 2, etc. The only time where all the records are saved is when I put fmt.Printf(" ")
at the end of the for loop. Notice it has to print a blank space it cannot just be fmt.Printf("")
. I am not sure what am I do not wrong. Any suggestions is appreciated. Also, no errors are produced, the program terminates normally.
I included the code with the related problem, let me know if you need me to post the entire code.
Go version: go1.1 windows/amd64
for {
record, err := c.Read()
if err == io.EOF {
break
} else if err != nil {
fmt.Printf("Error while reading %s: %s\n", filename, err)
} else {
//replace the single quote at the beginning and end of string
re, err := regexp.Compile("^'|'$")
params := make([]interface{}, 0, numElements)
valueHolders := make([]string, 0, numElements)
tmpFields := make([]string, 0, numElements)
count := 0
for i:=1;i<=numElements;i++ {
tmp := re.ReplaceAllString(record[i],"")
//insert only non-empty values
if len(tmp) > 0 {
params = params[0:count+1]
params[count] = tmp
valueHolders = valueHolders[0:count+1]
valueHolders[count] = "?"
tmpFields = tmpFields[0:count+1]
tmpFields[count] = fieldNames[i-1]
count++
}
}
query := "insert into [l2test].[dbo]."+tablename+" (" + strings.Join(tmpFields, ",") + ") values (" + strings.Join(valueHolders, ",") + ")"
stmt, err := dest.Prepare(query)
if stmt == nil {
fmt.Printf("Error preparing statment: %s\nQuery: %s\n%v\n\n", err, query, params)
} else {
stmt.Execute(params...)
stmt.Close()
}
}
fmt.Printf(" ")
}
答案1
得分: 0
当你在末尾删除一些元素时,通常是因为你从输入的末尾删除了它们,或者没有在输出上写入/刷新/提交/关闭它们。例如,在输入上,你可能忽略了EOF处的最后一个切片。我会以不同的方式编写你的EOF和错误测试。
你的数据库访问错误处理也需要改进。例如,它应该检查错误。
一旦我们发现数据库访问错误,将一些诊断信息(如query
和record
)添加到返回的错误(err
)中。
func insertRecord(conn *odbc.Connection, query string, params []interface{}) error {
stmt, err := conn.Prepare(query)
defer func() {
if stmt != nil {
stmt.Close()
}
}()
if err != nil {
return err
}
err = stmt.Execute(params...)
if err != nil {
return err
}
return nil
}
for {
record, err := c.Read()
if err != nil {
if err != io.EOF {
fmt.Printf("Error while reading %s: %s\n", filename, err)
break
}
if len(record) == 0 {
break
}
}
// 处理记录
query := "insert into [l2test].[dbo]." + tablename +
"(" + strings.Join(tmpFields, ",") + ")" +
" values (" + strings.Join(valueHolders, ",") + ")"
err = insertRecord(dest, query, params)
if err != nil {
err = fmt.Errorf("%s\n%s\n%v\n%s", err, query, params, strings.Join(record, "||"))
fmt.Println(err)
continue
}
}
英文:
When you drop a few elements at the end, it's usually because you dropped them from the end of the input or didn't write/flush/commit/close them on output. For example, on input you may be ignoring the last slice at EOF. I would write your EOF and error tests differently.
The error handling for your database access needs improvement too. For example, it should check for errors.
Once we find database access errors, add some diagnostic information, such as the query
and the record
, to the returned error (err
).
func insertRecord(conn *odbc.Connection, query string, params []interface{}) error {
stmt, err := conn.Prepare(query)
defer func() {
if stmt != nil {
stmt.Close()
}
}()
if err != nil {
return err
}
err = stmt.Execute(params...)
if err != nil {
return err
}
return nil
}
for {
record, err := c.Read()
if err != nil {
if err != io.EOF {
fmt.Printf("Error while reading %s: %s\n", filename, err)
break
}
if len(record) == 0 {
break
}
}
// do things with a record
query := "insert into [l2test].[dbo]." + tablename +
" (" + strings.Join(tmpFields, ",") + ")" +
" values (" + strings.Join(valueHolders, ",") + ")"
err = insertRecord(dest, query, params)
if err != nil {
err = fmt.Errorf("%s\n%s\n%v\n%s", err, query, params, strings.Join(record, "||"))
fmt.Println(err)
continue
}
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论