golang: CSV file to MS SQL does not work properly without placing fmt.Printf() towards end of for loop

huangapple go评论94阅读模式
英文:

golang: CSV file to MS SQL does not work properly without placing fmt.Printf() towards end of for loop

问题

我正在使用Go语言读取CSV文件,并使用go-odbc将记录保存在MS SQL数据库中。它工作得很好,但是有一个问题,就是有些记录(大约10条记录)无法保存。这是一个随机问题,有时候有3条记录无法保存,有时候有2条等等。唯一能够保存所有记录的方法是在for循环的末尾加上fmt.Printf(" ")。请注意,它必须打印一个空格,不能只是fmt.Printf("")。我不确定我做错了什么。欢迎任何建议。另外,没有产生错误,程序正常终止。

我在下面附上了相关问题的代码,如果需要我贴出整个代码,请告诉我。

Go版本:go1.1 windows/amd64

for {
    record, err := c.Read()
    if err == io.EOF {
        break
    } else if err != nil {  
        fmt.Printf("Error while reading %s: %s\n", filename, err)
    } else {
        //替换字符串开头和结尾的单引号
        re, err := regexp.Compile("^'|'$")
        params := make([]interface{}, 0, numElements)
        valueHolders := make([]string, 0, numElements)
        tmpFields := make([]string, 0, numElements)
        count := 0

        for i:=1;i<=numElements;i++ {  
            tmp := re.ReplaceAllString(record[i],"")

            //只插入非空值
            if len(tmp) > 0 {
                params = params[0:count+1]
                params[count] = tmp

                valueHolders = valueHolders[0:count+1]
                valueHolders[count] = "?"

                tmpFields = tmpFields[0:count+1]
                tmpFields[count] = fieldNames[i-1]

                count++
            }
        }

        query := "insert into [l2test].[dbo]."+tablename+" (" + strings.Join(tmpFields, ",") + ") values (" + strings.Join(valueHolders, ",") + ")"
        stmt, err := dest.Prepare(query)

        if stmt == nil {
            fmt.Printf("Error preparing statment: %s\nQuery: %s\n%v\n\n", err, query, params)
        } else {
            stmt.Execute(params...)
            stmt.Close()
        }
    }

    fmt.Printf(" ")
}
英文:

I am using Go to read CSV file and save the records in a MS SQL database using go-odbc. It works great, but I have an issue where some records (about 10 records) do not get stored. This a random problem, sometimes 3 do not get saved, other times 2, etc. The only time where all the records are saved is when I put fmt.Printf(&quot; &quot;) at the end of the for loop. Notice it has to print a blank space it cannot just be fmt.Printf(&quot;&quot;). I am not sure what am I do not wrong. Any suggestions is appreciated. Also, no errors are produced, the program terminates normally.

I included the code with the related problem, let me know if you need me to post the entire code.

Go version: go1.1 windows/amd64

for {
record, err := c.Read()
if err == io.EOF {
break
} else if err != nil {  
fmt.Printf(&quot;Error while reading %s: %s\n&quot;, filename, err)
} else {
//replace the single quote at the beginning and end of string
re, err := regexp.Compile(&quot;^&#39;|&#39;$&quot;)
params := make([]interface{}, 0, numElements)
valueHolders := make([]string, 0, numElements)
tmpFields := make([]string, 0, numElements)
count := 0
for i:=1;i&lt;=numElements;i++ {  
tmp := re.ReplaceAllString(record[i],&quot;&quot;)
//insert only non-empty values
if len(tmp) &gt; 0 {
params = params[0:count+1]
params[count] = tmp
valueHolders = valueHolders[0:count+1]
valueHolders[count] = &quot;?&quot;
tmpFields = tmpFields[0:count+1]
tmpFields[count] = fieldNames[i-1]
count++
}
}
query := &quot;insert into [l2test].[dbo].&quot;+tablename+&quot; (&quot; + strings.Join(tmpFields, &quot;,&quot;) + &quot;) values (&quot; + strings.Join(valueHolders, &quot;,&quot;) + &quot;)&quot;
stmt, err := dest.Prepare(query)
if stmt == nil {
fmt.Printf(&quot;Error preparing statment: %s\nQuery: %s\n%v\n\n&quot;, err, query, params)
} else {
stmt.Execute(params...)
stmt.Close()
}
}
fmt.Printf(&quot; &quot;)
}

答案1

得分: 0

当你在末尾删除一些元素时,通常是因为你从输入的末尾删除了它们,或者没有在输出上写入/刷新/提交/关闭它们。例如,在输入上,你可能忽略了EOF处的最后一个切片。我会以不同的方式编写你的EOF和错误测试。

你的数据库访问错误处理也需要改进。例如,它应该检查错误。

一旦我们发现数据库访问错误,将一些诊断信息(如queryrecord)添加到返回的错误(err)中。

func insertRecord(conn *odbc.Connection, query string, params []interface{}) error {
stmt, err := conn.Prepare(query)
defer func() {
if stmt != nil {
stmt.Close()
}
}()
if err != nil {
return err
}
err = stmt.Execute(params...)
if err != nil {
return err
}
return nil
}

for {
record, err := c.Read()
if err != nil {
if err != io.EOF {
fmt.Printf("Error while reading %s: %s\n", filename, err)
break
}
if len(record) == 0 {
break
}
}
// 处理记录
query := "insert into [l2test].[dbo]." + tablename +
"(" + strings.Join(tmpFields, ",") + ")" +
" values (" + strings.Join(valueHolders, ",") + ")"
err = insertRecord(dest, query, params)
if err != nil {
err = fmt.Errorf("%s\n%s\n%v\n%s", err, query, params, strings.Join(record, "||"))
fmt.Println(err)
continue
}
}
英文:

When you drop a few elements at the end, it's usually because you dropped them from the end of the input or didn't write/flush/commit/close them on output. For example, on input you may be ignoring the last slice at EOF. I would write your EOF and error tests differently.

The error handling for your database access needs improvement too. For example, it should check for errors.

Once we find database access errors, add some diagnostic information, such as the query and the record, to the returned error (err).

func insertRecord(conn *odbc.Connection, query string, params []interface{}) error {
stmt, err := conn.Prepare(query)
defer func() {
if stmt != nil {
stmt.Close()
}
}()
if err != nil {
return err
}
err = stmt.Execute(params...)
if err != nil {
return err
}
return nil
}

for {
record, err := c.Read()
if err != nil {
if err != io.EOF {
fmt.Printf(&quot;Error while reading %s: %s\n&quot;, filename, err)
break
}
if len(record) == 0 {
break
}
}
// do things with a record
query := &quot;insert into [l2test].[dbo].&quot; + tablename +
&quot; (&quot; + strings.Join(tmpFields, &quot;,&quot;) + &quot;)&quot; +
&quot; values (&quot; + strings.Join(valueHolders, &quot;,&quot;) + &quot;)&quot;
err = insertRecord(dest, query, params)
if err != nil {
err = fmt.Errorf(&quot;%s\n%s\n%v\n%s&quot;, err, query, params, strings.Join(record, &quot;||&quot;))
fmt.Println(err)
continue
}
}

huangapple
  • 本文由 发表于 2013年6月11日 01:18:43
  • 转载请务必保留本文链接:https://go.coder-hub.com/17029415.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定