如何使用dbSendQuery在R中将不同的CSV文件导入到一个SQL数据库表中?

huangapple go评论71阅读模式
英文:

How to use dbSendQuery to import different CSV files into 1 SQL database table using R?

问题

关于这个页面,我正在使用以下代码尝试使用R将多个CSV文件插入SQL数据库表中:

#步骤1:加载库
library(RPostgres)
library(data.table)
library(DBI)

#步骤2:创建所有文件的数据框
file_names1 <- dir("C:/Users/Akanksha/Desktop/BSRN_TestRowsOnly/", full.names = TRUE, recursive = T) #文件所在的路径
my_data_frame <- do.call(rbind, lapply(file_names1, read.csv))

#步骤3:使用RPostgres建立连接
dsn_database <- "..."   # 指定您的数据库名称
dsn_hostname <- "localhost"  # 指定主机名,例如:"aws-us-east-1-portal.4.dblayer.com"
dsn_port <- "5432"                # 指定端口号,例如:98939
dsn_uid <- "..."         # 指定用户名,例如:"admin"
dsn_pwd <- "..."        # 指定密码,例如:"xxx"

tryCatch({
  drv <- dbDriver("Postgres")
  print("连接到数据库中...")
  connec <- dbConnect(drv, 
                      dbname = dsn_database,
                      host = dsn_hostname, 
                      port = dsn_port,
                      user = dsn_uid, 
                      password = dsn_pwd)
  print("数据库已连接!")
},
error=function(cond) {
  print("无法连接到数据库。")
})

#步骤4:使用RPostgres运行和测试查询
dbSendQuery(connec, "DROP TABLE IF EXISTS Population")  # 表名为Population

#步骤5:导入数据
RPostgres::dbSendQuery(
  connec, 
  "INSERT INTO mytable (Population) VALUES (?, ?);",
  params = list(my_data_frame)
)

它返回了一个错误:

Error: Failed to prepare query: ERROR:  syntax error at or near "my_data_frame"
LINE 1: INSERT INTO mytable (Population) 'my_data_frame';
                                  ^
In addition: Warning message:
In result_create(conn@ptr, statement, immediate) :
  Closing open result set, cancelling previous query

我尝试了不同的方法,但没有成功。我真的卡在这个任务上,有人可以帮助我解决这个问题或建议一个新的代码来将不同的CSV文件复制粘贴到一个SQL表中吗?R和SQL对我来说都是新的东西。

英文:

Referring to this page,I am using following code trying to insert multiple csv files into SQL database table using R

#Step 1: Libraries
library(RPostgres)
library(data.table)
library(DBI)

#Step 2: make a dataframe of all files
file_names1 &lt;- dir(&quot;C:/Users/Akanksha/Desktop/BSRN_TestRowsOnly/&quot;, full.names = TRUE, recursive = T) #where you have your files
my_data_frame &lt;- do.call(rbind,lapply(file_names1,read.csv))


#Step 3: Establish Connection using RPostgres
dsn_database = &quot;...&quot;   # Specify the name of your Database
dsn_hostname = &quot;localhost&quot;  # Specify host name e.g.:&quot;aws-us-east-1-portal.4.dblayer.com&quot;
dsn_port = &quot;5432&quot;                # Specify your port number. e.g. 98939
dsn_uid = &quot;...&quot;         # Specify your username. e.g. &quot;admin&quot;
dsn_pwd = &quot;...&quot;        # Specify your password. e.g. &quot;xxx&quot;

tryCatch({
  drv &lt;- dbDriver(&quot;Postgres&quot;)
  print(&quot;Connecting to Database…&quot;)
  connec &lt;- dbConnect(drv, 
                      dbname = dsn_database,
                      host = dsn_hostname, 
                      port = dsn_port,
                      user = dsn_uid, 
                      password = dsn_pwd)
  print(&quot;Database Connected!&quot;)
},
error=function(cond) {
  print(&quot;Unable to connect to Database.&quot;)
})

#Step 4: Run and Test Queries using RPostgres
dbSendQuery(connec, &quot;DROP TABLE IF EXISTS Population&quot;)  #Table name is Population

#Step 5: Importing
RPostgres::dbSendQuery(
  connec, 
  &quot;INSERT INTO mytable (Population) &#39;my_data_frame&#39;;&quot;,
)

It returns an error:

Error: Failed to prepare query: ERROR:  syntax error at or near &quot;&#39;my_data_frame&#39;&quot;
LINE 1: INSERT INTO mytable (Population) &#39;my_data_frame&#39;;
                               ^
In addition: Warning message:
In result_create(conn@ptr, statement, immediate) :
  Closing open result set, cancelling previous query

I have tried different ways but no success. I am really stuck with this task, Could anyone please help me in resolving the present code or to suggest a new code to copy paste different csv files to 1 SQL table using R. R and SQL both are new things for me.

答案1

得分: 1

远程 SQL 服务器无法感知或了解您的本地工作环境,因此传递一个字符串字面值 &#39;my_data_frame&#39; 将不起作用。(如果是这样的话,那将是一个 极大的 安全问题。)传递给它数据。

注意:单独看您的 SQL 语句是正确的,但不一致的 drop table if exists Population 暗示着存在一个名为 Population表格,但 insert into mytable (Population) ... 暗示着存在一个名为 mytable 的表格以及该表格内的一个名为 Population. 从语法上来说是正确的,当然也有可能存在两个表格 mytablePopulation,其中前者恰好有一个名为 Population 的列,但是... 这看起来可能是一个错误。

使用 dbAppendTable。以下是一个使用 mtcars 和本地 sqlite 的工作示例。

library(DBI)
sqlite &lt;- dbConnect(RSQLite::SQLite())
dbWriteTable(sqlite, &quot;mytable&quot;, mtcars[1,], create = TRUE)
dbAppendTable(sqlite, &quot;mytable&quot;, mtcars[2:4,1:3])            # &lt;-----
# [1] 3
dbGetQuery(sqlite, &quot;select * from mytable&quot;)
#    mpg cyl disp  hp drat   wt  qsec vs am gear carb
# 1 21.0   6  160 110  3.9 2.62 16.46  0  1    4    4
# 2 21.0   6  160  NA   NA   NA    NA NA NA   NA   NA
# 3 22.8   4  108  NA   NA   NA    NA NA NA   NA   NA
# 4 21.4   6  258  NA   NA   NA    NA NA NA   NA   NA
英文:

The remote SQL server has no visibility or awareness of your local working environment, so passing a string literal &#39;my_data_frame&#39; will not work. (It would be a huge security problem if that were the case.) Pass it the data.

Note: your SQL statements are fine by themselves but inconsistent drop table if exists Population suggests there is a table named Population, but insert into mytable (Population) ... suggests there is a table named mytable and a column named Population within that table. They are syntactically fine, and it is certainly feasible to have two tables mytable and Population where the former happens to have a column named Population, but ... it seems like it's likely to be a mistake.

Use dbAppendTable. Working example using mtcars and a local sqlite.

library(DBI)
sqlite &lt;- dbConnect(RSQLite::SQLite())
dbWriteTable(sqlite, &quot;mytable&quot;, mtcars[1,], create = TRUE)
dbAppendTable(sqlite, &quot;mytable&quot;, mtcars[2:4,1:3])            # &lt;-----
# [1] 3
dbGetQuery(sqlite, &quot;select * from mytable&quot;)
#    mpg cyl disp  hp drat   wt  qsec vs am gear carb
# 1 21.0   6  160 110  3.9 2.62 16.46  0  1    4    4
# 2 21.0   6  160  NA   NA   NA    NA NA NA   NA   NA
# 3 22.8   4  108  NA   NA   NA    NA NA NA   NA   NA
# 4 21.4   6  258  NA   NA   NA    NA NA NA   NA   NA

huangapple
  • 本文由 发表于 2023年7月18日 01:32:04
  • 转载请务必保留本文链接:https://go.coder-hub.com/76706840.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定