如何使用R将不同的CSV文件导入/复制到SQL中的一个表格中?

huangapple go评论101阅读模式
英文:

How to import/copy different csv files into 1 table in SQL using R?

问题

我想将这些CSV文件导入SQL数据库并创建一个表,使用R编程。我找不到一个可以完成这个任务的函数。请问有谁可以帮忙?

期望的输出(一个SQL表):

  1. City_Name Pop PopDen Lit
  2. MI 45000 280 78
  3. MI 37000 310 89
  4. ...
  5. ...
  6. CH 97000 150 74
  7. CH 67000 220 82
  8. ...
英文:

I have multiple csv files (1 file for each city) having following structure.

An example of two out of many files:

CSV File 1:

  1. City_Name Pop PopDen Lit
  2. MI 45000 280 78
  3. MI 37000 310 89
  4. ...
  5. ...

CSV File 2:

  1. City_Name Pop PopDen Lit
  2. CH 97000 150 74
  3. CH 67000 220 82
  4. ...
  5. ...

I want to import these csv files into SQL database as 1 table using R. I am not able to find a function to do it. Could anyone please help.

Desired Output (1 Table in SQL):

  1. City_Name Pop PopDen Lit
  2. MI 45000 280 78
  3. MI 37000 310 89
  4. ...
  5. ...
  6. CH 97000 150 74
  7. CH 67000 220 82
  8. ...

答案1

得分: 3

我对Windows不太熟悉,但由于它非常流行,你应该可以轻松找到循环遍历*.csv文件的示例。首先准备一个psql语句,用于从一个csv文件上传数据。它会类似于:

  1. psql -c '\copy your_table(City_Name,Pop, PopDen, Lit) FROM '/path/to/csv/.txt' WITH (FORMAT CSV)'

注意:你需要在Windows上输入psql实用程序的完整路径。

测试它是否适用于一个文件,然后在所有CSV文件上进行循环遍历(在cmd.exe窗口中):

  1. FOR %f in (dir /b *.csv) DO
  2. psql -c '\copy your_table(City_Name,Pop, PopDen, Lit) FROM '%f' WITH (FORMAT CSV)'
  3. DONE

大致类似于这样...

英文:

I am not so familiar with Windows, but since it's so popular, you shouldn't have any problem finding examples of looping over *.csv files. First prepare a psql statement that uploads the data from one csv. It will be something like:

  1. psql -c '\copy your_table(City_Name,Pop, PopDen, Lit) FROM '/path/to/csv/.txt' WITH (FORMAT CSV)'

Note: you'll have to enter the full path on Windows to the psql utility.

Test that it works for one file, then loop over all CSV files (in cmd.exe window):

  1. FOR %f in (dir /b *.csv) DO
  2. psql -c '\copy your_table(City_Name,Pop, PopDen, Lit) FROM '%f' WITH (FORMAT CSV)'
  3. DONE

Something along those lines...

答案2

得分: 1

使用read.csv()不是将多个大型CSV文件导入R数据框的好选择。为了在R中读取多个CSV文件或从文件夹中读取所有文件,请使用data.table包。data.table是一个第三方库,您需要首先使用install.packages('data.table')来安装它。

  1. # 使用data.table包
  2. library(data.table)
  3. df <-
  4. list.files(path = "/Users/admin/apps/csv-courses/", pattern = "*.csv") %>%
  5. map_df(~fread(.))
  6. df
英文:

Using read.csv() is not a good option to import multiple large CSV files into R Data Frame. In order to read multiple CSV files or all files from a folder in R, use data.table package. data.table is a third-party library, you need to first install it by using install.packages('data.table').

  1. # Use data.table package
  2. library(data.table)
  3. df &lt;-
  4. list.files(path = &quot;/Users/admin/apps/csv-courses/&quot;, pattern = &quot;*.csv&quot;) %&gt;%
  5. map_df(~fread(.))
  6. df

huangapple
  • 本文由 发表于 2023年7月14日 06:09:11
  • 转载请务必保留本文链接:https://go.coder-hub.com/76683545.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定