如何使用rvest包在R中从网络抓取的数据中提取和显示图像？

huangapple
go评论92阅读模式
英文:

How can I extract and display images from web scraped data in R using rvest package?

问题

我目前正在尝试从维基百科页面中抓取美国所有法学院的名称和徽标。我已经成功创建了一个包含大学名称和图像链接的表格。但是，我想创建一个包含实际图像的大学名称表格。我正在使用R编程。

  1. library(tidyverse)
  2. library(rvest)
  4. # 读取数据
  5. url = 'https://en.wikipedia.org/wiki/List_of_law_schools_in_the_United_States'
  6. school_url = 'https://en.wikipedia.org/wiki/List_of_law_schools_in_the_United_States' %>%
  7.   read_html() %>%
  8.   html_elements('.wikitable a') %>%
  9.   html_attr('href')
  10. school_url = paste('https://en.wikipedia.org/', school_url, sep='')
  12. top3_school_urls = head(school_url, 3)
  14. # 网页抓取
  15. results = list()
  16. for (school_url in top3_school_urls) {
  17.   message('Scraping URL: ', school_url)
  18.   school_html = read_html(school_url) 
  19.   School = school_html %>%
  20.     html_element('h1 .mw-page-title-main') %>%
  21.     html_text2()
  22.   Logo = school_html %>%
  23.     html_element('.infobox-image a') %>%
  24.     html_attr('href')
  25.   Logo = paste('https://en.wikipedia.org/', Logo, sep='')
  27.   school_tibble = tibble(School, Logo)
  28.   results[[school_url]] = school_tibble
  29. }
  31. d = bind_rows(results, .id = 'url')
  32. d

我已经将数据读入R并使用rvest包解析了数据。我成功提取了图像的链接，但我想更进一步，将实际图像放入表格中。

英文:

I am currently trying to scrape the names and logos of all the Law Schools in the U.S. from a wiki page. I was able to create a table of university names and image links. However, I would like to create a table of university names with the actual images. I am using R programming.

  1. library(tidyverse)
  2. library(rvest)
  3. \###Reading in the Data
  4. url = 
  5. 'https://en.wikipedia.org/wiki/List_of_law_schools_in_the_United_States'
  6. school_url = 
  7. 'https://en.wikipedia.org/wiki/List_of_law_schools_in_the_United_States' 
  8.    %\>%
  9. read_html() %\>%
  10. html_elements('.wikitable a') %\>%
  11. html_attr('href')
  12. school_url = paste('https://en.wikipedia.org/', school_url, sep='')
  14. top3_school_urls = head(school_url, 3)
  16. ###Web Scraper
  17. results = list()
  18. for (school_url in top3_school_urls) {
  19.   message('Scraping URL: ', school_url)
  20.   school_html = read_html(school_url) 
  21.   School = school_html %>% 
  22.     html_element('h1 .mw-page-title-main') %>% html_text2()
  23.   Logo = school_html %>% html_element('.infobox-image a') %>% 
  24.           html_attr('href')
  25.   Logo = paste('https://en.wikipedia.org/', Logo, sep='')
  27.   school_tibble = tibble(School, Logo)
  28.   results[[school_url]] = school_tibble
  29. }
  31. d = bind_rows(results, .id = 'url')
  32. d`

I read the data into R and parsed the data using the rvest package. I was able to extract the links to the image but I would like to take it a step further and have the actual images in the table.

答案1

得分: 1

一个快速简单的选择是 kableExtra：

  1. ---
  2. title: "法学院"
  3. output: html_document
  4. ---
英文:

One quick and easy option is kableExtra:

  1. ---
  2. title: "law schools"
  3. output: html_document
  4. ---
  5. ```{r echo=FALSE, warning=FALSE}
  6. suppressPackageStartupMessages({
  7.   library(rvest)
  8.   library(stringr)
  9.   library(purrr)
  10.   library(dplyr)
  11.   library(kableExtra)
  12. })
  14. mw_api <- list(page  = "https://en.wikipedia.org/api/rest_v1/page/html/",
  15.                media = "https://en.wikipedia.org/api/rest_v1/page/media-list/")
  17. top_3 <- read_html(paste0(mw_api$page, "List_of_law_schools_in_the_United_States")) %>% 
  18.   # extract link elements only from 2nd column of the table
  19.   html_elements("table.wikitable tbody tr > td:nth-child(2) > a") %>% 
  20.   # keep only top 3
  21.   head(3) %>% 
  22.   # get link / wiki title and link text from single elements
  23.   map(~ tibble::tibble_row( wikititle = html_attr(.x, "href") %>% str_remove("^./"),
  24.                             title = html_text(.x) %>% str_trim())
  25.   ) %>% list_rbind() %>% 
  26.   # request media list for titles
  27.   mutate(logo = map_chr(wikititle, ~ paste0(mw_api$media,.x) %>% 
  28.                           jsonlite::read_json() %>% 
  29.                           pluck("items", 1, "srcset", 1, "src")),
  30.          logo = paste0("https:", logo))
  32. top_3 %>% 
  33.   mutate(logo = "") %>% 
  34.   kbl(booktabs = T) %>%
  35.   kable_paper(full_width = FALSE) %>%
  36.   column_spec(3, image = top_3$logo)
  38. top_3
  39. #> # A tibble: 3 × 3
  40. #>   wikititle                title                    logo                        
  41. #>   <chr>                    <chr>                    <chr>                       
  42. #> 1 Birmingham_School_of_Law Birmingham School of Law https://upload.wikimedia.or…
  43. #> 2 Cumberland_School_of_Law Cumberland School of Law https://upload.wikimedia.or…
  44. #> 3 Samford_University       Samford University       https://upload.wikimedia.or…
  45. ```

Renders as:
如何使用rvest包在R中从网络抓取的数据中提取和显示图像？

When rendering to pdf, files must be downloaded first:

  1. dir.create("tmp_img/")
  2. # Download all files listed in top_3$logo and updated 
  3. # logo to include local files paths instead of urls
  4. top_3 <- top_3 %>% 
  5.   mutate(logo = map_chr(logo, ~ {
  6.     destfile = file.path("tmp_img", basename(.x))
  7.     download.file(.x, destfile = destfile, mode = "wb")
  8.     destfile
  9.     }))
  11. top_3 %>% 
  12.   mutate(logo = "") %>% 
  13.   kbl(booktabs = T) %>%
  14.   # we can use "scale_down" for slightly better fit, but most likely  
  15.   # it needs some further tweaking
  16.   kable_styling(latex_options = c("scale_down")) %>%
  17.   column_spec(3, image = top_3$logo)

huangapple
  • 本文由 发表于 2023年5月21日 07:34:41
  • 转载请务必保留本文链接：https://go.coder-hub.com/76297738.html
  • r
  • web
  • web-scraping
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定