如何使用Python爬取多个网页并将它们从英语翻译成印地语？

huangapple
go评论97阅读模式
英文:

How to scrape multiple web-pages that will translate them from English to Hindi using python?

问题

I am struggling with the small issue, the code works and no errors. But I need to figure out how to translate multiple pages from the website. From English to Hindi and each pages has to be Hindi, so far I only translated one specific text from the main website.

  1. #Script scraps the website using request and beautifulSoup library
  3. from google_translate import browser
  4. from google_translate import selenium
  5. import requests
  6. from bs4 import BeautifulSoup
  7. URL = "https://www.classcentral.com/?"
  8. headers = {'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246"}
  9. # Here the user agent is for Edge browser on windows 10. You can find your browser user agent from the above given link.
  10. r = requests.get(url=URL, headers=headers)
  11. print(r.content)
  12. # Parsing the HTML
  13. soup = BeautifulSoup(r.content, 'html.parser')
  14. # find all the anchor tags with "href"
  15. for link in soup.find_all('a'):
  16.     print(link.get('href'))
  1. #Script transalate text into Hindi using google translate API
  3. import time
  4. from selenium import webdriver
  5. from selenium.webdriver.common.by import By
  6. import selenium
  7. # Give Language code in which you want to translate the text:=>
  8. lang_code = 'hi '
  10. # Provide text that you want to translate:=>
  11. input1 = " Find your next course.Class Central aggregates courses from many providers to help you find the best courses on almost any subject, wherever they exist"
  13. # launch browser with selenium:=>
  14. browser = webdriver.Chrome() #browser = webdriver.Chrome('path of chromedriver.exe file') if the chromedriver.exe is in different folder
  16. # copy google Translator link here:=>
  17. browser.get("https://translate.google.co.in/?sl=auto&tl="+lang_code+"&text="+input1+"&op=translate")
  19. # just wait for some time for translating input text:=>
  20. time.sleep(6)
  22. # Given below x path contains the translated output that we are storing in output variable:=>
  23. output1 = browser.find_element(By.CLASS_NAME,'HwtZe').text
  25. # Display the output:=>
  26. print("Translated Paragraph=> " + output1)
英文:

I am struggling with the small issue, the code works and no errors. But I need to figure out how to translate multiple pages from the website. From English to Hindi and each pages has to be Hindi, so far I only translated one specific text from the main website.

  1. #Script scraps the website using request and beautifulSoup library
  3. from google_translate import browser
  4. from google_translate import selenium
  5. import requests
  6. from bs4 import BeautifulSoup
  7. URL = "https://www.classcentral.com/?"
  8. headers = {'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36 Edge/12.246"}
  9. # Here the user agent is for Edge browser on windows 10. You can find your browser user agent from the above given link.
  10. r = requests.get(url=URL, headers=headers)
  11. print(r.content)
  12. # Parsing the HTML
  13. soup = BeautifulSoup(r.content, 'html.parser')
  14. # find all the anchor tags with "href"
  15. for link in soup.find_all('a'):
  16.     print(link.get('href'))
  1. #Script transalate text into Hindi using google translate API
  3. import time
  4. from selenium import webdriver
  5. from selenium.webdriver.common.by import By
  6. import selenium
  7. # Give Language code in which you want to translate the text:=>
  8. lang_code = 'hi '
  10. # Provide text that you want to translate:=>
  11. input1 = " Find your next course.Class Central aggregates courses from many providers to help you find the best courses on almost any subject, wherever they exist"
  13. # launch browser with selenium:=>
  14. browser = webdriver.Chrome() #browser = webdriver.Chrome('path of chromedriver.exe file') if the chromedriver.exe is in different folder
  16. # copy google Translator link here:=>
  17. browser.get("https://translate.google.co.in/?sl=auto&tl="+lang_code+"&text="+input1+"&op=translate")
  19. # just wait for some time for translating input text:=>
  20. time.sleep(6)
  22. # Given below x path contains the translated output that we are storing in output variable:=>
  23. output1 = browser.find_element(By.CLASS_NAME,'HwtZe').text
  25. # Display the output:=>
  26. print("Translated Paragraph:=> " + output1)

答案1

得分: 1

Google翻译存在一些限制。根据我的理解，您无法在单个请求中翻译所有字符。因此，我建议您将文本拆分成多个请求进行翻译。

在下面的代码中，我使用了googletrans模块，在从网站获取文本后将其翻译成印地语。作为替代方案，您可以尝试以下代码，希望对您有所帮助：

  1. import requests
  2. from bs4 import BeautifulSoup
  3. from googletrans import Translator
  5. translator = Translator()
  7. def scrape_web_page(url):
  8.     response = requests.get(url)
  9.     soup = BeautifulSoup(response.content, 'html.parser')
  10.     text = soup.get_text()
  11.     return text
  13. def language_translator(urls):
  14.     count = 1
  15.     for url in urls:
  16.         new_text = ""
  17.         new_text = str(f"从页面 {count}")
  18.         print(f'.........................从页面 {count} ...........................................')
  19.         text = scrape_web_page(url)
  20.         k = text.split()
  21.         for i in k:
  22.             # print(i)
  23.             translated_text = translator.translate(i, dest='hi')
  24.             new_text = new_text + " " + str(translated_text.text)
  25.             # print(translated_text.text)
  26.         count = count + 1
  27.         print(new_text)
  29. urls = [
  30.     'https://demo1/page1', 'https://demo1/page2'
  31. ]
  33. language_translator(urls)

注意：网站抓取涉及一些版权问题。

英文:

Google translate has some limitations. Based on my understanding you can’t translate all the characters in a single request. So I recommend you to translate the text in multiple requests.

In the below code, I am using the googletrans module and after fetching text from the website I am translating them into Hindi. As an alternative you can try below code,I hope this will helpful for you:

  1. import requests
  2. from bs4 import BeautifulSoup
  3. from googletrans import Translator
  5. translator = Translator()
  7. def scrape_web_page(url):
  8.     response = requests.get(url)
  9.     soup = BeautifulSoup(response.content, 'html.parser')
  10.     text = soup.get_text()
  11.     return text
  13. def language_translator(url):
  14.     count=1
  15.     for url in urls:
  16.         newtest=""
  17.         newtest=str(f"From page {count}")
  18.         print(f'.........................from page {count} ...........................................')
  19.         text = scrape_web_page(url)
  20.         k=text.split()
  21.         for i in k:
  22.             #print(i)
  23.             translated_text = translator.translate(i, dest='hi')
  24.             newtest=newtest+" "+str(translated_text.text)
  25.             #print(translated_text.text)
  26.         count=count+1
  27.         print(newtest)
  31. urls = [
  32.     'https://demo1/page1','https://demo1/page2'
  33.     
  34. ]
  36. language_translator(urls)

NB: There are some copyright issues involved in website scraping.

huangapple
  • 本文由 发表于 2023年2月27日 11:56:06
  • 转载请务必保留本文链接：https://go.coder-hub.com/75576659.html
  • google-translation-api
  • python
  • selenium-webdriver
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定