从我的网页中抓取数值并存储在SQLite数据库中。

huangapple go评论57阅读模式
英文:

Webscrape value from my webpage and store in database SQlite

问题

我想从我的网页中进行网页抓取然后每隔5分钟将其发送到一个SQLite数据库
我已经完成了Python的网页抓取

```python
import requests
from bs4 import BeautifulSoup
URL = "http://ludvikasegel.com/wx/cloudbase.asp"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
molnbas = soup.find_all("div", class_="cloudbase")
print (molnbas[0].text.strip())

现在我困在如何将其存储在SQLite数据库中。
在我的数据库中,我想要该值以及一些时间戳。
有任何关于如何做或者任何好的简单教程的建议吗?

谢谢

我已经查看了一些教程,但仍然不确定如何做。


<details>
<summary>英文:</summary>

I want to webscrape a value from my webpage and send it to a SQlite database every 5 minutes
I have done the webscrape python:

import requests
from bs4 import BeautifulSoup
URL = "http://ludvikasegel.com/wx/cloudbase.asp"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
molnbas = soup.find_all("div", class_="cloudbase")
print (molnbas[0].text.strip())


Now I&#39;m stucked how to store it in a SQlite database.
In my database I want the value and a some timestamp.
Any suggestions how to do it or any good simple tutorials?

Thanx

I have checked some tutorials but am still unsure how to do it.

</details>


# 答案1
**得分**: 1

首先创建一个名为 `schema.sql` 的文件。理论上,这可以直接放入 `init_db.py` 中,但我认为最好将其分离以提高代码的清晰度。
```sql
DROP TABLE IF EXISTS table_name;

CREATE TABLE table_name (
    value INTEGER,
    timestamp TEXT
);

然后创建一个名为 init_db.py 的文件。

import sqlite3

with sqlite3.connect('database.db') as con, open("schema.sql") as f:
    con.executescript(f.read())

将以下内容添加到每隔5分钟运行的脚本中。

import datetime 

desired_value = molnbas[0].text.strip()
timestamp = datetime.datetime.utcnow().isoformat()  # UTC时间戳,根据需要更改

with sqlite3.connect("database.db") as sqlite_connection:
    sqlite_connection.execute(
        "INSERT INTO table_name VALUES (?, ?)",
        (desired_value, timestamp)
    )
英文:

First make a schema.sql file. This can in theory be put directly into init_db.py but it's (in my opinion) good practise to separate it out for clarity.

DROP TABLE IF EXISTS table_name;

CREATE TABLE table_name (
    value INTEGER,
    timestamp TEXT
);

Then make an init_db.py file

import sqlite3

with sqlite3.connect(&#39;database.db&#39;) as con, open(&quot;schema.sql&quot;) as f:
    con.executescript(f.read())

Add this to your script that is run every 5th minute.

import datetime 

desired_value = molnbas[0].text.strip()
timestamp = datetime.datetime.utcnow().isoformat()  # UTC timestamp, change to whatever you want

with sqlite3.connect(&quot;database.db&quot;) as sqlite_connection:
    sqlite_connection.execute(
        &quot;INSERT INTO table_name VALUES (?, ?)&quot;,
        (desired_value, timestamp)
    )

huangapple
  • 本文由 发表于 2023年7月10日 19:45:10
  • 转载请务必保留本文链接:https://go.coder-hub.com/76653407.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定