2023年5月13日 20:52:35go评论84阅读模式

英文:

Scraping data from this site using scrapy

问题

I am a Chinese translation tool, and I'll provide the translated parts as requested:

"我是数据爬取的新手，正在学习如何操作。我打算从这个网站 https://www.twhouse.co.uk/index.php?route=product/catalog 中提取数据数值，我正在使用 Scrapy Shell 进行查询和组装我的爬虫。当我执行 response.css('div.caption span.stat-1').get() 时，我得到了以下结果：SKU: 8644。我想提取 SKU 的值。谢谢大家的支持。

在 Scrapy Shell 中，我想查询网址 https://www.twhouse.co.uk/index.php?route=product/catalog，执行 response.css('div.caption span.stat-1').get() 后，我得到了以下结果：SKU: 8644。我只想要这个中的 SKU 值。当我尝试改成 response.css('div.caption span.stats-label').get() 时，我得到了这个结果 SKU:，当我加入 '::text' 时，即 response.css('div.caption span.stats-label::text').get()，我得到了响应 'SKU:'，而不是 SKU 的值。我该如何获取值？"

Please note that I've provided the translation without additional content and have excluded the part where you asked not to answer the translation question. If you have any more specific translation requests, feel free to ask.

英文:

I am noob to data scraping and learning the rope
i am going to scrape data values from this website, https://www.twhouse.co.uk/index.php?route=product/catalog, i am using scrapy shell to interogate and assemble my crawler .
When i initiate response, response.css('div.caption span.stat-1').get()
i got this
SKU: 8644'
i want to extract the value of the sku.
thank you all for your support

from scrapy shell i want to interrogate the url https://www.twhouse.co.uk/index.php?route=product/catalog
response.css('div.caption span.stat-1').get()
gave me this,

'SKU: 8644'
i want only the sku value from this, when i change is to response.css('div.caption span.stats-label').get()
i got this SKU:', when i inserted '::text' response.css('div.caption span.stats-label::text').get() i got this response 'SKU:' , not the sku value. how do i get the value

答案1

得分: 2

以下是翻译好的内容：

HTML如下：

...
...
<span class="stat-1">
    <span class="stats-label">SKU:</span>
    <span>8811</span>
</span>
...
...

所以你想要获取外部span标签（"stat-1"）内的第二（最后一个）span标签。

Python代码如下：

scrapy shell https://www.twhouse.co.uk/index.php?route=product/catalog

>>> response.css('div.caption span.stat-1 span:last-child::text').get()
'8811'

如果你想要获取所有的文本，你可以使用 getall()，这样你会得到一个列表。

scrapy shell https://www.twhouse.co.uk/index.php?route=product/catalog

>>> response.css('div.caption span.stat-1 span:last-child::text').getall()
['8811', '8943', '8939', '8730', '8853', '8748', '8901', '8756', '8855', '8951', '8838', '8857', '8934', '8856', '8924', '9050', '8862', '8863', '8764', '9047', '9045', '9055', '8746', '8814', '8714', '8760', '8944', '8958', '8959', '8722', '8743', '8785', '8946', '8860', '8877', '8715', '9011', '8945', '9023', '8947', '9015', '8777', '8753', '8797', '8899', '8734', '8705', '9042', '8936', '8787', '8950', '8888', '8723', '9018', '9019', '8948', '8942', '8890', '8969', '8906', '8907', '8960', '9021', '8713', '9009', '9014', '9022', '8831', '8707', '8724', '9033', '9024', '9038', '8829', '9034', '9027', '9025', '9031', '9026', '9029', '9030', '9032', '9041', '9039', '9051', '9028', '8994', '8765', '8977', '8808', '8978', '8809', '8876', '9008', '8883', '8768', '8823', '8740', '8873', '9013']

英文:

The HTML looks like this:

...
...
&lt;span class=&quot;stat-1&quot;&gt;
    &lt;span class=&quot;stats-label&quot;&gt;SKU:&lt;/span&gt;
    &lt;span&gt;8811&lt;/span&gt;
&lt;/span&gt;
...
...

So you want to get the second (last) span tag inside the outer span tag ("stat-1").

scrapy shell https://www.twhouse.co.uk/index.php?route=product/catalog

&gt;&gt;&gt; response.css(&#39;div.caption span.stat-1 span:last-child::text&#39;).get()
&#39;8811&#39;

If you want to get all the text you can use getall() and you'll get them as a list.

scrapy shell https://www.twhouse.co.uk/index.php?route=product/catalog

&gt;&gt;&gt; response.css(&#39;div.caption span.stat-1 span:last-child::text&#39;).getall()
[&#39;8811&#39;, &#39;8943&#39;, &#39;8939&#39;, &#39;8730&#39;, &#39;8853&#39;, &#39;8748&#39;, &#39;8901&#39;, &#39;8756&#39;, &#39;8855&#39;, &#39;8951&#39;, &#39;8838&#39;, &#39;8857&#39;, &#39;8934&#39;, &#39;8856&#39;, &#39;8924&#39;, &#39;9050&#39;, &#39;8862&#39;, &#39;8863&#39;, &#39;8764&#39;, &#39;9047&#39;, &#39;9045&#39;, &#39;9055&#39;, &#39;8746&#39;, &#39;8814&#39;, &#39;8714&#39;, &#39;8760&#39;, &#39;8944&#39;, &#39;8958&#39;, &#39;8959&#39;, &#39;8722&#39;, &#39;8743&#39;, &#39;8785&#39;, &#39;8946&#39;, &#39;8860&#39;, &#39;8877&#39;, &#39;8715&#39;, &#39;9011&#39;, &#39;8945&#39;, &#39;9023&#39;, &#39;8947&#39;, &#39;9015&#39;, &#39;8777&#39;, &#39;8753&#39;, &#39;8797&#39;, &#39;8899&#39;, &#39;8734&#39;, &#39;8705&#39;, &#39;9042&#39;, &#39;8936&#39;, &#39;8787&#39;, &#39;8950&#39;, &#39;8888&#39;, &#39;8723&#39;, &#39;9018&#39;, &#39;9019&#39;, &#39;8948&#39;, &#39;8942&#39;, &#39;8890&#39;, &#39;8969&#39;, &#39;8906&#39;, &#39;8907&#39;, &#39;8960&#39;, &#39;9021&#39;, &#39;8713&#39;, &#39;9009&#39;, &#39;9014&#39;, &#39;9022&#39;, &#39;8831&#39;, &#39;8707&#39;, &#39;8724&#39;, &#39;9033&#39;, &#39;9024&#39;, &#39;9038&#39;, &#39;8829&#39;, &#39;9034&#39;, &#39;9027&#39;, &#39;9025&#39;, &#39;9031&#39;, &#39;9026&#39;, &#39;9029&#39;, &#39;9030&#39;, &#39;9032&#39;, &#39;9041&#39;, &#39;9039&#39;, &#39;9051&#39;, &#39;9028&#39;, &#39;8994&#39;, &#39;8765&#39;, &#39;8977&#39;, &#39;8808&#39;, &#39;8978&#39;, &#39;8809&#39;, &#39;8876&#39;, &#39;9008&#39;, &#39;8883&#39;, &#39;8768&#39;, &#39;8823&#39;, &#39;8740&#39;, &#39;8873&#39;, &#39;9013&#39;]

答案2

得分: 0

尝试访问 stat-1 中的第二个 span 子元素

response.css('div.caption span.stat-1 span:nth-child(2)::text').get()

英文:

Try to access second span child in stat-1

response.css(&#39;div.caption span.stat-1 span:nth-child(2)::text&#39;).get()

通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。

从该网站使用Scrapy爬取数据

问题

答案1

答案2

PyCharm 无法在导入时看到我新编译的 .pyc 文件。

在 dbc checklist – dash 中插入空格并包裹文本。

HttpResponseError: 此请求未获授权以使用Python Azure Function中的此权限执行此操作

Getting requirements to build wheel-error Pygame on Windows

What's the correct way to type hint an empty list as a literal in python?

如何在Highcharts Gantt中更改本地化的星期名称

如何在同一个流中使用多个过滤器和映射函数？

如何使用Map/Set来将代码优化到O(n)？

.NET MAUI Android在GitHub Actions上构建失败，错误代码为1。

如何在Playwright视觉比较中屏蔽多个定位器？

在C++中，可以使用可变模板参数来检索类型的内部类型。

selenium.common.exceptions.StaleElementReferenceException: Message: stale element reference: stale element not found

Creating and opening a URL to log in to Website via Basic Auth with Robot Framework/Selenium (Python)

AG Grid 在上下文菜单中以大文本形式打开

发表评论