如何通过Flutter从网站收集数据或进行网络抓取?

huangapple go评论71阅读模式
英文:

How to collect data from website or (web scraping) via flutter?

问题

我需要从网站https://ipcost.com进行网页抓取并收集数据,然后在我的Flutter应用程序中使用这些收集的数据(在这种情况下是我的IP详细信息)。

我的问题是如何获取这些信息并在用户界面中显示它。

英文:

I need to web scraping and collect data from the website https://ipcost.com and use that collected data (in this case, my IP details) in my Flutter app.

My question is how to get this information and display it in the UI.

如何通过Flutter从网站收集数据或进行网络抓取?

答案1

得分: 0

I tried the web_scraper package and it solved the problem.

Step 1 : 将 web_scraper 添加到依赖项中。

Step 2 : 检查网站并获取网站上所需信息的放置模式。在我的情况下,“Titles” 的模式是:

'div.e > small'

对于“Subtitle”,模式是:

'div.e > strong'

Step 3 : 我创建了一个 Webscraper 类的对象,并通过设置网站 URL 的值来定义它。

Step 4-1 : 在我的情况下,我不需要 urlPath,将其设置为空。

  final String urlPath = '';

Step 4-2 : 在我的情况下,Attributes 总是为空,我不需要它们。

  final List<String> attributeList = [''];

Step 5 : 我定义了一个方法,并在 initState 中调用它,以从网站获取值。

  void fetchData() async {
    if (await webScraper.loadWebPage(urlPath)) {
      setState(() {
        webScrapeResultTitle =
            webScraper.getElement(innerAddressTitle, attributeList);
        webScrapeResultDescription =
            webScraper.getElement(innerAddressDescription, attributeList);

        ///TODO: 通过查看这些打印信息,我确定了接收到的数据结构。

        // print("webScrapeResultTitle::::${webScrapeResultTitle!.toList()}");
        // print("webScrapeResultDescription::::${webScrapeResultDescription!.toList()}");
      });
    }
  }

Step 6 : 设置构建方法如下以显示结果。

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: webScrapeResultTitle == null || webScrapeResultDescription == null
          ? Center(
        child: LoadingWidget(),
      )
          : ListView.builder(
          itemCount: webScrapeResultDescription!.length,
          itemBuilder: (BuildContext context, int index) {
            return IpInfoItem(
                titleResult: webScrapeResultTitle![index][titleResultKey],
                descriptionResult: webScrapeResultDescription![index]
                [descriptionResultKey]);
          }),
    );
  }
}

如何通过Flutter从网站收集数据或进行网络抓取?

从这里查看我的完整代码

Gist全文段:Gist.Github.com

仓库:Github.com

英文:

I tried the web_scraper package and it solved the problem.

Step 1 : add web_scraper to dependencies .

Step 2 : Checking the website and getting the pattern of placing the desired information on the website. in my case
for "Titles" , pattern is :

&#39;div.e &gt; small&#39;

for "Subtitle" , pattern is :

&#39;div.e &gt; strong&#39;

Step 3 : I created an object of the Webscraper class and defined it by setting the value of the website URL.

Step 4-1 :
In my case , I don't need urlPath and set it empty .

  final String urlPath = &#39;&#39;;

Step 4-2 : In my case , Attributes always null and I don't need them.

  final List&lt;String&gt; attributeList = [&#39;&#39;];

Step 5 : I define one method and call them in initState , for get values from website .

  void fetchData() async {
    if (await webScraper.loadWebPage(urlPath)) {
      setState(() {
        webScrapeResultTitle =
            webScraper.getElement(innerAddressTitle, attributeList);
        webScrapeResultDescription =
            webScraper.getElement(innerAddressDescription, attributeList);


        ///TODO:  By review this prints  , I detect structure of recievied data .


        // print(&quot;webScrapeResultTitle::::${webScrapeResultTitle!.toList()}&quot;);
        // print(
        //     &quot;webScrapeResultDescription::::${webScrapeResultDescription!.toList()}&quot;);
      });
    }
  }

Step 6 : set build method like below for show results .

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: webScrapeResultTitle == null || webScrapeResultDescription == null
          ? Center(
        child: LoadingWidget(),
      )
          : ListView.builder(
          itemCount: webScrapeResultDescription!.length,
          itemBuilder: (BuildContext context, int index) {
            return IpInfoItem(
                titleResult: webScrapeResultTitle![index][titleResultKey],
                descriptionResult: webScrapeResultDescription![index]
                [descriptionResultKey]);
          }),
    );
  }
}

如何通过Flutter从网站收集数据或进行网络抓取?

See my full code from here

Gist full Snippets: Gist.Github.com

Repository : Github.com

huangapple
  • 本文由 发表于 2023年1月6日 10:53:05
  • 转载请务必保留本文链接:https://go.coder-hub.com/75026492.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定