how to get image source from icrawler

I am using icrawler in a project.
icrawler is a good tool for getting image from google search but how to get the image's url,
I noticed that the url I want is displayed in terminal :
the output from the terminal

2023-05-06 20:25:22,389 - INFO - parser - parsing result page https://www.google.com/search?q=programing+gif&ijn=0&start=0&tbs=&tbm=isch
2023-05-06 20:25:23,579 - INFO - downloader - image #1  https://cdn.videoplasty.com/animation/chill-coding-programming-lo-fi-animation-stock-animation-21874-1024x576.jpg

Following Swifty's comment the following is an approach to capture and filter the logging object used by icrawler to print the messages.

There are different logging objects for the various components of the crawler. The download messages are logged by your_crawler.downloader.logger. You can add a function to process the log messages using some_logger.addFilter(function_name). Note that this filter doesn't print the message by default, so if you still want to see the message in the terminal then you should print it from the function.

The function below uses a regular expression to search for the string image #n followed by the url. Then the url is stored in a list which is printed.

In the example I have used a GreedyImageCrawler because I couldn't get Google or Bing to work. The GreedyImageCrawler also didn't work properly (it doesn't exit when the maximum number of images is downloaded) but it does provide a working proof of concept for the url extraction.

import re
import sys
from icrawler.builtin import GreedyImageCrawler

all_urls = []

def checkCrawlURL(log_input):
    # re-print captured log message
    print("INFO - downloader -", log_input.getMessage())
    # extract url
    res = re.search(r"image #\d+\t(.*)", log_input.getMessage())
    if res:
       # add extracted url to list
       all_urls.append(res.group(1))
       print(all_urls)

greedy_crawler = GreedyImageCrawler(storage={'root_dir': 'image_dir'})
# pass download logger messages through function
greedy_crawler.downloader.logger.addFilter(checkCrawlURL)
greedy_crawler.crawl(domains='http://www.bbc.com/news', max_num=10,
                     min_size=None, max_size=None)

Result:

2023-05-06 19:18:23,031 - INFO - icrawler.crawler - start crawling...
2023-05-06 19:18:23,032 - INFO - icrawler.crawler - starting 1 feeder threads...
2023-05-06 19:18:23,032 - INFO - icrawler.crawler - starting 1 parser threads...
2023-05-06 19:18:23,033 - INFO - icrawler.crawler - starting 1 downloader threads...
2023-05-06 19:18:23,187 - INFO - parser - parsing result page http://www.bbc.com/news
INFO - downloader - image #1	https://ichef.bbci.co.uk/news/320/cpsprodpb/CCDA/production/_129624425_gettyimages-1487975069.jpg
['https://ichef.bbci.co.uk/news/320/cpsprodpb/CCDA/production/_129624425_gettyimages-1487975069.jpg']
INFO - downloader - image #2	https://ichef.bbci.co.uk/news/320/cpsprodpb/11A78/production/_129621327_f5b5bd4421299cab6e7587311796a63f8808b7db-1.jpg
['https://ichef.bbci.co.uk/news/320/cpsprodpb/CCDA/production/_129624425_gettyimages-1487975069.jpg', 'https://ichef.bbci.co.uk/news/320/cpsprodpb/11A78/production/_129621327_f5b5bd4421299cab6e7587311796a63f8808b7db-1.jpg']
[...]