英文:
How do you make faster A.P.I calls using multithreading without using requests in Python?
问题
我正在尝试获取标普500指数中每家公司的历史股票数据。问题是获取数据花费了很长时间。
from ApiStuff import ApiStuff
import fundamentalanalysis as fa
import pickle
tickers = pickle.load(open('S&P500_TICKERS.dat','rb'))
api_key = ApiStuff.api_key
data_from_tickers = []
for ticker in tickers:
balance_sheet_annually = fa.balance_sheet_statement(ticker, api_key, period="annual")
data_from_tickers.append(balance_sheet_annually)
我尝试在互联网上搜索如何加快速度,但它们使用其他模块(例如requests、aiohttp)来加快数据的检索,而我依赖于这个模块(fundamentalanalysis)来检索基本数据。
有没有办法让我继续使用这个模块,并通过描述的方法加快API请求的速度?
英文:
I'm trying to receive historical stock data for every company in the S&P 500. The problem is that it is taking a really longtime to get the data.
from ApiStuff import ApiStuff
import fundamentalanalysis as fa
import pickle
tickers = pickle.load(open('S&P500_TICKERS.dat','rb'))
api_key = ApiStuff.api_key
data_from_tickers = []
for ticker in tickers:
balance_sheet_annually = fa.balance_sheet_statement(ticker, api_key, period="annual")
data_from_tickers.append(balance_sheet_annually)
I tried searching on the internet on how to speed it up but they use other modules (i.e requests, aiohttp) for making the retrieval of the data faster and I am dependent on this module (fundamentalanalysis) to retrieve fundamental data.
Is there a way for me to still use this module and make api requests faster via the methods described?
答案1
得分: 1
可以使用多个进程来完成这个任务;concurrent.futures
正是为这种需求而设计的。另一方面,这也是一个学习开源的绝佳机会。fundamentalanalysis
的源代码可以在 Github 上找到。你正在使用的函数,balance_sheet_statement
,非常简单,基本上由一个 GET
请求、一些数据映射和构建 Pandas 数据帧组成。
使用 aiohttp
或 requests
复制这个逻辑要比处理多进程模块容易得多!
英文:
You certainly can do this with multiple processes; concurrent.futures
is made for this type of need. On the other hand, this is also a great learning opportunity for the use of open source. The source for fundamentalanalysis
is available on Github. The function you're using, balance_sheet_statement
, is very straightforward and basically consists of a GET
request, a couple of data mappings, and the construction of a Pandas dataframe.
Replicating this logic using aiohttp
or requests
is going to be easier than wrangling the multiprocessing modules!
答案2
得分: 1
如果 fundamentalanalysis
支持多线程,可以通过以下方式替换 for 循环来实现:
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=10) as e:
data_from_tickers = list(e.map(lambda t: fa.balance_sheet_statement(t, api_key, period="annual"), tickers))
最大工作线程数可以进行调整。
英文:
If fundamentalanalysis
supports multithreading, it can be done by replacing the for-loop with:
from concurrent.futures import ThreadPoolExecutor
with ThreadPoolExecutor(max_workers=10) as e:
data_from_tickers = list(e.map(lambda t: fa.balance_sheet_statement(t, api_key, period="annual"), tickers))
The maximum number of workers can be adjusted.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论