英文:
Increase Speed of Python Function
问题
我有一个感兴趣的区域。我想要应用相同的Python函数“calculate”100 * 100 = 10000次。
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pvlib.location import Location
from concurrent.futures import ThreadPoolExecutor
longitude_extent = (50, 60)
latitude_extent = (110, 120)
longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)
date_time = "2022-07-01 00:00:00"
def calculate(lat, lon):
site = Location(lat, lon, 'utc', 0)
solar_position = site.get_solarposition(date_time)['zenith']
return solar_position
result = np.zeros((100, 100))
for ix, lon in enumerate(longitudes):
for jx, lat in enumerate(latitudes):
res = calculate(lat, lon)
result[ix, jx] = res
However, it runs extremely slowly.
Any ideas to make it faster?
英文:
I have a region of interest.I wanted to apply the same python function "calculate" 100 * 100 = 10000 times.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pvlib.location import Location
from concurrent.futures import ThreadPoolExecutor
longitude_extent = (50, 60)
latitude_extent = (110, 120)
longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)
date_time = "2022-07-01 00:00:00"
def calculate (lat, lon):
site = Location(lat, lon, 'utc', 0)
solar_position = site.get_solarposition(date_time)['zenith']
return solar_position
result = np.zeros ((100, 100))
for ix, lon in enumerate(longitudes):
for jx, lat in enumerate(latitudes):
res = calculate(lat,lon)
result[ix, jx] = res
However, it runs extremely slowly.
Any ideas to make it faster?
答案1
得分: 1
以下内容的翻译如下:
这个代码在我的机器上(2.3 GHz 8核 Intel Core i9)上使用ProcessPoolExecutor
实现了大约5倍的加速。根据您的机器上处理器的数量,结果可能会有所不同:
from concurrent.futures import ProcessPoolExecutor
from itertools import product
import numpy as np
from pvlib.location import Location
longitude_extent = (50, 60)
latitude_extent = (110, 120)
longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)
date_time = "2022-07-01 00:00:00"
def calculate(pair):
return Location(*pair).get_solarposition(date_time)['zenith']
if __name__ == '__main__':
with ProcessPoolExecutor() as executor:
result = executor.map(calculate, product(latitudes, longitudes))
print(list(result))
一般来说,在尝试加速 Python 中的计算密集型工作负载时,您应该使用多进程而不是多线程,因为存在全局解释器锁。
英文:
The following achieves a ~5x speedup on my machine (2.3 GHz 8-Core Intel Core i9) over the presented solution using ProcessPoolExecutor
. YMMV depending on the number of processors on your machine:
from concurrent.futures import ProcessPoolExecutor
from itertools import product
import numpy as np
from pvlib.location import Location
longitude_extent = (50, 60)
latitude_extent = (110, 120)
longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)
date_time = "2022-07-01 00:00:00"
def calculate(pair):
return Location(*pair).get_solarposition(date_time)['zenith']
if __name__ == '__main__':
with ProcessPoolExecutor() as executor:
result = executor.map(calculate, product(latitudes, longitudes))
print(list(result))
In general, when trying to speed up compute-heavy workloads in Python, you're going to want to use multiprocessing instead of multithreading because of the Global Interpreter Lock.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论