提高Python函数的速度

huangapple go评论69阅读模式
英文:

Increase Speed of Python Function

问题

我有一个感兴趣的区域。我想要应用相同的Python函数“calculate”100 * 100 = 10000次。

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pvlib.location import Location
from concurrent.futures import ThreadPoolExecutor

longitude_extent = (50, 60)
latitude_extent = (110, 120)

longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)

date_time = "2022-07-01 00:00:00"

def calculate(lat, lon):
   site = Location(lat, lon, 'utc', 0)
   solar_position = site.get_solarposition(date_time)['zenith']
   return solar_position

result = np.zeros((100, 100))
for ix, lon in enumerate(longitudes):
   for jx, lat in enumerate(latitudes):
        res = calculate(lat, lon)
        result[ix, jx] = res

However, it runs extremely slowly.
Any ideas to make it faster?
英文:

I have a region of interest.I wanted to apply the same python function "calculate" 100 * 100 = 10000 times.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from pvlib.location import Location
from concurrent.futures import ThreadPoolExecutor


longitude_extent = (50, 60)
latitude_extent = (110, 120)

longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)


date_time = "2022-07-01 00:00:00"

def calculate (lat, lon):
   site = Location(lat, lon, 'utc', 0)
   solar_position = site.get_solarposition(date_time)['zenith']
   return solar_position

result = np.zeros ((100, 100))
for ix, lon in enumerate(longitudes):
   for jx, lat in enumerate(latitudes):
        res = calculate(lat,lon)
        result[ix, jx] = res

However, it runs extremely slowly.
Any ideas to make it faster?

答案1

得分: 1

以下内容的翻译如下:

这个代码在我的机器上(2.3 GHz 8核 Intel Core i9)上使用ProcessPoolExecutor实现了大约5倍的加速。根据您的机器上处理器的数量,结果可能会有所不同:

from concurrent.futures import ProcessPoolExecutor
from itertools import product

import numpy as np
from pvlib.location import Location


longitude_extent = (50, 60)
latitude_extent = (110, 120)

longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)

date_time = "2022-07-01 00:00:00"


def calculate(pair):
    return Location(*pair).get_solarposition(date_time)['zenith']


if __name__ == '__main__':
    with ProcessPoolExecutor() as executor:
        result = executor.map(calculate, product(latitudes, longitudes))

    print(list(result))

一般来说,在尝试加速 Python 中的计算密集型工作负载时,您应该使用多进程而不是多线程,因为存在全局解释器锁

英文:

The following achieves a ~5x speedup on my machine (2.3 GHz 8-Core Intel Core i9) over the presented solution using ProcessPoolExecutor. YMMV depending on the number of processors on your machine:

from concurrent.futures import ProcessPoolExecutor
from itertools import product

import numpy as np
from pvlib.location import Location


longitude_extent = (50, 60)
latitude_extent = (110, 120)

longitudes = np.linspace(50, 60, 100)
latitudes = np.linspace(110, 120, 100)

date_time = "2022-07-01 00:00:00"


def calculate(pair):
    return Location(*pair).get_solarposition(date_time)['zenith']


if __name__ == '__main__':
    with ProcessPoolExecutor() as executor:
        result = executor.map(calculate, product(latitudes, longitudes))

    print(list(result))

In general, when trying to speed up compute-heavy workloads in Python, you're going to want to use multiprocessing instead of multithreading because of the Global Interpreter Lock.

huangapple
  • 本文由 发表于 2023年5月24日 20:02:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/76323313.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定