如何避免重新计算多个变量，而不完全重构代码

huangapple
go评论94阅读模式
英文:

How to avoid recomputing several variables without completely refactoring code

问题

我明白你的要求，以下是你提供的代码的翻译部分：

  1. 我有一个名为'detectors'的函数类别，它接受一个视频、一个空间尺度参数和一个时间尺度参数，并检测一些兴趣点。我编写了一个'MultiscaleDetector'函数，它基本上接受空间尺度参数的列表和一个'detector'函数作为参数，并对每个尺度执行给定的检测因子。
  3. 这是它们的样子：
  5. ```python
  6. def GaborDetector(v, sigma, tau, threshold, num_points):
  7.     """
  8.     Gabor检测器
  9.     
  10.     关键参数：
  11.     video -- 输入视频（y_len，x_len，frames）
  12.     sigma -- 高斯核空间标准差
  13.     tau -- 高斯核时间标准差
  14.     kappa -- Gabor响应阈值
  15.     """
  16.     # 设置视频
  17.     video = v.copy()
  18.     video = video.astype(float) / video.max()
  19.     video = video_smoothen_space(video, sigma)
  20.     # 首先定义一个宽度为-2tau到2tau的线性空间
  21.     time = np.linspace(-2 * tau, 2 * tau, int(4 * tau + 1))
  22.     omega = 4 / tau
  23.     # 定义Gabor滤波器
  24.     h_ev = np.exp(-time**2 / (2 * tau**2)) * np.cos(2 * np.pi * omega * time)
  25.     h_od = np.exp(-time**2 / (2 * tau**2)) * np.sin(2 * np.pi * omega * time)
  26.     # 规范化L1范数
  27.     h_ev /= np.linalg.norm(h_ev, ord=1)
  28.     h_od /= np.linalg.norm(h_od, ord=1)
  29.     # 计算响应
  30.     response = (scp.convolve1d(video, h_ev, axis=2) ** 2) + (scp.convolve1d(video, h_od, axis=2) ** 2)
  31.     points = interest_points(response, num=num_points, threshold=threshold, scale=sigma)
  32.     return points
  1. def MultiscaleDetector(detector, video, sigmas, tau, num_points):
  2.     """
  3.     多尺度检测器
  5.     在多个尺度上执行检测器。检测器必须是一个接受视频作为输入的函数，以及其他参数，并返回兴趣点列表。
  6.     
  7.     关键参数：
  8.     detector -- 返回兴趣点的函数
  9.     video -- 输入视频（y_len，x_len，frames）
  10.     sigmas -- 尺度列表
  11.     """
  12.     
  13.     # 对于每个尺度，计算响应
  14.     points = []
  15.     for sigm in sigmas:
  16.         found = detector(video, sigm, tau)
  17.         points.append(found)
  19.     # 过滤点，目前无关紧要

正如你可能已经注意到的，'GaborDetector'函数内部进行了一些繁重的计算，这些计算仅依赖于时间参数。因此，'MultiscaleDetector'函数在每次调用时不必重新计算这些变量。

为了避免重构代码，我尝试了一个我希望是一个巧妙的技巧，但认为可能是徒劳的：

  1. def MultiscaleDetector(detector, video, sigmas, tau, num_points):
  2.     """
  3.     多尺度检测器
  5.     在多个尺度上执行检测器。检测器必须是一个接受视频作为输入的函数，以及其他参数，并返回兴趣点列表。
  6.     
  7.     关键参数：
  8.     detector -- 返回兴趣点的函数
  9.     video -- 输入视频（y_len，x_len，frames）
  10.     sigmas -- 尺度列表
  11.     """
  12.     
  13.     optimization_trick = lambda s_param: detector(video, s_param, tau)
  14.     # 对于每个尺度，计算响应
  15.     points = []
  16.     for sigm in sigmas:
  17.         found = optimization_trick(sigm)
  18.         points.append(found)
  20.     # 过滤点，目前无关紧要

我希望仅依赖于tau的变量在'optimization_trick'中保持“存储”，以避免重新计算。然而，当我计时不同的实现时，差异约为0.02秒，优化后的函数更快。

编辑：

实际的调用如下：

  1. # 读取视频
  2. video = read_video(video_name, num_frames, 0)
  3. # 获取兴趣点
  4. detector = lambda v, s, t: GaborDetector(v, s, t, 0.3, 500)
  5. scales = [3 * (1.1)**i for i in range(6)]
  6. start = time.time()
  7. points = MultiscaleDetector(detector, video, scales, 1.5, 500)
  8. end = time.time()
  9. print("时间：{}".format(end-start))

我尝试将我想要避免重新计算的所有变量放在一个不同的函数中，就像这样：

  1. def GaborDetectorTrial(v, sigma, tau, threshold, num_points):
  2.     """
  3.     Gabor检测器
  4.     
  5.     关键参数：
  6.     video -- 输入视频（y_len，x_len，frames）
  7.     sigma -- 高斯核空间标准差
  8.     tau -- 高斯核时间标准差
  9.     kappa -- Gabor响应阈值
  10.     """
  11.     @lru_cache(maxsize=None)
  12.     def time_function(tau):
  13.         time = np.linspace(-2 * tau, 2 * tau, int(4 * tau + 1))
  14.         omega = 4 / tau
  15.         # 定义Gabor滤波器
  16.         h_ev = np.exp(-time**2 / (2 * tau**2)) * np.cos(2 * np.pi * omega * time)
  17.         h_od = np.exp(-time**2 / (2 * tau**2)) * np.sin(2 * np.pi * omega * time)
  18.         # 规范化L1范数
  19.         h_ev /= np.linalg.norm(h_ev, ord=1)
  20.         h_od /= np.linalg.norm(h_od, ord=1)
  21.         return h_ev, h_od
  22.     # 设置视频
  23.     video = v.copy()
  24.     video = video.astype(float) / video
  26. <details>
  27. <summary>英文:</summary>
  29. I have a category of functions &#39;detectors&#39;, that, given a video, a space scale parameter and a time scale parameter, detect some interest points. I wrote a &#39;MultiscaleDetector&#39; function, that basically takes a list of space scale parameters and a &#39;detector&#39; function as parameters and executes the given detector factor for every scale.
  31. This is how they look:

def GaborDetector(v, sigma, tau, threshold, num_points):
"""
Gabor Detector

  1. Keyword arguments:
  2. video -- input video (y_len, x_len, frames)
  3. sigma -- Gaussian kernel space standard deviation
  4. tau -- Gaussian kernel time standard deviation
  5. kappa -- Gabor response threshold
  6. &quot;&quot;&quot;
  7. # setup video
  8. video = v.copy()
  9. video = video.astype(float)/video.max()
  10. video = video_smoothen_space(video, sigma)
  11. # first define a linspace of width -2tau to 2tau
  12. time = np.linspace(-2*tau, 2*tau, int(4*tau+1))
  13. omega = 4/tau
  14. # define the gabor filters
  15. h_ev = np.exp(-time**2/(2*tau**2)) * np.cos(2*np.pi*omega*time)
  16. h_od = np.exp(-time**2/(2*tau**2)) * np.sin(2*np.pi*omega*time)
  17. # normalize the L1 norm
  18. h_ev /= np.linalg.norm(h_ev, ord=1)
  19. h_od /= np.linalg.norm(h_od, ord=1)
  20. # compute the response
  21. response = (scp.convolve1d(video, h_ev, axis=2) ** 2) + (scp.convolve1d(video, h_od, axis=2) ** 2)
  22. points = interest_points(response, num=num_points, threshold=threshold, scale=sigma)
  23. return points

def MultiscaleDetector(detector, video, sigmas, tau, num_points):
"""
Multiscale Detector

  1. Executes a detector at multiple scales. Detector has to be a function that
  2. takes a video as input, along with other parameters, and returns a list of interest points.
  5. Keyword arguments:
  6. detector -- function that returns interest points
  7. video -- input video (y_len, x_len, frames)
  8. sigmas -- list of scales
  9. &quot;&quot;&quot;
  11. # for every scale, compute the response
  12. points = []
  13. for sigm in sigmas:
  14.     found = detector(video, sigm, tau)
  15.     points.append(found)
  17. # filter the points, currently irrelevant
  2. As you can maybe already notice, there is some heavy computation done inside the &quot;GaborDetector&quot; function, that depends only on the time parameter. Therefore, the &quot;MultiscaleDetector&quot; function unnecessarily recomputes these variables in each call.
  5. In an attempt to avoid refactoring the code, I came up with what I wished was a nice trick, but believed it would be vain:

def MultiscaleDetector(detector, video, sigmas, tau, num_points):
"""
Multiscale Detector

  1. Executes a detector at multiple scales. Detector has to be a function that
  2. takes a video as input, along with other parameters, and returns a list of interest points.
  5. Keyword arguments:
  6. detector -- function that returns interest points
  7. video -- input video (y_len, x_len, frames)
  8. sigmas -- list of scales
  9. &quot;&quot;&quot;
  11. optimization_trick = lambda s_param: detector(video, s_param, tau)
  12. # for every scale, compute the response
  13. points = []
  14. for sigm in sigmas:
  15.     found = optimization_trick(sigm)
  16.     points.append(found)
  18. # filter the points, currently irrelevant
  2. I hoped that the variables dependent only on tau would remain &quot;stored&quot; somehow in the &quot;optimization_trick&quot; and avoid being recomputed. However, when I timed the different implementations, the difference was around 0.02 seconds, with the &quot;optimized&quot; function being faster.
  4. EDIT:
  6. The actual call that takes place is the following:

read the video

video = read_video(video_name, num_frames, 0)

get the interest points

detector = lambda v, s, t: GaborDetector(v, s, t, 0.3, 500)
scales = [3*(1.1)**i for i in range(6)]
start = time.time()
points = MultiscaleDetector(detector, video, scales, 1.5, 500)
end = time.time()
print("Time: {}".format(end-start))

  2. I tried putting all the variables that i wanted to avoid in a different function like this:

def GaborDetectorTrial(v, sigma, tau, threshold, num_points):
"""
Gabor Detector

  1. Keyword arguments:
  2. video -- input video (y_len, x_len, frames)
  3. sigma -- Gaussian kernel space standard deviation
  4. tau -- Gaussian kernel time standard deviation
  5. kappa -- Gabor response threshold
  6. &quot;&quot;&quot;
  7. @lru_cache(maxsize=None)
  8. def time_function(tau):
  9.     time = np.linspace(-2*tau, 2*tau, int(4*tau+1))
  10.     omega = 4/tau
  11.     # define the gabor filters
  12.     h_ev = np.exp(-time**2/(2*tau**2)) * np.cos(2*np.pi*omega*time)
  13.     h_od = np.exp(-time**2/(2*tau**2)) * np.sin(2*np.pi*omega*time)
  14.     # normalize the L1 norm
  15.     h_ev /= np.linalg.norm(h_ev, ord=1)
  16.     h_od /= np.linalg.norm(h_od, ord=1)
  17.     return h_ev, h_od
  18. # setup video
  19. video = v.copy()
  20. video = video.astype(float)/video.max()
  21. video = video_smoothen_space(video, sigma)
  22. # compute the response
  23. h_ev, h_od = time_function(tau)
  24. response = (scp.convolve1d(video, h_ev, axis=2) ** 2) + (scp.convolve1d(video, h_od, axis=2) ** 2)
  25. points = interest_points(response, num=num_points, threshold=threshold, scale=sigma)
  26. return points
  1. The variables that i want to avoid recomputing are h_ev and h_od. The execution time was basically the same (actually, some ms slower). 
  3. </details>
  6. # 答案1
  7. **得分**: 0
  9. 你能不能把它拆分成多个函数？
  11. 示例：
  13. ```python
  14. def time_function(tau): # 计算 hev 和 hod
  15.     time = np.linspace(-2*tau, 2*tau, int(4*tau+1))
  16.     omega = 4/tau
  17.     # 定义 Gabor 滤波器
  18.     h_ev = np.exp(-time**2/(2*tau**2)) * np.cos(2*np.pi*omega*time)
  19.     h_od = np.exp(-time**2/(2*tau**2)) * np.sin(2*np.pi*omega*time)
  20.     # 归一化 L1 范数
  21.     h_ev /= np.linalg.norm(h_ev, ord=1)
  22.     h_od /= np.linalg.norm(h_od, ord=1)
  23.     return h_ev, h_od
  25. 然后：
  27. # 读取视频
  28. video = read_video(video_name, num_frames, 0)
  29. # 获取兴趣点
  30. custom_tau = {}
  32. scales = [3*(1.1)**i for i in range(6)]
  33. start = time.time()
  34. for i in range(0,10): # 循环是因为否则只调用一次 GaborDetector，所以问题不相关
  35.     # 在这里插入逻辑，因为为了使你的问题相关，Tau 不应总是相同
  36.     if tau not in custom_tau.keys(): # 逻辑来“缓存”相同 tau 值的 hev 和 hod
  37.         hev, hod = time_function(tau)
  38.         custom_tau[tau] = (hev, hod)
  39.     else:
  40.         hev, hod = custom_tau[tau]
  41.     gabor = GaborDetector(v, s, t, hev, hod, 0.3, 500)
  42.     points = MultiscaleDetector(gabor, video, scales, 1.5, 500)
  43. end = time.time()
  44. print("Time: {}".format(end-start))
  46. 最后，你可以更改 GarborCollector，以避免重新计算时间：
  48. ```python
  49. def GaborDetectorTrial(v, sigma, tau, hev, hod, threshold, num_points):
  51.     video = v.copy()
  52.     video = video.astype(float)/video.max()
  53.     video = video_smoothen_space(video, sigma)
  54.     response = (scp.convolve1d(video, h_ev, axis=2) ** 2) + (scp.convolve1d(video, h_od, axis=2) ** 2)
  55.     points = interest_points(response, num=num_points, threshold=threshold, scale=sigma)
  56.     return points

在这段代码中，你通过一个字典进行“缓存”，如果你已经为给定的“tau”计算了它，那么你将在字典中查找结果，否则你将计算它并将其放入字典中。我不得不推测和猜测你的代码如何工作，因为在你提供的情况下，由于只调用一次 garbor，he_v 和 ho_d 从不重新计算。

英文:

Can't you just split in multiple function?

Example :

  1. def time_function(tau): #To compute hev and hod
  2.         time = np.linspace(-2*tau, 2*tau, int(4*tau+1))
  3.         omega = 4/tau
  4.         # define the gabor filters
  5.         h_ev = np.exp(-time**2/(2*tau**2)) * np.cos(2*np.pi*omega*time)
  6.         h_od = np.exp(-time**2/(2*tau**2)) * np.sin(2*np.pi*omega*time)
  7.         # normalize the L1 norm
  8.         h_ev /= np.linalg.norm(h_ev, ord=1)
  9.         h_od /= np.linalg.norm(h_od, ord=1)
  10.         return h_ev, h_od

and then

  1. # read the video
  2. video = read_video(video_name, num_frames, 0)
  3. # get the interest points
  4. custom_tau = {}
  6. scales = [3*(1.1)**i for i in range(6)]
  7. start = time.time()
  8. for i in range(0,10): # for loop cause otherwise you call GaborDetector only once so the question is not revelant
  9.     #Insert logic here because for your question to be revelant Tau must not always be the same
  10.     if tau not in custom_tau.keys(): # Logic to &quot;cache&quot; hev and hod for the same tau value
  11.         hev, hod = time_function(tau)
  12.         custom_tau[tau] = (hev, hod)
  13.     else:
  14.         hev, hod = custom_tau[tau]
  15.     gabor = GaborDetector(v, s, t, hev, hod, 0.3, 500)
  16.     points = MultiscaleDetector(gabor, video, scales, 1.5, 500)
  17. end = time.time()
  18. print(&quot;Time: {}&quot;.format(end-start))

and finally you change GarborCollector to not recompute time

  1. def GaborDetectorTrial(v, sigma, tau,hev, hod, threshold, num_points):
  3.     video = v.copy()
  4.     video = video.astype(float)/video.max()
  5.     video = video_smoothen_space(video, sigma)
  6.     response = (scp.convolve1d(video, h_ev, axis=2) ** 2) + (scp.convolve1d(video, h_od, axis=2) ** 2)
  7.     points = interest_points(response, num=num_points, threshold=threshold, scale=sigma)
  8.     return points

What you do in this code is "caching" (through a dictionnary) hev and hod, so that if you already computed it for a given "tau", you instead lookup in the dictionnary for the result, otherwise you compute it and put it in the dictionnary

I had to extrapolate and guess how your code would work, because in the case you had given, he_v and ho_d are never re-computed since you call garbor only once

答案2

得分: 0

你必须在适当的作用域中定义要缓存的函数（time_function），在这里是在 GaborDetectorTrial 外部 - 就像你的代码中一样，time_function 的生命周期会在每次 GaborDetectorTrial 返回时结束，它的缓存也会结束。

英文:

You must define the to-be-cached function (time_function) in an appropriate scope, here: outside of GaborDetectorTrial - as it is in your code above, the time_function's life ends each time GaborDetectorTrial returns, and so does its cache's.

huangapple
  • 本文由 发表于 2023年6月15日 20:58:44
  • 转载请务必保留本文链接：https://go.coder-hub.com/76482735.html
  • design-patterns
  • functional-programming
  • optimization
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定