rasterio数据 – Python – 执行时间 – 预处理

huangapple
go评论110阅读模式
英文:

rasterio data - python - execution time - preprocessing

问题

I'm using Rasterio to work with a satellite image, and I need to iterate through the entire file. and applying the formula to each pixel. This process takes a long time and makes it difficult for me to try out different modifications because it takes a long time to see the results each time.

使用Rasterio处理卫星图像，我需要遍历整个文件并对每个像素应用公式。这个过程需要很长时间，使我难以尝试不同的修改，因为每次查看结果都需要很长时间。

any suggestions to improve time execution ?

有没有建议来改善执行时间？

And is it better to work on this project locally or via Jupiter, Google Colab, or other tools?

是在本地还是通过Jupyter、Google Colab或其他工具进行此项目更好？

  1. def dn_to_radiance(data_array, band_number):
  2.     # getting the G value
  3.     channel_gain = float(Landsat8_mlt_dict['RADIANCE_MULT_BAND_' + str(band_number) + ' '])
  4.     # Getting the B value
  5.     channel_offset = float(Landsat8_mlt_dict['RADIANCE_ADD_BAND_' + str(band_number) + ' '])
  6.     # creating a temp array to store the radiance value
  7.     # np.empty_like Return a new array with the same shape and type as a given array.
  8.     new_data_array = np.empty_like(data_array)
  10.     # loooping through the image
  11.     for i, row in enumerate(data_array):
  12.         for j, col in enumerate(row):
  14.             # checking if the pixel value is not nan, to avoid background correction
  15.             if data_array[i][j].all() != np.nan:
  16.                 new_data_array[i][j] = data_array[i][j] * channel_gain + channel_offset
  17.     print(f'Radiance calculated for band {band_number}')
  18.     return new_data_array
  20. Landsat8_mlt_dict = {}
  21. with open('LC08_L2SP_190037_20190619_20200827_02_T1_MTL.txt', 'r') as _:
  22.     for line in _:
  23.         line = line.strip()
  24.         if line != 'END':
  25.             key, value = line.split('=')
  26.             Landsat8_mlt_dict[key] = value
  28. def radiance_to_reflectance(arr, ESUN, ):
  29.     # getting the d value
  30.     d = float(Landsat8_mlt_dict['EARTH_SUN_DISTANCE '])
  31.     # calculating rh phi value from theta
  32.     phi = 90 - float(Landsat8_mlt_dict['SUN_ELEVATION '])
  33.     # creating the temp array
  34.     new_data_array = np.empty_like(arr)
  35.     # loop to finf the reflectance
  36.     for i, row in enumerate(arr):
  37.         for j, col in enumerate(row):
  38.             if arr[i][j].all() != np.nan:
  39.                 new_data_array[i][j] = np.pi * arr[i][j] * d ** 2 / (ESUN * cos(phi * math.pi / 180))
  40.     print("Reflectance of Band calculated")
  41.     return new_data_array
  1. def dn_to_radiance(data_array, band_number):
  2.     # 获取G值
  3.     channel_gain = float(Landsat8_mlt_dict['RADIANCE_MULT_BAND_' + str(band_number) + ' '])
  4.     # 获取B值
  5.     channel_offset = float(Landsat8_mlt_dict['RADIANCE_ADD_BAND_' + str(band_number) + ' '])
  6.     # 创建一个临时数组来存储辐射度值
  7.     # np.empty_like返回一个与给定数组具有相同形状和类型的新数组。
  8.     new_data_array = np.empty_like(data_array)
  10.     # 循环遍历图像
  11.     for i, row in enumerate(data_array):
  12.         for j, col in enumerate(row):
  14.             # 检查像素值是否不是NaN，以避免背景校正
  15.             if data_array[i][j].all() != np.nan:
  16.                 new_data_array[i][j] = data_array[i][j] * channel_gain + channel_offset
  17.     print(f'已计算波段{band_number}的辐射度')
  18.     return new_data_array
  20. Landsat8_mlt_dict = {}
  21. with open('LC08_L2SP_190037_20190619_20200827_02_T1_MTL.txt', 'r') as _:
  22.     for line in _:
  23.         line = line.strip()
  24.         if line != 'END':
  25.             key, value = line.split('=')
  26.             Landsat8_mlt_dict[key] = value
  28. def radiance_to_reflectance(arr, ESUN, ):
  29.     # 获取d值
  30.     d = float(Landsat8_mlt_dict['EARTH_SUN_DISTANCE '])
  31.     # 从theta计算rh phi值
  32.     phi = 90 - float(Landsat8_mlt_dict['SUN_ELEVATION '])
  33.     # 创建临时数组
  34.     new_data_array = np.empty_like(arr)
  35.     # 循环查找反射率
  36.     for i, row in enumerate(arr):
  37.         for j, col in enumerate(row):
  38.             if arr[i][j].all() != np.nan:
  39.                 new_data_array[i][j] = np.pi * arr[i][j] * d ** 2 / (ESUN * cos(phi * math.pi / 180))
  40.     print("已计算波段的反射率")
  41.     return new_data_array
英文:

I'm using Rasterio to work with a satellite image, and I need to iterate through the entire file. and applying the formula to each pixel. This process takes a long time and makes it difficult for me to try out different modifications because it takes a long time to see the results each time.
any suggestions to improve time execution ?
And is it better to work on this project locally or via Jupiter, Google Colab, or other tools?

  1. def dn_to_radiance(data_array, band_number):
  2.     # getting the G value
  3.     channel_gain = float(Landsat8_mlt_dict['RADIANCE_MULT_BAND_' + str(band_number) + ' '])
  4.     # Getting the B value
  5.     channel_offset = float(Landsat8_mlt_dict['RADIANCE_ADD_BAND_' + str(band_number) + ' '])
  6.     # creating a temp array to store the radiance value
  7.     # np.empty_like Return a new array with the same shape and type as a given array.
  8.     new_data_array = np.empty_like(data_array)
  10.     # loooping through the image
  11.     for i, row in enumerate(data_array):
  12.         for j, col in enumerate(row):
  14.             # checking if the pixel value is not nan, to avoid background correction
  15.             if data_array[i][j].all() != np.nan:
  16.                 new_data_array[i][j] = data_array[i][j] * channel_gain + channel_offset
  17.     print(f'Radiance calculated for band {band_number}')
  18.     return new_data_array
  19. Landsat8_mlt_dict = {}
  20. with open('LC08_L2SP_190037_20190619_20200827_02_T1_MTL.txt', 'r') as _:
  21.     # print(type(_))
  22.     for line in _:
  23.         line = line.strip()
  24.         if line != 'END':
  25.             key, value = line.split('=')
  26.             Landsat8_mlt_dict[key] = value
  29. # print(Landsat8_mlt_dict)
  30. def radiance_to_reflectance(arr, ESUN, ):
  31.     # getting the d value
  32.     d = float(Landsat8_mlt_dict['EARTH_SUN_DISTANCE '])
  33.     # calculating rh phi value from theta
  34.     phi = 90 - float(Landsat8_mlt_dict['SUN_ELEVATION '])
  35.     # creating the temp array
  36.     new_data_array = np.empty_like(arr)
  37.     # loop to finf the reflectance
  38.     for i, row in enumerate(arr):
  39.         for j, col in enumerate(row):
  40.             if arr[i][j].all() != np.nan:
  41.                 new_data_array[i][j] = np.pi * arr[i][j] * d ** 2 / (ESUN * cos(phi * math.pi / 180))
  42.     print(f"Reflectance of Band calculated")
  43.     return new_data_array

答案1

得分: 0

你可以使用第三方库，比如EOReader，来为你将Landsat波段转换为反射率。

  1. from eoreader.reader import Reader
  2. from eoreader.bands import RED, GREEN
  4. prod = Reader().open(r"LC08_L1TP_200030_20201220_20210310_02_T1.tar")
  6. # 将这些波段加载为xarray.DataArray字典
  7. bands = prod.load([RED, GREEN])
  8. green = bands[GREEN]
  9. red = bands[RED]

声明：我是EOReader的维护者

如果你想自己做，你应该学习如何在Python中处理数组，绝对不要使用循环！相反，你应该矢量化你的计算，这样速度会快得多！

英文:

You could use third-party libraries such as EOReader to convert Landsat bands to reflectance for you.

  1. from eoreader.reader import Reader
  2. from eoreader.bands import RED, GREEN
  4. prod = Reader().open(r"LC08_L1TP_200030_20201220_20210310_02_T1.tar")
  6. # Load those bands as a dict of xarray.DataArray
  7. bands = prod.load([RED, GREEN])
  8. green = bands[GREEN]
  9. red = bands[RED]

> Disclaimer: I am the maintener of EOReader

If you want to do that yourself, you should do some tutorials on how to handle arrays in Python.
Never ever loop over them! You should instead vectorize your computations: it will go way faster!

huangapple
  • 本文由 发表于 2023年2月18日 19:03:26
  • 转载请务必保留本文链接：https://go.coder-hub.com/75492884.html
  • preprocessor
  • python
  • rasterio
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定