替换 -3.402823466e+38 为 NaN 或在数据框中删除此值。

huangapple go评论74阅读模式
英文:

replace -3.402823466e+38 with NaN or get rid of this value in a dataframe

问题

I'm working with sentinel 2 images that cut with a bbox with QGis. Some of those images aren't completely under the bbox so Qgis put the lowest float value close to 0 as a NoData value.
我正在使用Sentinel 2图像,使用QGis切割了边界框。其中一些图像不完全在边界框内,因此Qgis将最接近0的最小浮点值作为NoData值。

I tried to indicate that I want a specific value as NoData but it doesn't work with every pixel since I have now the specific value I told him to put -4.444444 but -3.402823466e+38 values as well.
我尝试指定一个特定的值作为NoData,但不适用于每个像素,因为我现在有特定值-4.444444,但也有-3.402823466e+38的值。

So I tried to remove it from the dataframe I'm working on (image opened with rasterio), with :
因此,我尝试从我正在使用的数据帧中删除它(使用rasterio打开的图像),使用:

df_annee.replace(-3.402823466e+38, np.nan, inplace=True)

I tried to put the value of the dataframe in a variable that I'm calling to replace it :
我尝试将数据帧的值放入一个我称之为要替换的变量中:

i = df_mois.loc[:,'Valeurs'][0][5]
df_annee.replace(i, np.nan, inplace=True)

I tried to replace it directly from the array with the script I'm using to create my dataframe :
我尝试直接从用于创建数据帧的脚本中替换它:

src = (rasterio.open(each_directory))
array = src.read(1)
array[(array<-2)] = np.nan
array[(array > -2.412823466e+38 ) | (array < -1.401823466e+38 )] = np.nan
array[(array == -3.402823466e+38)] = np.nan

Nothing worked. The problem with this value is that I want to average the values of my images to plot it but seaborn plots consider -3.402823466e+38 as -3, my plot end up false and np.mean() or mean(df) doesn't recognize these values as NoData so my average ends up to be really close to 0 even when it's not.
什么都没用。这个值的问题是,我想要平均图像的值以绘制它,但seaborn绘图将-3.402823466e+38视为-3,我的绘图最终变得不准确,np.mean()或mean(df)不认为这些值是NoData,因此我的平均值最终非常接近0,即使实际上不是。

This loop can find all my values but not the -3.402823466e+38 one :
这个循环可以找到我所有的值,但不能找到-3.402823466e+38的值:

for i in range(len(df_mois.loc[:,"Valeurs"])):
for y in range(len(df_mois.loc[:,"Valeurs"][0])):
for x in range(y):
#print(df_mois.loc[:,"Valeurs"][i][y][x])
if df_mois.loc[:,"Valeurs"][i][y][x] < 0.001 and df_mois.loc[:,"Valeurs"][i][y][x] > -0.00001:
print(df_mois.loc[:,"Valeurs"][i][y][x])
print('coucou')

Another weird thing is that my numbers are in float 32 but I have +38 numbers in my dataframe ...
另一个奇怪的事情是,我的数字是32位浮点数,但我的数据帧中有+38的数字...

What can I do ?
我可以做什么?

英文:

I'm working with sentinel 2 images that cut with a bbox with QGis. Some of those images aren't completely under the bbox so Qgis put the lowest float value close to 0 as a NoData value.
I tried to indicate that i want a specific value as a NoData but it doesn't work with every pixel since I have now the specific value I told him to put -4.444444 but -3.402823466e+38 values aswell.

So I tried to remove it from the dataframe I'm working on (image opened with rasterio), with :

 df_annee.replace(-3.402823466e+38, np.nan, inplace=True)

I tried to put the value of the dataframe in a variable that I'm calling to replace it :

i = df_mois.loc[:,&#39;Valeurs&#39;][0][5]
df_annee.replace(i, np.nan, inplace=True)

I tried to replace it directly from the array with the script i'm using to create my dataframe :

src = (rasterio.open(each_directory))
array = src.read(1)
array[(array&lt;-2)] = np.nan
array[(array &gt; -2.412823466e+38 ) | (array &lt; -1.401823466e+38 )] = np.nan
array[(array == -3.402823466e+38)] = np.nan

Nothing worked. The problem with this value is that I want to average the values of my images to plot it but seaborn plots consider -3.402823466e+38 as -3, my plot end up false and np.mean() or mean(df) doesn't recognize these values as NoData so my average ends up to be really close to 0 even when it's not.

This loop can find all my values but not the -3.402823466e+38 one :

for i in range(len(df_mois.loc[:,&quot;Valeurs&quot;])):
    for y in range(len(df_mois.loc[:,&quot;Valeurs&quot;][0])):
        for x in range(y): 
            #print(df_mois.loc[:,&quot;Valeurs&quot;][i][y][x])
            if df_mois.loc[:,&quot;Valeurs&quot;][i][y][x] &lt; 0.001 and df_mois.loc[:,&quot;Valeurs&quot;][i][y][x] &gt; -0.00001:
                print(df_mois.loc[:,&quot;Valeurs&quot;][i][y][x])
                print(&#39;coucou&#39;)

Another weird thing is that, my numbers are in float 32 but I have +38 numbers in my dataframe ...

What can I do ?

答案1

得分: 0

以下是翻译好的部分:

这个特定数值出现的原因对我来说不清楚。另一方面,我通过查看它首次出现在我的图像中的时间来解决了我的问题。一开始我没有注意到它,因为我在QGis上进行了几次处理,它能够处理它。

起初我,

1 - 用otbcli_superimpose在Python中叠加了我的波段,分辨率为20,一个波段为10,以节省时间,没有意识到我的所有图像的空间分辨率都不相同(因为我使用的是度数-EPSG 4326-)

2- 使用QGIS中的BandMath对我的图像进行了平均处理
*第二个问题是NoData值。至于空间分辨率,QGis中的Bandmath能够处理它(或隐藏它),但Python不能。有一个问题,但是QGis只是继续运行,而没有告诉我有问题,所以我能够使用Bandmath平均处理我的图像而没有注意到任何问题。

3- 想要使用Pandas、seaborn绘制这些图像,那时我注意到我的矩阵很奇怪,而且我无法处理-3.402823466e+38的值。

所以我再次进行了整个过程:

1- otbcli_superimpose 将所有图像与一张图像叠加
2- gdalwarp -cutline 以相同的方式剪切所有图像,使用*-dstnodata**指定nodata值*
3- gdalmerge.py 合并不在相同瓦片中的图像
4- otbcli_BandMath 对它们进行平均处理。

我认为以后如果我需要在Python中打开我的图像,我不会再使用QGis,因为我不知道如何指定一些参数,而QGis可以隐式地运行,这不是一个好的做法。

英文:

The reason why this specific value appears is unclear to me.
On the other hand, I was able to resolve my problem by looking at when it first started to appears in my images. I didn't noticed it at first because I did several treatments on QGis and it was able to handle it.

At first I,

1 - superimposed in Python with otbcli_superimpose
only my bands at a resolution of 20 with a band at 10 to save time and didn't realised all my images weren't at the same spatial resolution (because I was in degre-EPSG 4326-)

2- averaged my images with BandMath in QGIS
The second problem was the NoData values. As for the spatial resolution, Bandmath in QGis was able to handle it (or hide it) but not python.There was a problem but QGis just went for it without telling me there was a problem, so I was able to average my images with Bandmath without noticing anything.

3- wanted to plot those images with Pandas, seaborn
that's when I noticed my matrices were weird and that I wasn't able to handle the -3.402823466e+38 value.

So I did all the process again with :

1- otbcli_superimpose all my images with one image
2- gdalwarp -cutline to cut all my images the same way, specifying the nodata value with -dstnodata
3- gdalmerge.py to merge the images that where not in the same tile
4- otbcli_BandMath to average them.

I think in the future I'll not use QGis again if I need to open my images with python afterwards I have no idea how to specify some parameters and QGis can run with it being implicit so that's not a good practice.

huangapple
  • 本文由 发表于 2023年6月26日 19:51:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/76556421.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定