问题

I'm trying use multi-channel file to training unet classify.

我的目标是使用多通道文件来训练Unet分类器。

My dataset is 5 * 1-channel & 1 * 3-channel image, and I expected it can be 8-channel .npy file.

我的数据集包括5张1通道图像和1张3通道图像，我希望它可以成为一个8通道的.npy文件。

I use np.concatenate to merge, but it cannot apply on a 1-channel image. Here is my code:

我使用np.concatenate来合并，但它不能应用于1通道图像。以下是我的代码：

for i in range(6):
  img = data[i]
  images.append(img)
img_batch = np.concatenate(images, axis=3)

So, it should expand 1-channel-image to 3-channel first, then concatenate, then to 18-channel .mpy file

因此，应该首先将1通道图像扩展为3通道，然后进行连接，然后生成18通道的.npy文件。

Image.open("class1_image1.jpg").convert("RGB")   #expand 1-channel-image to 3-channel image

Is there a method to merge multiple images into a multi-channel .npy file without expanding 1-channel image?

是否有一种方法可以将多个图像合并成多通道的.npy文件，而不需要扩展1通道图像？

Or 18-channel and 8-channel are the same file for unet classify task?

或者对于Unet分类任务，18通道和8通道是相同的文件吗？

for i in range(6):
  img = data[i]
  images.append(img)
img_batch = np.concatenate(images, axis=3)

Image.open("class1_image1.jpg").convert("RGB")   #expand 1-channel-image to 3-channel image

答案1

得分: 1

可以提供图像（至少是单通道图像）吗？图像的通道数量不重要，甚至图像的通道数量可以不同，但我将演示如何合并两个3通道图像和一个单通道图像：

我们可以使用 numpy 将图像的RGB值（或者是第三维的任意通道）插入到第四维中，然后对第四维求和。

首先我们有一些图像（获取方式不重要，我只是在此示例中使用了谷歌图像），确保它们的尺寸相同：

from skimage import io as io
image1 = io.imread('https://www.allrecipes.com/thmb/c_2gXiAwkO6u1UJCY-1eAVCy0h0=/1500x0/filters:no_upscale():max_bytes(150000):strip_icc()/54679_perfect-baked-potato-Rita-1x1-1-91711252bb3740088c8ea55c5f9bef1c.jpg')
image2 = io.imread('https://cdn.loveandlemons.com/wp-content/uploads/2020/01/baked-potato.jpg')
image3 = io.imread('https://www.southerncravings.com/wp-content/uploads/2020/01/Crispy-Baked-Potatoes-7.jpg')

image1 = image1[:, 200:-200, :]
image2 = image2[:-60, :-60, :]
import numpy as np
image3 = np.sum(image3[:-300, :-100, :], axis=2)

image1：

![enter image description here][1]

image2：

![enter image description here][2]

image3（只有一个通道）：

![enter image description here][3]

image3（在第三维连接后，所以有3个通道；下面会介绍）：

![enter image description here][4]

我们需要扩展图像，使其具有图像列表中通道数不足的图像的最大通道数，可以使用 numpy 的 concatenate 函数在第三维上进行操作：

import numpy as np
def make_list_of_images(images_list):
    list_of_images = []
    for _image in images_list:
        if len(_image.shape) == 2:
            _image.shape += (1,)
        if _image.shape[2] < np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) > 2]):
            _image = np.concatenate([_image for _ in range(np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) > 2]) - _image.shape[2] + 1)], axis=2)
        list_of_images.append(_image)
    return list_of_images

我们需要对RGB值（或第三维通道）进行归一化处理，以下是一个用于实现此功能的函数：

import numpy as np
def normalize_rgb_values(rgb_values, max_value=1.0):
    norm_rgb_values = (rgb_values - np.mean(rgb_values)) / np.var(rgb_values)**0.5
    norm_rgb_values += abs(np.min(norm_rgb_values))
    norm_rgb_values *= (max_value / np.max(norm_rgb_values))
    return np.round(norm_rgb_values, decimals=0).astype(int) if max_value == 255 else np.round(norm_rgb_values, decimals=9).astype(float)

以下是我们通过枚举 list_of_images 并将每个 _image 插入到第四维中来填充的 images numpy 数组：

import numpy as np
max_number_of_channels = np.max([_image_channels.shape[2] for _image_channels in list_of_images])
images = np.zeros((image1.shape[0], image1.shape[1], max_number_of_channels, len(list_of_images))).astype(float)
for _image_num, _image in enumerate(list_of_images):
    images[:, :, :, _image_num] = _image

而不是连接，我们可以使用 numpy 的 sum 来合并图像（在第四维上求和）：

import numpy as np
summed_images = np.sum(images, axis=3)
from matplotlib import pyplot as plt
plt.imshow(normalize_rgb_values(summed_images))
plt.show()

合并后的图像（即 summed_images）：

![enter image description here][5]

以下是所有代码：

from skimage import io as io
image1 = io.imread('https://www.allrecipes.com/thmb/c_2gXiAwkO6u1UJCY-1eAVCy0h0=/1500x0/filters:no_upscale():max_bytes(150000):strip_icc()/54679_perfect-baked-potato-Rita-1x1-1-91711252bb3740088c8ea55c5f9bef1c.jpg')
image2 = io.imread('https://cdn.loveandlemons.com/wp-content/uploads/2020/01/baked-potato.jpg')
image3 = io.imread('https://www.southerncravings.com/wp-content/uploads/2020/01/Crispy-Baked-Potatoes-7.jpg')
image1 = image1[:, 200:-200, :]
image2 = image2[:-60, :-60, :]
import numpy as np
image3 = np.sum(image3[:-300, :-100, :], axis=2)

def make_list_of_images(images_list):
    list_of_images = []
    for _image in images_list:
        if len(_image.shape) == 2:
            _image.shape += (1,)
        if _image.shape[2] < np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) > 2]):
            _image = np.concatenate([_image for _ in range(np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) > 2]) - _image.shape[2] + 1)], axis=2)
        list_of_images.append(_image)
    return list_of_images

list_of_images = make_list_of_images([image1, image2, image3])

def normalize_rgb_values(rgb_values, max_value=1.0):
    norm_rgb_values = (rgb_values - np.mean(rgb_values)) / np.var(rgb_values)**0.5
    norm_rgb_values += abs(np.min(norm_rgb_values))
    norm_rgb_values *= (max_value / np.max(norm_rgb_values))
    return np.round(norm_rgb_values, decimals=0).astype(int) if max_value == 255 else np.round(norm_rgb_values, decimals=9).astype(float)

from matplotlib import

<details>
<summary>英文:</summary>

Can you provide the images (at least the 1-channel images)? It doesn&#39;t matter how many channels the images have or even if the images have different numbers of channels, but I&#39;ll do an example of merging two 3-channel images and one 1-channel image:

We can merge the images&#39; RGB (or however many 3rd dimension channels) values with `numpy` by inserting the images into the 4th dimension (then summing over the 4th dimension).

First we have some images (doesn&#39;t matter how we get them, I just used Google images for this example), and make sure they&#39;re the same size:

    from skimage import io as io
    image1 = io.imread(&#39;https://www.allrecipes.com/thmb/c_2gXiAwkO6u1UJCY-1eAVCy0h0=/1500x0/filters:no_upscale():max_bytes(150000):strip_icc()/54679_perfect-baked-potato-Rita-1x1-1-91711252bb3740088c8ea55c5f9bef1c.jpg&#39;)
    image2 = io.imread(&#39;https://cdn.loveandlemons.com/wp-content/uploads/2020/01/baked-potato.jpg&#39;)
    image3 = io.imread(&#39;https://www.southerncravings.com/wp-content/uploads/2020/01/Crispy-Baked-Potatoes-7.jpg&#39;)
    
    image1 = image1[:, 200:-200, :]
    image2 = image2[:-60, :-60, :]
    import numpy as np
    image3 = np.sum(image3[:-300, :-100, :], axis=2)

`image1`:

[![enter image description here][1]][1]

`image2`:

[![enter image description here][2]][2]

`image3` (just 1-channel):

[![enter image description here][3]][3]

`image3` (after concatenating in the 3rd dimension, so 3-channels; described next):

[![enter image description here][4]][4]

We&#39;re going to have to expand the images that have fewer than the max number of channels in our list of images by using `numpy`&#39;s `concatenate` function in the 3rd dimension:

    import numpy as np
    def make_list_of_images(images_list):
        list_of_images = []
        for _image in images_list:
            if len(_image.shape) == 2:
                _image.shape += (1,)
            if _image.shape[2] &lt; np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) &gt; 2]):
                _image = np.concatenate([_image for _ in range(np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) &gt; 2]) - _image.shape[2] + 1)], axis=2)
            list_of_images.append(_image)
        return list_of_images

We&#39;re going to have to normalize the RGB (or 3rd dimension channel) values, so here is a function to do that:

    import numpy as np
    def normalize_rgb_values(rgb_values, max_value=1.0):
        norm_rgb_values = (rgb_values - np.mean(rgb_values)) / np.var(rgb_values)**0.5
        norm_rgb_values += abs(np.min(norm_rgb_values))
        norm_rgb_values *= (max_value / np.max(norm_rgb_values))
        return np.round(norm_rgb_values, decimals=0).astype(int) if max_value == 255 else np.round(norm_rgb_values, decimals=9).astype(float)

Here&#39;s the `images` `numpy` array of zeros we fill by enumerating through our `list_of_images` and inserting each `_image` into the 4th dimension:

    import numpy as np
    max_number_of_channels = np.max([_image_channels.shape[2] for _image_channels in list_of_images])
    images = np.zeros((image1.shape[0], image1.shape[1], max_number_of_channels, len(list_of_images))).astype(float)
    for _image_num, _image in enumerate(list_of_images):
        images[:, :, :, _image_num] = _image

And instead of concatenating, we can use `numpy`&#39;s `sum` to merge the `images` (summing over the 4th dimension):

    import numpy as np
    summed_images = np.sum(images, axis=3)
    from matplotlib import pyplot as plt
    plt.imshow(normalize_rgb_values(summed_images))
    plt.show()

Merged images (i.e. `summed_images`):

[![enter image description here][5]][5]

And here is the code altogether:

    from skimage import io as io
    image1 = io.imread(&#39;https://www.allrecipes.com/thmb/c_2gXiAwkO6u1UJCY-1eAVCy0h0=/1500x0/filters:no_upscale():max_bytes(150000):strip_icc()/54679_perfect-baked-potato-Rita-1x1-1-91711252bb3740088c8ea55c5f9bef1c.jpg&#39;)
    image2 = io.imread(&#39;https://cdn.loveandlemons.com/wp-content/uploads/2020/01/baked-potato.jpg&#39;)
    image3 = io.imread(&#39;https://www.southerncravings.com/wp-content/uploads/2020/01/Crispy-Baked-Potatoes-7.jpg&#39;)
    image1 = image1[:, 200:-200, :]
    image2 = image2[:-60, :-60, :]
    import numpy as np
    image3 = np.sum(image3[:-300, :-100, :], axis=2)
    
    def make_list_of_images(images_list):
        list_of_images = []
        for _image in images_list:
            if len(_image.shape) == 2:
                _image.shape += (1,)
            if _image.shape[2] &lt; np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) &gt; 2]):
                _image = np.concatenate([_image for _ in range(np.max([_image_channels.shape[2] for _image_channels in images_list if len(_image_channels.shape) &gt; 2]) - _image.shape[2] + 1)], axis=2)
            list_of_images.append(_image)
        return list_of_images
    
    list_of_images = make_list_of_images([image1, image2, image3])
    
    def normalize_rgb_values(rgb_values, max_value=1.0):
        norm_rgb_values = (rgb_values - np.mean(rgb_values)) / np.var(rgb_values)**0.5
        norm_rgb_values += abs(np.min(norm_rgb_values))
        norm_rgb_values *= (max_value / np.max(norm_rgb_values))
        return np.round(norm_rgb_values, decimals=0).astype(int) if max_value == 255 else np.round(norm_rgb_values, decimals=9).astype(float)
    
    from matplotlib import pyplot as plt
    for _image in list_of_images:
        if np.max(_image) &gt; 1.0:
            plt.imshow(normalize_rgb_values(_image))
        else:
            plt.imshow(_image)
        plt.show()
    
    max_number_of_channels = np.max([_image_channels.shape[2] for _image_channels in list_of_images])
    images = np.zeros((image1.shape[0], image1.shape[1], max_number_of_channels, len(list_of_images))).astype(float)
    for _image_num, _image in enumerate(list_of_images):
        images[:, :, :, _image_num] = _image
    
    summed_images = np.sum(images, axis=3)
    plt.imshow(normalize_rgb_values(summed_images))
    plt.show()

If you&#39;re `open`ing the image files with `Image` `from` `PIL` you might have to make it a `uint8` `type` `numpy` `array` first:

    import numpy as np
    image = normalize_rgb_values(np.array(image).astype(np.uint8))


  [1]: https://i.stack.imgur.com/NJbNQ.png
  [2]: https://i.stack.imgur.com/H1BQX.png
  [3]: https://i.stack.imgur.com/1Tmwp.png
  [4]: https://i.stack.imgur.com/uExSE.png
  [5]: https://i.stack.imgur.com/v3AH0.png





</details>