在Python中处理原始文件。数值错误:无法将大小为5038848的数组重新形状。

huangapple go评论67阅读模式
英文:

Working with raw file on Python. ValueError: cannot reshape array of size 5038848 into shape

问题

我使用Python编程语言编写了代码该代码应首先打开原始文件并将其输出为图像接下来您需要将像素拆分为RGB值并计算用于颜色校正的3x3矩阵但在此之前您需要进行黑电平校正
图片显示了算法的大致操作原始文件为14位具有RG模式
原始文件链接[text](https://drive.google.com/file/d/1TQQqcHVBw1OdQR9rR04MRanFdEeNXPkX/view?usp=drive_link)
[图像描述](https://i.stack.imgur.com/ltCJI.png)
代码

```python
import numpy as np
from PIL import Image

def open_raw_file(raw_file_path, width, height):
    with open(raw_file_path, 'rb') as file:
        raw_data = file.read()

    # 假设原始数据以RGB格式存在,每个像素由3个字节(R、G、B)组成
    # 根据您的原始文件格式调整数据格式(如果需要)
    image_data = np.frombuffer(raw_data, dtype=np.uint8)
    image_data = image_data.reshape((height, width, 3))

    return image_data

def apply_black_level_correction(image_data, black_level):
    corrected_image_data = np.maximum(image_data - black_level, 0)
    return corrected_image_data

def split_into_channels(image_data):
    red_channel = image_data[:, :, 0]
    green_channel = image_data[:, :, 1]
    blue_channel = image_data[:, :, 2]
    return red_channel, green_channel, blue_channel

def calculate_color_correction_matrix(image_data):
    # 假设需要一个3x3的颜色校正矩阵
    # 根据您的算法或要求计算矩阵
    color_correction_matrix = np.eye(3)  # 用于演示的单位矩阵
    return color_correction_matrix

def save_image(image_data, output_path):
    image = Image.fromarray(image_data)
    image.save(output_path)

# 主要代码
raw_file_path = 'macbeth_2592x1944_RG.raw'
output_image_path = 'output_image13.jpg'
width = 640  # 根据原始文件的尺寸进行调整
height = 480  # 根据原始文件的尺寸进行调整
black_level = 20  # 根据需要调整黑电平值

# 打开原始文件并将其转换为图像
image_data = open_raw_file(raw_file_path, width, height)

# 进行黑电平校正
corrected_image_data = apply_black_level_correction(image_data, black_level)

# 拆分为RGB通道
red_channel, green_channel, blue_channel = split_into_channels(corrected_image_data)

# 计算颜色校正矩阵
color_correction_matrix = calculate_color_correction_matrix(corrected_image_data)

# 执行颜色校正
red_channel_corrected = np.dot(red_channel, color_correction_matrix[0])
green_channel_corrected = np.dot(green_channel, color_correction_matrix[1])
blue_channel_corrected = np.dot(blue_channel, color_correction_matrix[2])

# 将颜色通道合并回图像
output_image_data = np.stack((red_channel_corrected, green_channel_corrected, blue_channel_corrected), axis=2)

# 保存校正后的图像
save_image(output_image_data, output_image_path)

我得到了这个错误:

Traceback (most recent call last):
File "C:\Users\DC\PycharmProjects\pythonProject4\lab13.py", line 43, in <module>
image_data = open_raw_file(raw_file_path, width, height)
File "C:\Users\DC\PycharmProjects\pythonProject4\lab13.py", line 11, in open_raw_file
image_data = image_data.reshape((height, width, 3))
ValueError: cannot reshape array of size 5038848 into shape (480,640,3)

所以我尝试运行这段代码时:

  1. 如我所说,原始文件是14位的,但我使用了8位(没有帮助)。
  2. 我尝试使用尺寸:宽640,高480和宽2592,高1944(原始原始尺寸)(没有帮助)。
  3. 我尝试更改reshape函数,将image_data = image_data.reshape((height, width, 3))更改为image_data = image_data.reshape(height, width)。这些都没有帮助我。
英文:

I wrote code in Python programming language. This code should first open the raw file and output it as an image. Next, you need to split the pixels into RGB values and calculate the 3x3 matrix for color correction. But before that, you need to do a black level correction.
The picture shows the approximate operation of the algorithm. Raw file is 14 bit and has RG pattern.
Link to raw file: text
enter image description here
The code:

import numpy as np
from PIL import Image
def open_raw_file(raw_file_path, width, height):
with open(raw_file_path, &#39;rb&#39;) as file:
raw_data = file.read()
# Assuming raw data is in RGB format, each pixel consists of 3 bytes (R, G, B)
# Adjust the data format based on your specific raw file format if necessary
image_data = np.frombuffer(raw_data, dtype=np.uint8)
image_data = image_data.reshape((height, width, 3))
return image_data
def apply_black_level_correction(image_data, black_level):
corrected_image_data = np.maximum(image_data - black_level, 0)
return corrected_image_data
def split_into_channels(image_data):
red_channel = image_data[:, :, 0]
green_channel = image_data[:, :, 1]
blue_channel = image_data[:, :, 2]
return red_channel, green_channel, blue_channel
def calculate_color_correction_matrix(image_data):
# Assuming a 3x3 color correction matrix is required
# Calculate the matrix based on your specific algorithm or requirements
color_correction_matrix = np.eye(3)  # Identity matrix for demonstration purposes
return color_correction_matrix
def save_image(image_data, output_path):
image = Image.fromarray(image_data)
image.save(output_path)
# Main code
raw_file_path = &#39;macbeth_2592x1944_RG.raw&#39;
output_image_path = &#39;output_image13.jpg&#39;
width = 640  # Adjust as per your raw file dimensions
height = 480  # Adjust as per your raw file dimensions
black_level = 20  # Adjust the black level value as per your requirement
# Open raw file and convert it into an image
image_data = open_raw_file(raw_file_path, width, height)
# Apply black level correction
corrected_image_data = apply_black_level_correction(image_data, black_level)
# Split into RGB channels
red_channel, green_channel, blue_channel = split_into_channels(corrected_image_data)
# Calculate color correction matrix
color_correction_matrix = calculate_color_correction_matrix(corrected_image_data)
# Perform color correction
red_channel_corrected = np.dot(red_channel, color_correction_matrix[0])
green_channel_corrected = np.dot(green_channel, color_correction_matrix[1])
blue_channel_corrected = np.dot(blue_channel, color_correction_matrix[2])
# Merge color channels back into image
output_image_data = np.stack((red_channel_corrected, green_channel_corrected, blue_channel_corrected), axis=2)
# Save the corrected image
save_image(output_image_data, output_image_path)

I got this error:

Traceback (most recent call last):
File &quot;C:\Users\DC\PycharmProjects\pythonProject4\lab13.py&quot;, line 43, in &lt;module&gt;
image_data = open_raw_file(raw_file_path, width, height)
File &quot;C:\Users\DC\PycharmProjects\pythonProject4\lab13.py&quot;, line 11, in open_raw_file
image_data = image_data.reshape((height, width, 3))
ValueError: cannot reshape array of size 5038848 into shape (480,640,3)

So i try to run this code with:

  1. As I said raw file is 14 bit, but I run it with 8 bit (didnt help).
  2. I tried with size: w:640, h:480 and w:2592, h:1944 (original raw size) (Didnt help).
  3. I tried to change reshape function changing image_data = image_data.reshape((height, width, 3)) to image_data = image_data.reshape(height, width).
    None of that helped me.

答案1

得分: 1

你分享的图片大小为10,077,696字节,假设文件名正确且图像为2592x1944像素,这意味着每个像素有两个字节,因为

2592 x 1944 x 2 = 10,077,696

因此,如果我使用ImageMagick将其解压为16位灰度图像,如下所示:

convert -depth 16 -size 2592x1944 gray:macbeth_2592x1944_RG.raw -verbose info:

我获得以下结果:

Image:
Filename: gray:macbeth_2592x1944_RG.raw
Format: GRAY (Raw gray samples)
Class: DirectClass
Geometry: 2592x1944+0+0
Units: Undefined
Colorspace: Gray
Type: Grayscale
Endianness: Undefined
Depth: 16-bit
Channel depth:
gray: 15-bit <--- 这里
Channel statistics:
Pixels: 5038848
Gray:
min: 808 (0.0123293) <--- 这里
max: 16256 (0.248051) <--- 这里
mean: 3913.4 (0.0597147)
...
...

我现在可以看到每个像素包含15位合法数据,并且样本的灰度范围为808到16256。

这使得它看起来很暗,因为没有超过0..65535的完整16位范围的25%。因此,我会应用自动级别来校正对比度并将其转换为JPEG,如下所示:

convert -depth 16 -size 2592x1944 gray:macbeth_2592x1944_RG.raw -auto-level a.jpg

在Python中处理原始文件。数值错误:无法将大小为5038848的数组重新形状。


因此,看起来2592x1944是正确的几何尺寸,并且每个像素有16位。名称暗示16位可能是8位红色和8位绿色或两者的某种贝尔混合。描述中的黑电平800看起来是正确的,因为灰度级别的范围为808到16256,这意味着正确的范围是8到15,456。


回到你的Python代码,这意味着你需要使用:

width = 2592
height = 1944

以及以下内容来读取每个像素的2个字节并进行正确的重塑:

image_data = np.frombuffer(raw_data, dtype=np.uint16).reshape((height, width))

英文:

The image you have shared is 10,077,696 bytes, and assuming the filename is correct and the image is 2592x1944, that means it has two bytes per pixel because

2592 x 1944 x 2 = 10,077,696

So, if I unpack it as 16-bit greyscale with ImageMagick like this:

convert -depth 16 -size 2592x1944 gray:macbeth_2592x1944_RG.raw -verbose info:

I obtain:

Image:
Filename: gray:macbeth_2592x1944_RG.raw
Format: GRAY (Raw gray samples)
Class: DirectClass
Geometry: 2592x1944+0+0
Units: Undefined
Colorspace: Gray
Type: Grayscale
Endianness: Undefined
Depth: 16-bit
Channel depth:
gray: 15-bit                    &lt;--- HERE
Channel statistics:
Pixels: 5038848
Gray:
min: 808  (0.0123293)         &lt;--- HERE
max: 16256 (0.248051)         &lt;--- HERE
mean: 3913.4 (0.0597147)
...
...

I can now see it contains 15-bits of legitimate data per pixel and the greyscale range of the samples is 808..16256.

That would make it dark, as nothing exceeds 25% of the full 16-bit range of 0..65535. So I would apply auto-levels to correct the contrast and make it into JPEG like this:

convert -depth 16 -size 2592x1944 gray:macbeth_2592x1944_RG.raw -auto-level a.jpg

在Python中处理原始文件。数值错误:无法将大小为5038848的数组重新形状。


It therefore looks like 2592x1944 is the correct geometry, and there are 16-bits per pixel. The name implies the 16-bits might be 8-bits of red and 8-bits of green or some Bayer-like mash-up of the two. The black level of 800 in the description looks like it is correct because the range of the grey levels is 808..16256, so that would mean the correct range is 8..15,456.


Coming back to your Python, that would mean you need to use:

width = 2592 
height = 1944

and the following to read 2 bytes per pixel and reshape correctly:

image_data = np.frombuffer(raw_data, dtype=np.uint16).reshape((height, width))

答案2

得分: 1

以下是翻译好的部分:

  • 我们可以从readme.txt和文件名中确定图像格式。

  • readme.txt的内容:

    • Bayer模式 = RG
    • 黑电平 = 800
    • bpp = 16
    • 位深度 = 14
  • 我们可以得出结论,图像采用Bayer CFA格式(可能是RGGB Bayer排列)。

  • 黑电平为800。

  • 每像素有16位(我们可以假设每像素是uint16元素)。

  • 白电平为2^14-1 = 16383(应用位深度=14)。

  • 文件名为macbeth_2592x1944_RG.raw

  • 我们可以得出分辨率为2592x1944(宽度=2592,高度=1944)。

  • 在Bayer镶嵌格式中,每个像素可以是红色、绿色或蓝色(这是一种类似灰度图像格式,但每个像素上的滤光片确定了颜色)。

  • 要将Bayer转换为RGB,可以使用OpenCV的cv2.cvtColor方法。

  • 从Bayer到RGB的转换过程被称为Demosaicing(去马赛克)。

  • Demosaicing算法的输入是单通道图像,输出是3通道的RGB图像。

  • 读取原始图像 - 确保以uint16类型读取2592x1944像素。

  • 黑电平校正后应用Demosaicing:

  • 以下是应用颜色校正的代码示例。

  • 在将图像保存为JPEG格式之前,我们必须将图像转换为uint8类型(像素范围[0, 255])。

  • 更新的代码示例。

  • 注意:输出由于缺乏颜色校正而呈绿色(这是正常的)。

请注意:上述是对给定代码段的翻译。如果您需要更多详细信息或具体问题的回答,请随时提出。

英文:

We may figure out the image format from the readme.txt and from the file name.

Content of readme.txt:

Bayer pattern = RG
black level = 800
bpp = 16
bit depth = 14

We may conclude that the image is in Bayer CFA format (probably RGGB Bayer alignment).
The black level is 800.
There are 16 bits per pixel (we may assume each pixel is uint16 element).
The white level is 2**14-1 = 16383 (applying bit depth = 14).

The file name is macbeth_2592x1944_RG.raw.
We may conclude that the resolution is 2592x1944 (width=2592, height=1944).


In Bayer mosaic format, each pixel may be either red, green or blue (it's kind of grayscale image format, but the filter on each pixel determines the color).
For converting Bayer to RGB, we may use OpenCV cv2.cvtColor method.
The process of converting from Bayer to RGB is named Demosaicing.
The input to the Demosaicing algorithm is single channel image, and the output is 3 channels RGB image.


Reading the raw image - make sure to read 2592x1944 pixels with uint16 type:

width = 2592
height = 1944
def open_raw_file(raw_file_path, width, height):
with open(raw_file_path, &#39;rb&#39;) as file:
raw_data = file.read()
image_data = np.frombuffer(raw_data, dtype=np.uint16)  # According to the readme.txt bpp = 16 (16 bits per pixel).
image_data = image_data.reshape(height, width)  # The image is in raw Bayer format - before Demosaicing there is only one channel.

After the black level correction apply Demosaicing:

rgb_image = cv2.cvtColor(corrected_image_data, cv2.COLOR_BayerBG2RGB)  # COLOR_BayerBG2RGB gives the best results.

There is an issue with the way color correction is applied, but since the matrix is an eye matrix, let's skip the details.


Before saving the image as JPEG format, we have to convert the image to type uint8 (pixel range [0, 255]).
We may apply linear transformation followed by rounding, clipping and casting:

while_level = 2**14 - 1  # bit depth = 14
output_image_data_8bpp = output_image_data.astype(np.float32) * (255 / (while_level - black_level))  # Scale to range [0, 255]
output_image_data_8bpp = output_image_data_8bpp.round().clip(0, 255).astype(np.uint8)  # Convert to uint8 with rounding and clipping. (JPEG format must be uint8).

Updated code sample:

import numpy as np
import cv2
from PIL import Image
def open_raw_file(raw_file_path, width, height):
with open(raw_file_path, &#39;rb&#39;) as file:
raw_data = file.read()
# Assuming raw data is in RGB format, each pixel consists of 3 bytes (R, G, B)
# Adjust the data format based on your specific raw file format if necessary
#image_data = np.frombuffer(raw_data, dtype=np.uint8)
image_data = np.frombuffer(raw_data, dtype=np.uint16)  # According to the readme.txt bpp = 16 (16 bits per pixel).
#image_data = image_data.reshape((height, width, 3))
image_data = image_data.reshape(height, width)  # The image is in raw Bayer format - before Demosaicing there is only one channel.
return image_data
def apply_black_level_correction(image_data, black_level):
corrected_image_data = np.maximum(image_data - black_level, 0)
return corrected_image_data
def split_into_channels(image_data):
red_channel = image_data[:, :, 0]
green_channel = image_data[:, :, 1]
blue_channel = image_data[:, :, 2]
return red_channel, green_channel, blue_channel
def calculate_color_correction_matrix(image_data):
# Assuming a 3x3 color correction matrix is required
# Calculate the matrix based on your specific algorithm or requirements
color_correction_matrix = np.eye(3)  # Identity matrix for demonstration purposes
return color_correction_matrix
def save_image(image_data, output_path):
image = Image.fromarray(image_data)
image.save(output_path)
# Main code
raw_file_path = &#39;macbeth_2592x1944_RG.raw&#39;
output_image_path = &#39;output_image13.jpg&#39;
# The file name is macbeth_2592x1944_RG.raw so the width is 2592 and the height is 1944
width = 2592  # 640  # Adjust as per your raw file dimensions
height = 1944  # 480  # Adjust as per your raw file dimensions
black_level = 800 #20  # Note: According readme.txt black level equals 800 # Adjust the black level value as per your requirement
# Open raw file and convert it into an image
image_data = open_raw_file(raw_file_path, width, height)
# Apply black level correction (it&#39;s fine to apply black level correction before Demosaicing).
corrected_image_data = apply_black_level_correction(image_data, black_level)
# The file name is macbeth_2592x1944_RG so the Bayer patten is RGGB.
# Apply Demosaicing - convert from Bayer CFA to RGB
rgb_image = cv2.cvtColor(corrected_image_data, cv2.COLOR_BayerBG2RGB)  # COLOR_BayerBG2RGB gives the best results.
# Split into RGB channels
#red_channel, green_channel, blue_channel = split_into_channels(rgb_image) # (corrected_image_data)
# Calculate color correction matrix
color_correction_matrix = calculate_color_correction_matrix(corrected_image_data)
# Perform color correction
# Color correction scale [r, g, b] triplet by column vector (or row vector by [r, g, b] triplet)
red_channel_corrected = np.dot(rgb_image, color_correction_matrix[0])  #np.dot(red_channel, color_correction_matrix[0])
green_channel_corrected = np.dot(rgb_image, color_correction_matrix[1])  #np.dot(green_channel, color_correction_matrix[1])
blue_channel_corrected = np.dot(rgb_image, color_correction_matrix[2])  #np.dot(blue_channel, color_correction_matrix[2])
# Merge color channels back into image
output_image_data = np.stack((red_channel_corrected, green_channel_corrected, blue_channel_corrected), axis=2)
# According to readme.txt bit depth = 14
# We may assume that the white level is 2**14-1 = 16383.
# Convert from 16 bpp to 8 bits in range [0, 255] using linear transformation:  
while_level = 2**14 - 1  # bit depth = 14
output_image_data_8bpp = output_image_data.astype(np.float32) * (255 / (while_level - black_level))  # Scale to range [0, 255]
output_image_data_8bpp = output_image_data_8bpp.round().clip(0, 255).astype(np.uint8)  # Convert to uint8 with rounding and clipping. (JPEG format must be uint8).
# Save the corrected image
save_image(output_image_data_8bpp, output_image_path)

Output image (downscaled):

在Python中处理原始文件。数值错误:无法将大小为5038848的数组重新形状。

Note:
The output is greenish due to the lack of color correction (this is normal).

huangapple
  • 本文由 发表于 2023年5月26日 16:01:50
  • 转载请务必保留本文链接:https://go.coder-hub.com/76338814.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定