以下参数不支持使用原生Keras格式:[‘options’]

huangapple go评论54阅读模式
英文:

The following argument(s) are not supported with the native Keras format: ['options']

问题

我正在构建一个基于狗与猫数据集的Keras深度学习算法。我可以在Colab中运行我的代码,但在Jupyter lab中却出现了以下错误:

"不支持以下参数的原生Keras格式:['options']"

以下是代码:

import os
import shutil
import pathlib

original_dir = pathlib.Path("/content/drive/MyDrive/Reva/dogs_vs_cats/train/train")
new_base_dir = pathlib.Path("/content/drive/MyDrive/Reva/dogs_vs_cats/")

def make_subset(subset_name, start_index, end_index):
    for category in ("cat", "dog"):
        dir = new_base_dir / subset_name / category

        # 检查文件夹是否存在,如果存在则删除
        if os.path.exists(dir):
            shutil.rmtree(dir)

        # 再次创建文件夹
        os.makedirs(dir)

        fnames = [f"{category}.{i}.jpg" for i in range(start_index, end_index)]
        for fname in fnames:
            shutil.copyfile(src=original_dir / fname,
                            dst=dir / fname)

make_subset("train", start_index=0, end_index=1000)
make_subset("validation", start_index=1000, end_index=1500)
make_subset("test", start_index=1500, end_index=2500)

from tensorflow.keras.utils import image_dataset_from_directory

train_dataset = image_dataset_from_directory(
    new_base_dir / "train",
    image_size=(180, 180),
    batch_size=32)
validation_dataset = image_dataset_from_directory(
    new_base_dir / "validation",
    image_size=(180, 180),
    batch_size=32)
test_dataset = image_dataset_from_directory(
    new_base_dir / "test",
    image_size=(180, 180),
    batch_size=32)

from tensorflow import keras
from tensorflow.keras import layers

inputs = keras.Input(shape=(180, 180, 3))
x = layers.Rescaling(1./255)(inputs)
x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)

model.compile(loss="binary_crossentropy",
              optimizer="rmsprop",
              metrics=["accuracy"])

callbacks = [
    keras.callbacks.ModelCheckpoint(
        filepath="convnet_from_scratch.keras",
        save_best_only=True,
        monitor="val_loss")
]

history = model.fit(
    train_dataset,
    epochs=30,
    validation_data=validation_dataset,
    callbacks=callbacks)

我需要知道如何解决上面的代码。欢迎任何提高运行代码所需时间的建议。

英文:

I am building a Keras deep learning Algorithm on Dogs vs cats dataset. I am able to run my code in colab. But in Jupyter lab I am getting this error.

"The following argument(s) are not supported with the native Keras format: ['options']"

Below is the code:

import os
import shutil
import pathlib
original_dir = pathlib.Path("/content/drive/MyDrive/Reva/dogs_vs_cats/train/train")
new_base_dir = pathlib.Path("/content/drive/MyDrive/Reva/dogs_vs_cats/")
def make_subset(subset_name, start_index, end_index):
for category in ("cat", "dog"):
dir = new_base_dir / subset_name / category
# Check if the folder exists and delete it if it does
if os.path.exists(dir):
shutil.rmtree(dir)
# Create the folder again
os.makedirs(dir)
fnames = [f"{category}.{i}.jpg" for i in range(start_index, end_index)]
for fname in fnames:
shutil.copyfile(src=original_dir / fname,
dst=dir / fname)
make_subset("train", start_index=0, end_index=1000)
make_subset("validation", start_index=1000, end_index=1500)
make_subset("test", start_index=1500, end_index=2500)
from tensorflow.keras.utils import image_dataset_from_directory
train_dataset = image_dataset_from_directory(
new_base_dir / "train",
image_size=(180, 180),
batch_size=32)
validation_dataset = image_dataset_from_directory(
new_base_dir / "validation",
image_size=(180, 180),
batch_size=32)
test_dataset = image_dataset_from_directory(
new_base_dir / "test",
image_size=(180, 180),
batch_size=32)
from tensorflow import keras
from tensorflow.keras import layers
inputs = keras.Input(shape=(180, 180, 3))
x = layers.Rescaling(1./255)(inputs)
x = layers.Conv2D(filters=32, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=64, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=128, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.MaxPooling2D(pool_size=2)(x)
x = layers.Conv2D(filters=256, kernel_size=3, activation="relu")(x)
x = layers.Flatten()(x)
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs=inputs, outputs=outputs)
model.compile(loss="binary_crossentropy",
optimizer="rmsprop",
metrics=["accuracy"])
callbacks = [
keras.callbacks.ModelCheckpoint(
filepath="convnet_from_scratch.keras",
save_best_only=True,
monitor="val_loss")
]
history = model.fit(
train_dataset,
epochs=30,
validation_data=validation_dataset,
callbacks=callbacks)

I need to know how to resolve the above code. Any suggestions to improve the time required to run the code is also welcome.

答案1

得分: 1

TensorFlow团队没有修改ModelCheckpoint。错误发生是因为在保存模型时不应该有一个叫做"options"的参数,但ModelCheckpoint包括了它。

解决方法是手动删除那部分。前往定义ModelCheckpoint的位置(如果您使用VSCode,可以很容易地完成)。

使用Ctrl+F查找以下行:
"options=self._options,"

将所有出现的内容替换为空格并保存。这将解决问题。

英文:

TensorFlow team did not modify the ModelCheckpoint. The error occurs because there should not be an argument called "options" when saving the model, but ModelCheckpoint includes it.

The solution is to remove that part manually. Go to the location where ModelCheckpoint is defined (if you are using VSCode, it can be easily done).

Find the following line using Ctrl+F:
"options=self._options,"

Replace all occurrences with empty spaces and save. This will resolve the issue.

答案2

得分: 1

正如我在评论中提到的,与Keras保存和TF/Keras版本相关的奇怪行为似乎存在问题。在colab上运行TF/Keras版本2.13(目前最新版本)时,我能够复制您的错误。在colab上的标准安装版本是2.12,错误不会出现。

因此,一个解决方案是将TF/Keras降级到2.12.x,或者将以下代码:

keras.callbacks.ModelCheckpoint(
        filepath="convnet_from_scratch.keras",
        ..)

更改为:

keras.callbacks.ModelCheckpoint(
        filepath="convnet_from_scratch.x",
        ..)

其中的x可以是您喜欢的任何内容(不要使用"keras"),以避免保存为.keras格式。

英文:

As I mentioned in the comments, there seems to be a weird behaviour related to keras saving and also versioning of TF/Keras. I could replicate your error when running TF/Keras with version 2.13 (newest right now) on colab. Standard install on colab is 2.12, where the error doesn't come up.
So one solution would be to downgrade TF/Keras to 2.12.x, or change

keras.callbacks.ModelCheckpoint(
filepath="convnet_from_scratch.keras",
..)

to

keras.callbacks.ModelCheckpoint(
filepath="convnet_from_scratch.x",
..)

where x stands for whatever you fancy (NOT "keras") to not save in the .keras format.

答案3

得分: 0

最好是创建自己的ModelCheckPoint函数:

def save_model_checkpoint(epoch, logs):
    if logs['val_loss'] < save_model_checkpoint.best_val_loss:
        save_model_checkpoint.best_val_loss = logs['val_loss']
        model.save_weights(new_base_dir / 'model_checkpoint')
        print('模型检查点已保存。')

save_model_checkpoint.best_val_loss = float('inf')

# 在这里初始化你的模型

model.fit(
    # 其他model.fit参数...
    callbacks=[tf.keras.callbacks.LambdaCallback(on_epoch_end=save_model_checkpoint)]
)

save_model_checkpoint函数用于监控验证损失并保存最佳模型的权重。它在每个周期结束时被调用。
save_model_checkpoint函数的best_val_loss属性被初始化为float('inf'),以跟踪到目前为止见过的最佳验证损失。
在函数内部,如果当前验证损失(logs['val_loss'])低于best_val_loss,则更新best_val_loss,并使用model.save_weights()保存模型权重。你可以根据需要自定义保存路径和文件名。
model.fit()调用期间,你可以使用tf.keras.callbacks.LambdaCallback(on_epoch_end=save_model_checkpoint)save_model_checkpoint函数作为回调传递,以确保在每个周期结束时调用save_model_checkpoint函数。

英文:

It is best if you create your own ModelCheckPoint function:

def save_model_checkpoint(epoch, logs):
if logs[&#39;val_loss&#39;] &lt; save_model_checkpoint.best_val_loss:
save_model_checkpoint.best_val_loss = logs[&#39;val_loss&#39;]
model.save_weights(new_base_dir / &#39;model_checkpoint&#39;)
print(&#39;Model checkpoint saved.&#39;)
save_model_checkpoint.best_val_loss = float(&#39;inf&#39;)
# Initialize your model here
model.fit(
# Other model.fit arguments...
callbacks=[tf.keras.callbacks.LambdaCallback(on_epoch_end=save_model_checkpoint)]
)

The save_model_checkpoint function is defined to monitor the validation loss and save the weights of the best model. It is called at the end of each epoch.
The best_val_loss attribute of the save_model_checkpoint function is initialized to float(&#39;inf&#39;) to track the best validation loss seen so far.
Inside the function, if the current validation loss (logs[&#39;val_loss&#39;]) is lower than the best_val_loss, the best_val_loss is updated, and the model weights are saved using model.save_weights(). You can customize the saving path and filename as needed.
During the model.fit() call, you can pass the save_model_checkpoint function as a callback using tf.keras.callbacks.LambdaCallback(on_epoch_end=save_model_checkpoint). This ensures that the save_model_checkpoint function is called at the end of each epoch.

huangapple
  • 本文由 发表于 2023年7月17日 13:04:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/76701617.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定