英文:
Keras custom Optimizer ValueError: Missing learning rate
问题
I'm trying to create a custom optimizer using the Keras library in TensorFlow. I'm coding the optimizer from scratch. It's showing the following error:
ValueError: Missing learning rate, please set self.learning_rate at optimizer creation time.
In my optimizer's creation, I'm adding the self.learning_rate
and still the problem persists. Here's my code for the __init__
function of the optimizer:
def __init__(self, lr=0.001, beta_1=0.9, beta_2=0.999,
momentum=0.0, epsilon=None, decay=0.0, name='AdamSGD', **kwargs):
super(AdamSGD, self).__init__(name=name, **kwargs)
self.iterations = K.variable(0, dtype='int64', name='iterations')
self.learning_rate = K.variable(lr, name='lr')
self.beta_1 = K.variable(beta_1, name='beta_1')
self.beta_2 = K.variable(beta_2, name='beta_2')
self.momentum = K.variable(momentum, name='momentum')
self.epsilon = epsilon or K.epsilon()
self.decay = K.variable(decay, name='decay')
This is how I'm compiling my model and where the error is being triggered:
model.compile(
optimizer=AdamSGD(lr=0.01),
loss='categorical_crossentropy',
metrics=['accuracy']
)
The entire error:
24 # Compile the model
25 model.compile(
---> 26 optimizer=AdamSGD(lr=0.01),
27 loss='categorical_crossentropy',
28 metrics=['accuracy']
24 self.iterations = K.variable(0, dtype='int64', name='iterations')
---> 25 self.learning_rate = K.variable(lr, name='lr')
26 self.beta_1 = K.variable(beta_1, name='beta_1')
27 self.beta_2 = K.variable(beta_2, name='beta_2')
60 try:
---> 61 if getattr(self, name) is value:
62 # Short circuit for `self.$x = self.$x`.
63 return
330 def learning_rate(self):
331 if not hasattr(self, "_learning_rate") or self._learning_rate is None:
--> 332 raise ValueError(
333 "Missing learning rate, please set self.learning_rate at"
334 " optimizer creation time."
ValueError: Missing learning rate, please set self.learning_rate at optimizer creation time.
If someone has any suggestions or knows where I'm going wrong, please let me know as I'm stuck on this error for quite some time now.
英文:
I'm trying to create a custom optimizer using the Keras library in TensorFlow. I'm coding the optimizer from scratch. It's showing the following error:
ValueError: Missing learning rate, please set self.learning_rate at optimizer creation time.
In my optimizer's creation, I'm adding the self.learning_rate
and still the problem persists. Here's my code for the init function of the optimizer:
def __init__(self, lr=0.001, beta_1=0.9, beta_2=0.999,
momentum=0.0, epsilon=None, decay=0.0, name='AdamSGD', **kwargs):
super(AdamSGD, self).__init__(name=name, **kwargs)
self.iterations = K.variable(0, dtype='int64', name='iterations')
self.learning_rate = K.variable(lr, name='lr')
self.beta_1 = K.variable(beta_1, name='beta_1')
self.beta_2 = K.variable(beta_2, name='beta_2')
self.momentum = K.variable(momentum, name='momentum')
self.epsilon = epsilon or K.epsilon()
self.decay = K.variable(decay, name='decay')
This is how I'm compiling my model and where the error is being triggered:
model.compile(
optimizer=AdamSGD(lr=0.01),
loss='categorical_crossentropy',
metrics=['accuracy']
)
The entire error:
24 # Compile the model
25 model.compile(
---> 26 optimizer=AdamSGD(lr=0.01),
27 loss='categorical_crossentropy',
28 metrics=['accuracy']
24 self.iterations = K.variable(0, dtype='int64', name='iterations')
---> 25 self.learning_rate = K.variable(lr, name='lr')
26 self.beta_1 = K.variable(beta_1, name='beta_1')
27 self.beta_2 = K.variable(beta_2, name='beta_2')
60 try:
---> 61 if getattr(self, name) is value:
62 # Short circuit for `self.$x = self.$x`.
63 return
330 def learning_rate(self):
331 if not hasattr(self, "_learning_rate") or self._learning_rate is None:
--> 332 raise ValueError(
333 "Missing learning rate, please set self.learning_rate at"
334 " optimizer creation time."
ValueError: Missing learning rate, please set self.learning_rate at optimizer creation time.
If someone has any suggestions or knows where I'm going wrong, please let me know as I'm stuck on this error for quite some time now.
答案1
得分: 2
错误消息看起来有些反直觉,但你应该在你的构造函数中“简单地”设置属性 self._learning_rate
。你可以使用函数 self._build_learning_rate(learning_rate)
来做到这一点。
class AdamSGD(keras.optimizers.Optimizer):
def __init__(self, lr=0.001, beta_1=0.9, beta_2=0.999,
momentum=0.0, epsilon=None, decay=0.0, name='AdamSGD', **kwargs):
super(AdamSGD, self).__init__(name=name, **kwargs)
self.iterations = K.variable(0, dtype='int64', name='iterations')
self._learning_rate = self._build_learning_rate(lr)
self.beta_1 = K.variable(beta_1, name='beta_1')
self.beta_2 = K.variable(beta_2, name='beta_2')
self.momentum = K.variable(momentum, name='momentum')
self.epsilon = epsilon or K.epsilon()
self.decay = K.variable(decay, name='decay')
此函数使优化器能够接受调度或浮点数值作为学习率。
你可能还需要实现一些额外的方法,参见代码中的这条评论:
创建自定义优化器
如果你打算创建自己的优化算法,请继承自这个类,并覆盖以下方法:
build
: 创建与优化器相关的变量,比如在 SGD 优化器中的momentums
。update_step
: 实现你的优化器的更新逻辑。get_config
: 优化器的序列化,包括所有超参数。
英文:
The error message is counter intuitive, but you should "simply" set the attribute self._learning_rate
in your constructor. You can do so with the function self._build_learning_rate(learning_rate)
.
class AdamSGD(keras.optimizers.Optimizer):
def __init__(self, lr=0.001, beta_1=0.9, beta_2=0.999,
momentum=0.0, epsilon=None, decay=0.0, name='AdamSGD', **kwargs):
super(AdamSGD, self).__init__(name=name, **kwargs)
self.iterations = K.variable(0, dtype='int64', name='iterations')
self._learning_rate = self._build_learning_rate(lr)
self.beta_1 = K.variable(beta_1, name='beta_1')
self.beta_2 = K.variable(beta_2, name='beta_2')
self.momentum = K.variable(momentum, name='momentum')
self.epsilon = epsilon or K.epsilon()
self.decay = K.variable(decay, name='decay')
This function enables the optimizer to accept both a schedule or a float value as learning rate.
You might also need to implement some extra methods, see this comment in the code:
> ### Creating a custom optimizer
> If you intend to create your own optimization algorithm, please inherit from
> this class and override the following methods:
> - build
: Create your optimizer-related variables, such as momentums
in SGD optimizer.
> - update_step
: Implement your optimizer's updating logic.
> - get_config
: serialization of the optimizer, include all hyper parameters.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论