英文:
Neural network with sigmoid function always get value closer to 0.5 fo XOR
问题
我正在从头开始练习使用神经网络来处理XOR问题。然而,无论接收到什么输入,神经网络的输出总是接近0.5。
以下是我的代码
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import pandas as pd
def generate_XOR_easy():
inputs = []
labels = []
for i in range(11):
hasil = [round(0.1*i, 1), round(0.1*i, 1)]
inputs.append(hasil)
labels.append(0)
if 0.1*i == 0.5:
continue
hasil2 = [round(0.1*i,1), round(1-0.1*i, 1)]
inputs.append(hasil2)
labels.append(1)
return np.array(inputs), np.array(labels).reshape(21,1)
def sigmoid(x):
return 1.0/(1.0+np.exp(-x))
def deriv_sig(x):
return np.multiply(x, 1.0-x)
epochs = 1000
n_input = 2
hidden_one = 3
hidden_two = 3
n_output = 1
learning_rate = 0.001
np.random.seed(10)
weigth_one = np.random.rand(n_input, hidden_one)
weight_two = np.random.rand(hidden_one, hidden_two)
weight_three = np.random.rand(hidden_two, n_output)
xor_data = generate_XOR_easy()
x_train, x_test, y_train, y_test = train_test_split(xor_data[0], xor_data[1], test_size=0.2, random_state=4)
for epoch in range(epochs):
#前馈
input_to_h1 = np.dot(x_train, weigth_one)
output_h1 = sigmoid(input_to_h1)
input_to_h2 = np.dot(output_h1, weight_two)
output_h2 = sigmoid(input_to_h2)
input_to_output = np.dot(output_h2, weight_three)
output_output = sigmoid(input_to_output)
#反向传播
error = output_output - y_train
error_2_1 = error * deriv_sig(output_output)
error_1_w3 = np.dot(output_h2.T, error_2_1)
w3_error1 = np.dot(error_2_1, weight_three.T)
h2_front = w3_error1 * deriv_sig(output_h2)
h2Back_w2 = np.dot(output_h1.T, h2_front)
h2Back_h1front = np.dot(h2_front, weight_two.T)
h1front_h1back = h2Back_h1front * deriv_sig(output_h1)
h1back_w1 = np.dot(x_train.T, h1front_h1back)
weight_three -= learning_rate * error_1_w3
weight_two -= learning_rate * h2Back_w2
weigth_one -= learning_rate * h1back_w1
当我在每次迭代时打印输出时,结果会逐渐接近0.5(对于所有输入)。由于我还是新手,需要更多练习机器学习和深度学习代码,所以代码看起来有点混乱。有人知道问题出在哪吗?我非常感谢任何帮助和建议。谢谢。
英文:
I am practicing with neural network from scratch to handle XOR problem. However, the neural network output always getting close to 0.5 no matter what input it received.
Here is my code
import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
import pandas as pd
def generate_XOR_easy():
inputs = []
labels = []
for i in range(11):
hasil = [round(0.1*i, 1), round(0.1*i, 1)]
inputs.append(hasil)
labels.append(0)
if 0.1*i == 0.5:
continue
hasil2 = [round(0.1*i,1), round(1-0.1*i, 1)]
inputs.append(hasil2)
labels.append(1)
return np.array(inputs), np.array(labels).reshape(21,1)
def sigmoid(x):
return 1.0/(1.0+np.exp(-x))
def deriv_sig(x):
return np.multiply(x, 1.0-x)
epochs = 1000
n_input = 2
hidden_one = 3
hidden_two = 3
n_output = 1
learning_rate = 0.001
np.random.seed(10)
weigth_one = np.random.rand(n_input, hidden_one)
weight_two = np.random.rand(hidden_one, hidden_two)
weight_three = np.random.rand(hidden_two, n_output)
xor_data = generate_XOR_easy()
x_train, x_test, y_train, y_test = train_test_split(xor_data[0], xor_data[1],
test_size=0.2, random_state=4)
for epoch in range(epochs):
#feedforward
input_to_h1 = np.dot(x_train, weigth_one)
output_h1 = sigmoid(input_to_h1)
input_to_h2 = np.dot(output_h1, weight_two)
output_h2 = sigmoid(input_to_h2)
input_to_output = np.dot(output_h2, weight_three)
output_output = sigmoid(input_to_output)
#backpropagation
error = output_output - y_train
error_2_1 = error * deriv_sig(output_output)
error_1_w3 = np.dot(output_h2.T, error_2_1)
w3_error1 = np.dot(error_2_1, weight_three.T)
h2_front = w3_error1 * deriv_sig(output_h2)
h2Back_w2 = np.dot(output_h1.T, h2_front)
h2Back_h1front = np.dot(h2_front, weight_two.T)
h1front_h1back = h2Back_h1front * deriv_sig(output_h1)
h1back_w1 = np.dot(x_train.T, h1front_h1back)
weight_three -= learning_rate * error_1_w3
weight_two -= learning_rate * h2Back_w2
weigth_one -= learning_rate * h1back_w1
When I print the output every iteration, the result will go closer to 0.5 (for all input).
The code is very messy since I am still new and need more practice with machine learning and deep learning code. Is there anyone know whats wrong? I really appreciate any help and suggestion.
Thank you
答案1
得分: 0
我自己找出了问题,实际上它很简单。在我增加学习率和迭代次数后,我获得了很好的结果。我希望我的问题和解决方案可能会帮助其他遇到相同问题的人。
英文:
I figured out the problem by myself and actually it is very simple. I got good result after I increase the learning rate and number of iteration. I hope my problem and solution might help other who got the same problem.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论