I'm designing a NeuralNetwork class to, well, be a neural network. Currently it uses a genetic algorithm to learn, but I'm trying to make it as well rounded as possible, so I'm writing a backpropagation algorithm for it as well. My question is, does anyone here know anything about the backpropagation algorithm, and if so, would you please let me know how to correctly update the bias on a neuron?
Thanks in advance!
The output you get is compared with the output you expect. Using some forumla you compute the error between the two. A common form of error is the total squared error between the output, (O_q), and the expected output, (r_q). ("_" means subscript)
therefore the output error is E_q = sigma_1^q (r_q - O-q)
sum of the difference between an output and its expected output from all nodes (of which
there are q). sigma_1^q is the sum of bla between 1 and q.
That gives you the output error (well, at least one version of it).
Now it gets more complex. You can only perform back-prop if the transfer functions of
your nuerons can be differentiated. (often a sigmoidal transfer functionis used) (this is
not actually true but assume it is as non-dfferentialble functions cause all kinds of
havoc to break loose). The transfer function is the bit you do after taking the inputs to
a neuron, multiplying them with thier coeefficients and summing the result.
I just realised. Do you mean the Bias node that each nueron has? The one that is stuck at value 1. If so then it is treated like any other input to a neuron. It has an associated weight (cooefficient) that is updated in exactly the same manner as all other inputs to the nueron (even though it is often treated as a virtual input to a node)
matfud