Файл: Embedded Robotics (Thomas Braunl, 2 ed, 2006).pdf

19 Neural Networks

•Weights from the input layer to the hidden layer, summarized as ma-

trix win i,j (weight of connection from input neuron i to hidden neuron j).

•Weights from the hidden layer to the output layer, summarized as ma-

trix wout i,j (weight of connection from hidden neuron i to output neuron j).

No weights are required from sensors to the first layer or from the output layer to actuators. These weights are just assumed to be always 1. All other weights are normalized to the range [–1 .. +1].

Figure 19.6: Feed-forward evaluation

Calculation of the output function starts with the input layer on the left and propagates through the network. For the input layer, there is one input value (sensor value) per input neuron. Each input data value is used directly as neuron activation value:

a(nin1) = o(nin1) = 1.00 a(nin2) = o(nin2) = 0.50

For all subsequent layers, we first calculate the activation function of each neuron as a weighted sum of its inputs, and then apply the sigmoid output function. The first neuron of the hidden layer has the following activation and output values:

a(nhid1) = 1.00 · 0.2 + 0.50 · 0.3 = 0.35 o(nhid1) = 1 / (1 + e–0.35) = 0.59

The subsequent steps for the remaining two layers are shown in Figure 19.6 with the activation values printed in each neuron symbol and the output values below, always rounded to two decimal places.

Once the values have percolated through the feed-forward network, they will not change until the input values change. Obviously this is not true for networks with feedback connections. Program 19.1 shows the implementation of

282

19 Neural Networks

only incremental feedback for input/output pairs is reinforcement learning [Sutton, Barto 1998]. This on-line technique can be seen as either supervised or unsupervised, since the feedback signal only refers to the network’s current performance and does not provide the desired network output. In the following, the backpropagation method is presented.

A feed-forward neural network starts with random weights and is presented a number of test cases called the training set. The network’s outputs are compared with the known correct results for the particular set of input values and any deviations (error function) are propagated back through the net.

Having done this for a number of iterations, the NN hopefully has learned the complete training set and can now produce the correct output for each input pattern in the training set. The real hope, however, is that the network is able to generalize, which means it will be able to produce similar outputs corresponding to similar input patterns it has not seen before. Without the capability of generalization, no useful learning can take place, since we would simply store and reproduce the training set.

The backpropagation algorithm works as follows:

1.Initialize network with random weights.

2.For all training cases:

a.Present training inputs to network and calculate output.

b.For all layers (starting with output layer, back to input layer):

i.Compare network output with correct output (error function).

ii.Adapt weights in current layer.

For implementing this learning algorithm, we do know what the correct results for the output layer should be, because they are supplied together with the training inputs. However, it is not yet clear for the other layers, so let us do this step by step.

Firstly, we look at the error function. For each output neuron, we compute the difference between the actual output value outi and the desired output dout i. For the total network error, we calculate the sum of square difference:

The next step is to adapt the weights, which is done by a gradient descent approach:

So the adjustment of the weight will be proportional to the contribution of the weight to the error, with the magnitude of change determined by constant

284

Backpropagation

K. This can be achieved by the following formulas [Rumelhart, McClelland 1986]:

dout1 = 1.0, and choosing K = 0.5 to further simplify the formula, we can now update the four weights between the hidden layer and the output layer. Note

that all calculations have been performed with full floating point accuracy, while only two or three digits are printed.

The only remaining step is to adapt the win weights. Using the same formula, we need to know what the desired outputs dhid k are for the hidden layer. We get these values by backpropagating the error values from the output layer multiplied by the activation value of the corresponding neuron in the hidden layer, and adding up all these terms for each neuron in the hidden layer. We could also use the difference values of the output layer instead of the error values, however we found that using error values improves convergence. Here, we use the old (unchanged) value of the connection weight, which again improves convergence. The error formula for the hidden layer (difference between desired and actual hidden value) is:

In the example in Figure 19.5, there is only one output neuron, so each hidden neuron has only a single term for its desired value. The value and difference values for the first hidden neuron are therefore:

285

19 Neural Networks

Ehid 1 = Eout 1 · wout 1,1

= 0.4 · 0.8 = 0.32

diffhid 1 = Ehid 1 · (1 – o(nhid 1)) · o(nhid 1) = 0.32 · (1 – 0.59) · 0.59 = 0.077

Program 19.2: Backpropagation execution

1float backprop(float train_in[NIN], float train_out[NOUT])

2/* returns current square error value */

3{ int i,j;

4float err_total;

5float N_out[NOUT],err_out[NOUT];

6float diff_out[NOUT];

7float N_hid[NHID], err_hid[NHID], diff_hid[NHID];

8

9//run network, calculate difference to desired output

10feedforward(train_in, N_hid, N_out);

11err_total = 0.0;

12for (i=0; i<NOUT; i++)

13{ err_out[i] = train_out[i]-N_out[i];

14diff_out[i]= err_out[i] * (1.0-N_out[i]) * N_out[i];

15err_total += err_out[i]*err_out[i];

16}

17

18// update w_out and calculate hidden difference values

19for (i=0; i<NHID; i++)

20{ err_hid[i] = 0.0;

21for (j=0; j<NOUT; j++)

22{ err_hid[i] += err_out[j] * w_out[i][j];

23w_out[i][j] += diff_out[j] * N_hid[i];

24}

25diff_hid[i] = err_hid[i] * (1.0-N_hid[i]) * N_hid[i];

26}

27

28// update w_in

29for (i=0; i<NIN; i++)

30for (j=0; j<NHID; j++)

31w_in[i][j] += diff_hid[j] * train_in[i];

33return err_total;

34}

The weight changes for the two connections from the input layer to the first hidden neuron are as follows. Remember that the input of the hidden layer is the output of the input layer:

'win k,i = 2 · K · diffhid i · inputk(nhid i) for K = 0.5

= diffhid i · o(nin k)

286