TEST BANK FOR Artificial Neural Networks By B. Yegnanarayana (Instructors Solution Manual)

Solution Manual to
Ariticial Neural Networks
(B. Yegnanarayana, Prentice Hall of India Pvt Ltd, New Delhi, 1999)
B. Yegnanarayana and S. Ramesh
Dept. of Computer Science and Engineering
Indian Institute of Technology Madras
Chennai - 600036
March 2001
CHAPTER 1
Solution to problem 1 (a):
s
-1
-1 q=0
+1
a1
a2
a3
Using the logic function,
f(x) = 1; x >
= 0; x
where x =
X
i
wiai, the truth table is obtained by giving all possible combinations of a1; a2; a3. The
results are shown in the following table.
Truth table:
a1 a2 a3 s
0 0 0 0
0 0 1 1
0 1 0 0
0 1 1 0
1 0 0 0
1 0 1 0
1 1 0 0
1 1 1 0
To derive the logic function, one can use the following Karnaugh map (K-map) representation of
the truth table. The logic function is obtained from the map by expressing the terms corresponding
to the entries with 1.
0 0
0
1 0 0
0 1 0 0
1 00 01 10
a2a3
11
a
Logic function s(a1; a2; a3) = a1a2a3
1
(b)
q=1 q=0
+1
-1
+1
+1 s
a2
1
a3
a
Truth table:
a1 a2 a3 s
0 0 0 0
0 0 1 1
0 1 0 0
0 1 1 1
1 0 0 0
1 0 1 1
1 1 0 0
1 1 1 0
K-map representation of the truth table is as follows:
0 1
0
1 0 0
0 1 1 0
00 01 11 10
a1
a2a3
Logic function s(a1; a2; a3) = a2a3 + a1a3
(c)
+1 q=0 q=-1 s
-1
+1
a -1
Truth table:
a s(t 􀀀 1) s(t)
0 0 1
0 1 1
1 0 0
1 1 1
2
K-map representation of the truth table is as follows:
1
1
1
0
a
1
1
0
s(t-1) 0
Logic function s(t) = a + s(t 􀀀 1)
Solution to problem 2:
Using the following basic logic networks
nor
q=1 s =
or
q=0
q=0 q= -1 s = =
s = a1+
+1
+1
+1
+1
+1
+1
-1
a1
a2
a1
a2
a1
a2
a1 a2
a2
a1+ a2 a1 a2
and
(a) The logic network for s(a1; a2; a3) = a1a3 + a2a3 + a1a3 is given by
q=0
q=0
q= -1
q=0
q=1
q=1
a3
s
+1
+1
+1
-1
a1 a1 a3
a2 a3
a1+ a3 a1 a3
a2
+1 +1
+1
+1
+1
+1
+1
3
(b) The logic network for s(a1; a2; a3) = a1a2a3 is given by (c)
q=1 +1 s
+1
+1
+1 q=1
a2
a1
a3
The logic network for s(a1; a2; a3) = a1a2a3 is given by
q=0 +1 +1
s
+1
a2
q=1
a3
a1
a1 a3
q= -1
a1+ a3
Solution to problem 3:
By applying the operations of an M-P neuron, the output of the network shown in Fig.P1.2 for the
input [1 1 1]T is s = 0.
Solution to problem 4:
For the output function: f(x) =
8><
>:
1; x > 0
0; x 0
Input vectors: i1 = [1 1 0 0]T ; i2 = [1 0 0 1]T ; i3 = [0 0 1 1]T ; i4 = [0 1 1 0]T
Choose the learning parameter = 1
Let the initial weight vector w(0) = [1 0 0 0]T
(a)
For the output function shown in Fig.P1.3b.
Output of the rst input vector s = f(wT (0)i1) = 1
Weight update w(1) = w(0) + si1 = [1 0 0 0]T + [1 1 0 0]T = [2 1 0 0]T
Output of the second input vector s = f(wT (1)i2) = 1
Weight update w(2) = w(1) + si2 = [2 1 0 0]T + [1 0 0 1]T = [3 1 0 1]T
4
Output of the third input vector s = f(wT (2)i3) = 1
Weight update w(3) = w(2) + si3 = [3 1 0 1]T + [0 0 1 1]T = [3 1 1 2]T
Output of the fourth input vector s = f(wT (3)i4) = 1
Weight update w(4) = w(3) + si4 = [3 1 1 2]T + [0 1 1 0]T = [3 2 2 2]T
(b)
For the sigmoid output function shown in Fig.P1.3c f(x) =
1
1 + e􀀀x
Output for the rst input vector s = f(wT (0)i1) = f(1) = 0:731
Weight update w(1) = w(0) + si1
= [1 0 0 0]T + 0:73 [1 1 0 0]T
= [1 0 0 0]T + [0:73 0:73 0 0]T
= [1:73 0:73 0 0]T
Output for the second input vector s = f(wT (1)i2) = f(1:73) = 0:85
Weight update w(2) = w(1) + si2
= [1:73 0:73 0 0]T + 0:85 [1 0 0 1]T
= [1:73 0:73 0 0]T + [0:85 0 0 0:85]T
= [2:58 0:73 0 0:85]T
Output for the third input vector s = f(wT (2)i3) = f(0:85) = 0:70
Weight update w(3) = w(2) + si3
= [2:58 0:73 0 0:85]T + 0:7 [0 0 1 1]T
= [2:58 0:73 0 0:85]T + [0 0 0:7 0:7]T
= [2:58 0:73 0:7 1:55]T
Output for the fourth input vector s = f(wT (3)i4) = f(1:43) = 0:81
Weight update w(4) = w(3) + si4
= [2:58 0:73 0:7 1:55]T + 0:81 [0 1 1 0]T
= [2:58 0:73 0:7 1:55]T + [0 0:81 0:81 0]T
= [2:58 1:54 1:51 1:55]T
5
The examples show that the successive weights accumulate the weighted sum of the input vectors.
Also the signicance of non-zero (arbitrary) initial weight vector can be seen. Note that with zero
initial weight vector there will be no change in the weights for the rst output function (Fig.P1.3b).
Solution to problem 5 (a):
Input: [1100]T [1001]T [0011]T [0110]T
Output: [11]T [10]T [01]T [00]T
Perceptron law applicable for binary output function: Weight update
wij = (bi 􀀀 si)aj ,
where si = f(x) =
8><
>:
1; x > 0
0; x 0

a1
a2
a3
a4
11
12
w13
w14
w w21 22
w23
w24
s2
s1
w
f(.)
f(.)
w
Choose the initial weights:
w1 = [w11 w12 w13 w14]T = [0:8 0:9 0:6 0:6]T
1 = 0:5
w2 = [w21 w22 w23 w24]T = [0:9 0:7 0:6 0:8]T
2 = 0:5
The weights and thresholds after 1000 iterations for dierent learning rate parameter:
w11 w12 w13 w14 1 w21 w22 w23 w24 2
0.3 1.1 0.6 0.0 0.6 0.8 0.6 0.4 0.3 0.5 1.1
0.1 0.8 0.6 0.1 0.4 1 0.5 0.4 0.4 0.5 1.1
0.04 0.8 0.66 0.2 0.44 0.9 0.5 0.46 0.48 0.52 1.02
0.01 0.8 0.66 0.22 0.46 0.88 0.5 0.49 0.5 0.51 1.0
6
The Matlab program used to implement this question is given below:
function[ ]=prob1_5a()
% Given Inputs and Desired Outputs
% The value '-1' at the end of each row of inputs will act as Bias input
Inputs = [1 1 0 0 -1;1 0 0 1 -1;0 0 1 1 -1;0 1 1 0 -1];
des_output = [1 1;1 0;0 1;0 0];
max_iteration = 1000;
% From eta value 1 to 0.1 in steps of 0.1
for eta = 1 : -0.1 : 0.1
% Next line can be uncommented to run program with random weights
% weights = randn(2,5);
% Next line can be commented while using random weights
weights = [.8 .9 .6 .6 .5;.9 .7 .6 .8 .5]; % Initial weights
for iteration = 1 : max_iteration
for i = 1 : 4
for k = 1 : 2
wted_sum = 0;
for j = 1 : 5
wted_sum = wted_sum + Inputs(i,j) * weights(k,j);
end
calc_output(k) = wted_sum;
end
wt_change=eta*(des_output(i,1)-sgn(calc_output(1)))*Inputs(i,:);
weights(1,:) = weights(1,:) + wt_change;
wt_change=eta*(des_output(i,2)-sgn(calc_output(2)))*Inputs(i,:);
weights(2,:) = weights(2,:) + wt_change;
end
end
end
7
(b)
Delta learning law for f(x) =
1
1 + e􀀀x
Weight update wij = (bi 􀀀 f(wTa))f_(wTa)aj
The network is trained using the same initial weight and bias values used in part (a).
Weights after 1000 iterations for dierent learning rate parameter:
w11 w12 w13 w14 1 w21 w22 w23 w24 2
0.3 3.9164 0.5149 -2.9916 0.5098 0.9753 0.4977 0.4955 0.4977 0.5000 1.0046
0.1 3.2949 0.5174 -2.3702 0.5073 0.9753 0.4982 0.4942 0.4957 0.4997 1.0061
0.04 2.7382 0.5241 -1.8135 0.5006 0.9753 0.5035 0.4830 0.4847 0.5052 1.0118
0.01 1.8260 0.5603 -0.8980 0.4677 0.9721 0.5428 0.4354 0.4372 0.5446 1.0200
The Matlab program used to implement this question is given below. In this program sigmoid and
derv sigmoid are function calls which implements sigmoidal function and rst derivative of sigmoidal
function.
function[ ]=prob1_5b()
% Given Inputs and Desired Outputs
% The value '-1' at the end of each row of inputs will act as Bias input
Inputs = [1 1 0 0 -1;1 0 0 1 -1;0 0 1 1 -1;0 1 1 0 -1];
des_output = [1 1;1 0;0 1;0 0];
output_error = 10;
max_iteration = 1000;
% From eta value 1 to 0.1 in steps of 0.1
for eta = 1 :-0.1 :0.1
% Next line can be uncommented to run program with random weights
% weights = randn(2,5);
% Next line can be commented while using random weights
weights = [.8 .9 .6 .6 .5;.9 .7 .6 .8 .5]; % Initial weights
for iteration = 1 : max_iteration
for i = 1 : 4
for k = 1 : 2
8
wted_sum = 0;
for j = 1 : 5
wted_sum = wted_sum + Inputs(i,j) * weights(k,j);
end
calc_output(k) = wted_sum;
end
wt_change = eta * (des_output(i,1) - sigmoid(calc_output(1))) *...
derv_sigmoid(calc_output(1)) * Inputs(i,:);
weights(1,:) = weights(1,:) + wt_change;
wt_change = eta * (des_output(i,2) - sigmoid(calc_output(2))) *...
derv_sigmoid(calc_output(1)) * Inputs(i,:);
weights(2,:) = weights(2,:) + wt_change;
end
end
end
Discussion: The weights will not be the same for dierent values of .
Solution to problem 6: