Professional Documents
Culture Documents
AI010 804L01 Neural Networks
AI010 804L01 Neural Networks
Module 1
What Are Artificial Neural Networks?
An extremely simplified model of the brain/biological
neuron system
Essentially a function approximator
Transforms inputs into outputs to the best of its ability
It is a massively parallel distributed processing system
madeup of highly interconnected neural computing
elements that have the ability to learn and there by acquire
knowledge and make it available for use
Classification
Pattern recognition
feature extraction
image matching
Noise Reduction
Recognize patterns in the inputs and produce noiseless
outputs
Prediction
Extrapolation based on historical data
Why Use Neural Networks?
Ability to learn
NN’s figure out how to perform their function on their own
Determine their function based only upon sample inputs
Ability to generalize
i.e. produce reasonable outputs for inputs it has not been taught
how to deal with
Human Brain
Composed of many “neurons” that co-operate to perform the
desired function
It is the increased neural activity which is responsible for
learning and memory
I = w1x1+w2x2+w3x3+ .............+wnxn
Activation functions
To generate the final output Y , the sum is passed through a
non linear fiter Ø called activation function or transfer
function or squash function
Y= Ø (I)
a) Thresholding function
sum is compared with a thershold value
b) Signum function
also called quantizer function
c) Sigmoidal function
Continuous function varies gradually between the
asymptotic values 0 and 1 or -1 and +1
Differentiable
d) Hyperbolic tangent function
it can produce negative output values
Typical architectures
Represented by directed graph
Graph G is an ordered 2- duple (V,E) consisting of a set V
of vertices and a set E of edges
v1 v3
e1
e4 v5
e2
e5
v2 v4
e3
Single layer feedforward network
Recurrent network
Characteristics of neural networks
Mapping capabilities - they can map i/p pattern to
associated o/p patterns
NN learn by examples – NNs can be trained with known
examples of a problem before they are tested for their
inference capability on unknown instances of problem
Capability to generalize – can predict new outcomes from
past trends
Robust systems and fault tolerant – they can recall full
pattern from the incomplete , partial or noisy pattern
Can process information in parallel at high speed and in a
distributed manner.
Training and learning methods
Training methods
Supervised
Unsupervised
Reinforced
Learning Rules
1. Hebbian learning
Based on corrective weight adjustments
W = ∑ Xi Yi T
3. Competitive learning
“Winner – takes – all strategy”
4. Stochastic learning
Weights are adjusted in probabilistic manner
Classification Learning algorithm
stochastic
Error correction hebbian competitive
(gradient descent)
back propagation
least mean square
Taxonomy of NN architectures
ADALINE ,ART
AM ,BAM
Boltzmann Machine ,BSB
CCN,Cauchy Machine
Hopfield Network, LVQ
MADALINE, MLFF
Neocogniton,Perceptron
RBF, RNN, SOFM
LEARNING METHOD
TYPE GRADIENT HEBBIAN COMPETITIVE STOCHASTI
OF DESCENT C
ARCHIT SS FF ADALINE AM LVQ -
ECTURE Hopfield Hopfield SOFM
Percepron
ML CCN Neocognito - -
FF MLFF n
RBF
RN RNN BAM ART Boltzmann
BSB machine
Hopfield Cauchy
machine
Early neural network architectures
Rosenblatt’s perceptron
Computational model of the retina of the eye.
Sensory unit association unit response unit
Basic learning algorithm for training
perceptron
1. Wij (k+1) = Wij k
2. Wij (k+1) = Wij k -ɳxi
3.
Wij (k+1) = Wij k +ɳxi
Limitation
Can handle problems with linear separable solution
space
Linear sereparability
X - OR problem
The problem for ANN is to identify the input as odd parity and the even
parity
Why Perceptron is unable to find solution for
nonlinear problem
Perceprton learning rule
Simplified Rule
Theorem
ADALINE Network
MADALINE network
Applications of ANNs
1. Pattern Recongnition(PR)/ Image processing
2. Optimisation / constraint satisfaction
3. Forecasting & Risk management
4. Control systems