Introduction
Neural-networks is one of those words that is getting fashionable
in the new era of technology. Most people have heard of them, but
very few actually know what they are. This essay is designed to introduce
you to all the basics of neural networks — their function, generic
structure, terminology, types and uses.
The term 'neural network' is in fact a biological term, and what we
refer to as neural networks should really be called Artificial Neural
Networks (ANNs). I will use the two terms interchangeable throughout
the essay, though. A real neural network is a collection of neurons,
the tiny cells our brains are comprised of. A network can consist
of a few to a few billion neurons connected in an array of different
methods. ANNs attempt to model these biological structures both in
architecture and operation. There is a small problem: we don't quite
know how biological NNs work! Therefore, the architecture of neural
networks changes greatly from type to type. What we do know is the
structure of the basic neuron.
The Neuron
Although it has been proposed that there are anything between 50 and
500 different types of neurons in our brain, they are mostly just
specialized cells based upon the basic neuron. The basic neuron consists
of synapses, the soma, the axon and dendrites. Synapses are connections
between neurons - they are not physical connections, but miniscule
gaps that allow electric signals to jump across from neuron to neuron.
These electrical signals are then passed across to the soma which
performs some operation and sends out its own electrical signal to
the axon. The axon then distributes this signal to dendrites. Dendrites
carry the signals out to the various synapses, and the cycle repeats.
Just as there is a basic biological neuron, there is basic artificial
neuron. Each neuron has a certain number of inputs, each of which
have a weight assigned to them. The weights simply are an indication
of how 'important' the incoming signal for that input is. The net
value of the neuron is then calculated - the net is simply the weighted
sum, the sum of all the inputs multiplied by their specific weight.
Each neuron has its own unique threshold value, and it the net is
greater than the threshold, the neuron fires (or outputs a 1), otherwise
it stays quiet (outputs a 0). The output is then fed into all the
neurons it is connected to.
Learning
As this talk about weights and thresholds leads to an obvious question.
How are all these values set? There are nearly as many training methods
as there are network types (a lot!), but some of the more popular
ones include back-propagation, the delta rule and Kohonen learning.
As architectures vary, so do the learning rules, but most rules can
be categorized into two areas - supervised and unsupervised. Supervised
learning rules require a 'teacher' to tell them what the desired output
is given an input. The learning rules then adjusts all the necessary
weights (this can be very complicated in networks), and the whole
process starts again until the data can be correctly analyzed by the
network. Supervised learning rules include back-propagation and the
delta rule. Unsupervised rules do not require teachers because they
produce their own output which is then further evaluated.
Architecture
This area of neural networking is the "fuzziest" in terms
of a definite set of rules to abide by. There are many types of networks
- ranging from simple boolean networks (Perceptrons), to complex self-organizing
networks (Kohonen networks), to networks modelling thermodynamic properties
(Boltzmann machines)! There is, though, a standard network architecture.
The network consists of several "layers" of neurons, an
input layer, hidden layers, and output layers. Input layers take the
input and distribute it to the hidden layers (so-called hidden because
the user cannot see the inputs or outputs for those layers). These
hidden layers do all the necessary computation and output the results
to the output layer, which (surprisingly) outputs the data to the
user. Now, to avoid confusion, I will not explore the architecture
topic further. To read more about different neural nets, see the Generation5
essays.
Even after discussing
neurons, learning and architecture we are still unsure about what
exactly neural networks do!
The
Function of ANNs
Neural networks are designed to work with patterns - they can be classified
as pattern classifiers or pattern associators. The networks can takes
a vector (series of numbers), then classify the vector. For example,
my ONR program takes an image of a number and outputs the number itself.
Or my PDA32 program takes a coordinate and can classify it as either
class A or class B (classes are determined by learning from examples
provided). More practical uses can be seen in military radars where
radar returns can be classified as enemy vehicles or trees (read more
in the Applications in the Military essay).
Pattern associators takes one vector and output another. For example,
my HIR program takes a 'dirty' image and outputs the image that represents
the one closest to the one it has learnt. Again, at a more practical
level, associative networks can be used in more complex applications
such as signature/face/fingerprint recognition.
The Ups and Downs of Neural Networks
There are many good points to neural-networks and advances in this
field will increase their popularity. There are excellent as pattern
classifiers/recognizors - and can be used where traditional techniques
do not work. Neural-networks can handle exceptions and abnormal input
data, very important for systems that handle a wide range of data
(radar and sonar systems, for example). Many neural networks are biologically
plausible, which means they may provide clues as to how the brain
works as they progress. Advances in neuroscience will also help advance
neural networks to the point where they will be able to classify objects
with the accuracy of a human at the speed of a computer! The future
is bright, the present however...
Yes, there are quite a few down points to neural networks. Most of
them, though, lie with our lack of hardware. The power of neural-networks
lie in their ability to process information in a parallel fashion
(that is, process multiple chunks of data simultaneously). Unfortunately,
machines today are serial - they only execute one instruction at a
time. Therefore, modelling parallel processing on serial machines
can be a very time-consuming process. As with everything in this day
and age, time is of the essence, which often leaves neural networks
out of the list of viable solutions to a problem.
Other problems
with neural networks are the lack of defining rules to help construct
a network given a problem - there are many factors to take into consideration:
the learning algorithm, architecture, number of neurons per layer,
number of layers, data representation and much more. Again, with time
being so important, companies cannot afford to invest to time to develop
a network to solve the problem efficiently. This will all change as
neural networking advances.
Conclusion
Hopefully, by now you have a good understanding of the basics of neural
networks. Generation5 has recently had a lot of information added
on neural networking, both in essays and in programs. We have examples
of Hopfield networks, perceptrons (2 example programs), and even some
case-studies on back-propagation. Please browse through the site to
find out more!
Compiled
from different essays at Generation5 website
