Artificial neural networks are based on
biological neural systems. The
elementary building block of which is the neuron. A neuron in a real biological system may have over 10,000 inputs and send its
output to a 100 or more other neurons. Neurons are wired up in a
3-dimensional pattern. No artificial network to date has achieved that
level of complexity.
The single cell neuron consists of the cell body (soma), the dendrites, and the axon. The dendrites receive signals from the axons of other neurons and the axon takes impulses away from the soma. A single neuron may receive input from as many as 8x104 neighboring neurons.
The job of the neuron is to basically integrate all the inputs it receives and decide whether to fire or not. Below a threshold level of activity, the neuron produces no output.
Artificial Neural Networks work on the same principles.
The consist of a collection of
artificial neurons that are weighted to produce a specific
output. Normally there will be several layers The diagram below
illustrates a simple neural network. The lines represent weighted
connections (i.e., a scaling factor) between neurons. In this case there
are 4 inputs and 2 outputs.
The original idea of artificial neural networks stemmed from psychological work in animal learning. D.O. Hebb in 1949 theorized that learning occurred in brains when synapses and neurons fire repeatedly which in a way 'trains' the network to recognized the same stimulus when it occurs again (1).
Earlier, in 1943, McCulloch and Pitts presented a neural computing model called the MCP neuron in a paper, "A logical calculus of the ideas immanent in nervous activity", (3) In the paper they tried to explain how the brain could produce complex patterns from a connection of basic neurons. They gave a simple model of such a neuron that consisted of a collection of inputs and a single output. The inputs were either excitatory (+1) or inhibatory (-1). The function for the neuron weights and sums the results to produce either a 1 or a -1. McCulloch and Pitts demonstrated how a collection of these neurons could compute any logical function.
In this paper McCulloch and Pitts tried to understand how the brain could produce highly complex patterns by using many basic cells that are connected together. These basic brain cells are called neurons, and McCulloch and Pitts gave a highly simplified model of a neuron in their paper.
The table shows how such a neuron would compute a logical AND function, meaning both inputs would have to be positive for the output to be positive.
Input signals (x's) - These signals can come from the outside world or from other neurons in the system.
Set of Weights (w's) - The weights describe the relative importance of each input.
Activation Level (sum of x*w) - The activation level is the sum of the scaled inputs
Threshold Function (f) - This function decides whether the activation level is high enough to produce a positive output for the neuron.
An example artificial neuron is shown below:
Certain types of problems that best suited to a solution using neural networks:
Classification: Deciding which category an input belongs to - such as handwriting analysis
Pattern Recognition: Finding values in noisy data - useful for medical imaging
Prediction: Deciding an outcome based on past events - Useful for predicting stock performance
Optimization: Finding the optimal solution to a problem with certain constraints - Such as the traveling salesman problem