Simple mathematical model for a neuron.
Invented by Rosenblatt in the 1960s.
A perceptron is a function f: Rⁿ -> R defined by 2 functions g: Rⁿ -> R and t: R -> R per f(x) = t(g(x)). R is the set of real numbers.
g is called the summation function and t is called the activation/transfer function.
Usually g(x) is the inner product of x with a weight vector w minus a real threshold value theta; g(x) = < x,w> - threshold
t can be any monotonic function, but usually boundedness is demanded. Very often the sign function is used.

Perceptrons were believed to be very powerful in the 1960s, until Minsky and Papert proved in their book "Perceptrons" that a single perceptron (with the sign function) can't solve the XOR problem.

Note that often authors use the above example for g, the sign function for t and regard other functions for t as non-perceptrons.

A perceptron is a simple kind of neural network. A perceptron may have m weighted inputs and a bias, which are summed. This sum is passed through a hard limit transfer function, which forces the output to one if the input is greater than or equal to zero, or to zero if the input is less than zero. This hard limit function is what makes a perceptron a perceptron. More general neural networks can use many different transfer functions.

A neural network consisting of a single node can classify input vectors into one of two categories. In general, a perceptron network with n nodes can classify input vectors into n^2 categories.

A single layer perceptron network can be trained using the perceptron learning rule. More complex networks use backpropagation training.

An example perceptron might have two inputs:

              +------+
p1 --> w1 --> | ---- |
              | \    | --> hardlim(p1w1 + p2w2 + b) --> a
p2 --> w2 --> | /    |
              | ---- |
              +------+
                  |
                  b

In this example, the input vector P is (p1, p2), and the weight vector is (w1, w2). The input is weighted and summed together with the bias, and then passed to the hard limiter, so we have:

a = hardlim(PW^T + b)

If p1 and p2 are either 1 or 0, and the weight matrix is (0.5, 0.5), and the bias b is -1, then this perceptron performs the boolean AND function. Example:

hardlim((1 0)(0.5 0.5)^T + (-0.9)) = hardlim(-0.4) = 0

hardlim((0 1)(0.5 0.5)^T + (-0.9)) = hardlim(-0.4) = 0

hardlim((0 0)(0.5 0.5)^T + (-0.9)) = hardlim(-0.9) = 0

hardlim((1 1)(0.5 0.5)^T + (-0.9)) = hardlim(0.1) = 1

There are many combinations of weights and bias's that can produce the same output.

If the perceptron has two inputs, then one can graph those inputs by placing the first input on the x axis, and the second input on the y axis. A decision boundary defined by the weight vector can be made to intersect this plane, so the all of the inputs on one side of the line correspond to an output of zero and all of the inputs on the other side of the line correspond to a one.

If the weight vector is (1 0), then we can see that the output doesn't depend on the second input at all; the decision boundary is a vertical line at x=0 (assuming a bias of zero).

The bias input serves to move the decision boundary off of the origin. If we again have a weight vector of (1 0), and a bias of 1, then the decision boundary has moved to x = -1 (a = hardlim((-1 1) (1 0) + 1) = hardlim(-1 + 1) = hardlim(0) = 1). Any value for p1 greater than or equal to -1 results in an output of 1, while a p1

If the perceptron has three inputs, then the input space becomes a volume, and the decision boundary becomes a surface. Higher dimensions are hard to visualize.

---

Computational Intellegence class notes, from memory as practice.

The perceptron is a model of computation that draws inspiration from the physiology of neurons. While individually not as powerful as a Turing machine, when linked together in networks perceptrons are turing complete. They were invented by Frank Rosenblatt as the AI analog to human neurons [1] back when people thought artificial intelligence was about replicating human intelligence in machine form. They're still useful today in image processing — GIMP, Photoshop, and most other digital image editors implement them in some of their filtering algorithms.

The simplest perceptron consists of a single input channel and a trigger that fires when a certain threshold has been reached. A somewhat sad example of a single-channel perceptron is a speaker system: it takes an electronic signal from a CD player or whatever and amplifies it so that it can be heard from three blocks away. The output of this audial perceptron is not the music, but what happens if the amplifier pushes the volume beyond the limits of the speakers: they blow up, sending you the "signal" that it's time to stop bothering your neighbors so much.

Mathematically, this sort of perceptron fires whenever the amplified input is greater than the threshold. Let's call w the amount of amplification, I the signal on the input channel and t the threshold of the perceptron. Then the perceptron fires whenever wI ≥ t. This seems rather trivial, but from here it's possible to generalize the perceptron to multiple input channels.

When modeling a neuron or a sensory organ, it doesn't make much sense to say that there's only one input channel. The eye sees a great deal of raw data — best estimate is around 10,000,000 bits/sec [2]. It's not too much of a stretch to hypothesize that each rod and cone on the retina represents a separate input channel.* A perceptron here might draw upon the inputs of multiple channels before deciding whether or not to fire.

The simplest way to make a binary decision (to fire, or not to fire) based on multiple inputs is by democracy: assuming the inputs all have similar strength distributions**, average the inputs together, and then fire if the average is greater than threshold. To model more complex behavior, we might weigh each input channel differently when considering the average, in the same way that we value the presence of a dot more than the presence of a crossbar in deciding whether or not a given letter is a lowercase j.

The most general perceptron of this kind has several input channels (In), each with a corresponding weight or amplification factor (wn). If we treat the two as vectors, the perceptron fires whenever w ⋅ I ≥ t. This sort of perceptron has been useful in some machine vision problems: mostly edge finding and other geometry problems like that. More sophisticated approaches — networks of perceptrons — tend to be required for anything more complicated than that, like face or handwriting recognition.

It also has some applications to logic, namely a branch called threshold logic that studies how powerful the perceptron model is in doing logical operations when compared to other computational models. For instance, the NOT-gate can be modeled as a perceptron with one input channel (weight -1) and a threshold of zero. (We're assuming that the inputs are bipolar, i.e., either one or zero. Fuzzy logic can be done by accepting values inbetween.) The AND-gate is a perceptron with two input channels, both given half weight, with a threshold of one. And the OR-gate is a perceptron with two input channels of normal weight, with a threshold of one.

Knowing this, we know that perceptron networks are just as powerful as propositional logic. But individual perceptrons are much weaker: for instance, they cannot model the XOR-gate (a result known as the XOR Problem, proved by Marvin Minsky).

Finally, here is some Scheme code I hacked together to create individual perceptrons for the purpose of modeling logic gates.

(define (.* x y)
  (if (or (null? x) (null? y))
      0
      (+ (* (car x) (car y)) (.* (cdr x) (cdr y)))))

(define (percept weights thresh)
  (lambda (inputs)
    (>= (.* weights inputs) thresh)))

(define (not-percept in)
  ((percept '(-1) 0) in))
(define (and-percept in)
  ((percept '(0.5 0.5) 1) in))
(define (or-percept in)
  ((percept '(1 1) 1) in)))

*This isn't really true, because brightness and color aren't independent of one another. But it's close enough.
**Or normalize each channel, which amounts to the same thing.

[1] http://en.wikipedia.org/wiki/Perceptron (25 Feb 2008).
[2] http://www.uphs.upenn.edu/news/News_Releases/jul06/retinput.htm