Introduction

Many popular (and powerful) computational techniques are inspired by biology, physics, or chemistry. When the brain learns, the neurons (brain cells) adjust their synaptic connections to other neurons. A classic computational model that tries to capture the way in which neurons operate is called perceptron. It is a neat little mathematical representation where an input (represented by a vector) and the current "state of learning" of the perceptron (represented by another vector known as its weights) are combined with linear algebra (an inner product) to produce value that can then be fed into an activation function (often a threshold) to produce a yes/no output from it. One can "make the neuron learn" (train it) by providing inputs for which the desired outputs are known ahead of time, then compare the generated output with the desired one, and adjust the weights little by little until the perceptron systematically gets it right. Of course, the model being so simple, it cannot learn just anything — only linearly separable tw0-class problems — but if one combines many perceptrons (to build a brain instead of just a single cell), the possibilities become vast.

Learning outcomes

This module will help you do the following:

• Understand the inner workings of a perceptron
• Train a perceptron by adjusting the weights upon errors
• Compute common performance measures for a perceptron
• Combine multiple perceptrons to create a layer of neurons
• Represent base 10 (decimal) class labels in base 2 (binary)

Warm-up

Our attempts to comprehend artificial brains benefit from a refresher on how our own brains operate. The below video might help.

Warm-up assessment

Training a computational perceptron (neuron) is in effect trial and error, with feedback on what the error was. Based on having watched the video (please refrain from looking things up again for the benefit of exercising original thought), elaborate a bit on your own experience and understanding of how many rounds of trial-error do babies (of any species you have had a chance to observe) require at minimum to acquire advanced skills such as communication or mobility: dozens, hundreds, thousands, millions? Does the amount seem to depend on the complexity of the skill? List at least three different skills along with your gut feeling on how much practice it takes to develop to an acceptable level.

Concepts

After this module, you should be familiar with the following concepts:

• Classification
• Linearly separable
• Vector
• Inner product
• Activation function
• Error
• Confusion matrix

Remember that you can always look concepts up in the glossary. Should anything be missing or insufficient, please report it.