Introduction to Perceptron

What is a perceptron?

Perceptron is a binary linear classifier used in supervised learning to classify the given input data.

Visualize a perceptron

In the case of two input data points, the perceptron model makes a line that separates the two classes.

Boundary

The boundary line that separates the two classes are:

$$w_1x_1 + w_2x_2 + b = 0$$

Here:

• $x_1$ and $x_2$ are the inputs
• $w_1$ and $w_2$ are the weights
• $b$ is the bias

Prediction

The predicted value on applying the step function is given by:

$\hat{y}$=$\Bigg\{$ $\begin{matrix} 1 & {if\; w_1x_1 +w_2x_2 + b >= 0}\\ \\ 0 & otherwise \end{matrix}$

In the case of three input data points, the perceptron model would separate the two classes using a plane:

Boundary

The plane separating the data points would be:

$$w_1x_1 + w_2x_2 + w_3x_3 + b = 0$$

Here,

• $x_1$, $x_2$, $x_3$ are the inputs
• $w_1$, $w_2$, $w_3$ are the weights
• $b$ is the bias

Prediction

The predicted value on applying the step function is given by:

$\hat{y}$ = $\Bigg\{$ $\begin{matrix} 1 & {if\; w_1x_1 +w_2x_2 + w_3x_3 + b >= 0}\\ \\ 0 & otherwise \end{matrix}$

In the case of n-dimensional data points, $n-1$ hyperdimensional plane is the separating boundary.

Boundary

The hyperdimensional plane would be:

$w_1x_1 + w_2x_2 + w_3x_3 + ... + w_nx_n + b = 0$

Here,

• $x_1$, $x_2$, $x_3$, … $x_n$ are the inputs
• $w_1$, $w_2$, $w_3$, … $w_n$ are the weights
• $b$ is the bias

Prediction

The predicted value on applying the step function is given by:

$\hat{y}$ = $\Bigg\{$ $\begin{matrix} 1 & {if\; w_1x_1 +w_2x_2 + w_3x_3 +...+ w_nx_n +b >= 0}\\ \\ 0 & otherwise \end{matrix}$

📝 Note: The process of calculating the perceptron output and then applying the step function is called “feedforward operation”. As the perceptron is a single-layered neural network, a perceptron is a single-layer feedforward operation. The step function is called an “activation function” because it activates the perceptron output.

Takeaway

• Perceptron can be used to solve a two-class classification problem.

• The boundary that separates the two classes is given by:
  $w_1x_1 + w_2x_2 + w_3x_3 + ... + w_nx_n + b = 0$

  This is known as a single-layer feedforward operation.

• The prediction is given by:

  $\hat{y}$ = $\Bigg\{$ $\begin{matrix} 1 & {if\; w_1x_1 +w_2x_2 + w_3x_3 +...+ w_nx_n +b >= 0}\\ \\ 0 & otherwise \end{matrix}$

• The step function is the activation function.

---