Non-linear separable Boolean Functions

<aside> 💡 There are $2^{2^n}$boolean functions for $n$ inputs.

</aside>

Representation Power of a Network of Perceptrons

<aside> 💡 Any boolean function of n inputs can be represented exactly by a network of perceptrons containing 1 hidden layer with $2^n$ perceptrons and one output layer containing 1 perceptron.

Note: A network of $2^n + 1$ perceptrons is not necessary but sufficient

</aside>

Sigmoid Neuron

<aside> 💡 A sigmoid fires only if the weighted sum of its inputs is greater than the threshold$(-w_0)$

</aside>

Let $y = f(x)$

where, $x \in \mathbb{R^n}$ and

                $y \\in \\mathbb{R}$

• The thresholding logic used by a perceptron is very harsh. It is a characteristic of the perceptron function itself which behaves like a step function.
• For most real world applications we would expect a smoother decision function which gradually changes from 0 to 1.
• Sigmoid Function: