The sigmoid function is one of the most commonly used neural activations functions. Also, it is used in logistics regression. The advantage of using sigmoid function is that instead of giving discrete values i.e. 0 and 1 it gives a continuous value between 0 and 1. This makes it useful in predicting probabilities.

$$\sigma = \left(\frac{1}{1+e^{-x}}\right)$$

We can rearrange it by using power notation

$$\sigma = \left(1 + e^{-x}\right)^{-1}$$

Differentiating both side with respect to x.

$$\frac{d\sigma}{dx} = \frac{d}{dx}\left(1 + e^{-x}\right)^{-1}$$

$$\frac{d\sigma}{dx} = (-1) * (1+e^{-x})^{-2} * \frac{d}{dx}(1 + e^{-x})$$

$$\frac{d\sigma}{dx} = (-1) *(1+e^{-x})^{-2} *(-1)*(e^{-x})$$

$$\frac{d\sigma}{dx} = (1+e^{-x})^{-2} * (e^{-x})$$

$$\frac{d\sigma}{dx} = \frac{e^{-x}}{(1+e^{-x})^2}$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \frac{e^{-x}}{1 + e^{-x}}$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \frac{1 +e^{-x} -1 }{1 + e^{-x}}$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \left(\frac{1 +e^{-x}}{1 + e^{-x}} – \frac{1}{1 + e ^{-x}}\right)$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \left( 1 – \frac{1}{1 + e ^{-x}}\right)$$

$$\sigma^\mathbf| = \sigma\left(x\right)\left(1-\sigma\left(x\right)\right)$$

Shares