Derivative of Sigmoid Function

[et_pb_section fb_built=”1″ admin_label=”section” _builder_version=”3.22″ collapsed=”off”][et_pb_row admin_label=”row” _builder_version=”3.25″ background_size=”initial” background_position=”top_left” background_repeat=”repeat” collapsed=”on”][et_pb_column type=”4_4″ _builder_version=”3.25″ custom_padding=”|||” custom_padding__hover=”|||”][et_pb_text admin_label=”Text” _builder_version=”3.27.4″ background_size=”initial” background_position=”top_left” background_repeat=”repeat”]

The sigmoid function is one of the most commonly used neural activations functions. Also, it is used in logistics regression. The advantage of using sigmoid function is that instead of giving discrete values i.e. 0 and 1 it gives a continuous value between 0 and 1. This makes it useful in predicting probabilities.

In this article, we will calculate derivative of sigmoid funciton. Let’s start with sigmoid formula :-

$$\sigma = \left(\frac{1}{1+e^{-x}}\right) $$

We can rearrange it by using power notation

$$\sigma = \left(1 + e^{-x}\right)^{-1}$$

Differentiating both side with respect to x.

$$\frac{d\sigma}{dx} = \frac{d}{dx}\left(1 + e^{-x}\right)^{-1}$$

$$\frac{d\sigma}{dx} = (-1) * (1+e^{-x})^{-2} * \frac{d}{dx}(1 + e^{-x})$$

$$\frac{d\sigma}{dx} = (-1) *(1+e^{-x})^{-2} *(-1)*(e^{-x})$$

$$\frac{d\sigma}{dx} = (1+e^{-x})^{-2} * (e^{-x})$$

$$\frac{d\sigma}{dx} = \frac{e^{-x}}{(1+e^{-x})^2}$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \frac{e^{-x}}{1 + e^{-x}}$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \frac{1 +e^{-x} -1 }{1 + e^{-x}}$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \left(\frac{1 +e^{-x}}{1 + e^{-x}} – \frac{1}{1 + e ^{-x}}\right)$$

$$\frac{d\sigma}{dx} = \frac{1}{1+e^{-x}} * \left( 1 – \frac{1}{1 + e ^{-x}}\right)$$

$$\sigma^\mathbf| = \sigma\left(x\right)\left(1-\sigma\left(x\right)\right)$$


Leave a Reply

Your email address will not be published.

Pin It on Pinterest

%d bloggers like this: