Sigmoid
Also: logistic function, sigmoid function
The logistic function σ(x) = 1 / (1 + e^(−x)) maps large negative inputs toward 0 and large positive inputs toward 1. It is the output of logistic regression and binary classifiers, but saturates at both ends, so it is rarely used in deep hidden layers.
σ(x) = 1 / (1 + e^(−x))
Example
σ(0) = 0.5, σ(2) ≈ 0.881 and σ(−2) ≈ 0.119. Its derivative σ(x)(1 − σ(x)) peaks at only 0.25, which is why stacked sigmoids shrink gradients.