Softmax Function

The softmax function (σ) is a mathematical function that normalizes a vector of real numbers into a probability distribution. It applies the exponential function to each element in the vector and then divides the results by the sum of the exponentiated values.

We can define softmax using the following equation:

$${\sigma (x)}_i=\frac{e^{x_i}}{\sum _{j=1}^K{e}^{x_j}}$$

where:

• x is an input vector of real numbers
• σ(x) is an output vector whose sum is 1
• K is the number of elements in x and σ(x) and indicates the number of possible outcomes
• x_i is the ith element in x
• σ(x)_i is the ith element in σ(x) and represents the probability of the ith outcome

---

This page references the following sources:

• DeepLearning.AI Deep Learning Specialization (course 2, week 3)
• Google Machine Learning Glossary
• Wikipedia's "Softmax function" article