Activation Function

An activation function is a mathematical function applied to a weighted sum of inputs to a node in a neural network to calculate the node’s output. Nonlinear activation functions enable networks to learn complex relationships.

Common activation functions include:

• ReLU
• sigmoid
• softmax
• tanh

---

This page references the following sources:

• DeepLearning.AI Deep Learning Specialization (course 1, week 3)
• Google Machine Learning Glossary
• Wikipedia's "Activation function" article

Further reading:

• Deep Learning (ch. 6)
• Papers with Code's "Activation Functions" category