Activation functions: Why are they important and how to use them?

Source

Activation Functions:

What should we look for in an activation function?

Types of Activation function:

Source
Source
Source
Source
Source
Source
Source

Swish Vs Relu:

Source: Swish paper

When to use which activation function?

Resources:

I’m changing.