In the context of artificial neural networks, the rectifier, or ReLU for rectified linear unit, is an activation function defined as the non-negative part of its input, equal to the input itself when the input is positive and to zero otherwise, a shape sometimes called a ramp function. This behavior is analogous to half-wave rectification in electrical engineering, and ReLU has become one of the most popular activation functions used in artificial neural networks, with applications in computer vision, speech recognition through deep neural networks, and computational neuroscience. This description is adapted from Wikipedia contributors under CC BY-SA 4.0; changes were made. https://creativecommons.org/licenses/by-sa/4.0/
Facts
Connections
Attributed To
Source Rectified Linear Unit (Wikipedia)
Is Kind Of Object
Sources
1. Rectified Linear Unit (Wikipedia)
Lead section
In the context of artificial neural networks, the rectifier or ReLU (rectified linear unit) activation function is an activation function defined as the non-negative part of its argument, i.e., the ramp function: ReLU ( x ) = x + = max ( 0 , x ) = x + | x | 2 = { x if x > 0 , 0 x ≤ 0 {\displaystyle \operatorname {ReLU} (x)=x^{+}=\max(0,x)={\frac {x+|x|}{2}}={\begin{cases}x&{\text{if }}x>0,\\0&x\leq 0\end{cases}}} where x {\displaystyle x} is the input to a neuron.
Lead
The ReLU was first used by Alston Householder in 1941
Attributed To: Alston Householder, Article text
The ReLU was first used by Alston Householder in 1941 as a mathematical abstraction of biological neural networks.
View the SourceReader Challenges (0)
No disputes yet. Spotted an error or a better source? Open the first one.
Sign in to dispute this or suggest a correction.