Abstract: We demonstrate reconfigurable tanh- and ReLU-like nonlinear activation functions for incoherent neuromorphic photonics using a balanced photodiode assembled with a programmable electronic ...

But, the vanishing gradient problem persists even in the case of Tanh. Rectified Linear Unit or ReLU is now one of the most widely used activation functions. The function operates on max(0,x), which ...