🚀 go-pugleaf

In article <17615@glacier.STANFORD.EDU> jbn@glacier.UUCP (John B. Nagle) writes:
[JN]Recognize that the transfer function in a neural network threshold unit
[JN]doesn't really have to be a sigmoid function.  It just has to look roughly
[JN]like one.  The behavior of the net is not all that sensitive to the 
[JN]exact form of that function.  It has to be continuous and monotonic, 
[JN]reasonably smooth, and rise rapidly in the middle of the working range.
[JN]The trigonometric form of the transfer function is really just a notational
[JN]convenience.
[JN]
[JN]   It would be a worthwhile exercise to come up with some other forms
[JN]of transfer function with roughly the same graph, but better matched to
[JN]hardware implementation.  How do real neurons do it?
[JN] 
[JN] 					John Nagle


Try this one :   f(x) = x / (1 + |x|)


It is continuous and differentiable:  

f'(x)  =  1 / (1 + |x|) ** 2   =   ( 1 - |f|) ** 2 .

- Paul Munro