LYAPUNOV - LIKE APPROACH FOR STUDYING GLOBAL CONVERGENCE OF NEURAL NETWORKS WITH DISCONTINUOUS OR NON-LIPSCHITZ ACTIVATIONS

MATHEMATICS

The paper considers a class of additive neural networks where the neuron activations are modelled by discontinuous functions or by continuous non-lipschitz functions. Some tools are developed which enable us to apply a Lyapunov-like approach to differential equations with discontinuous right-hand side modelling the neural network dynamics. The tools include a chain rule for computing the time derivative along the neural network solutions of a non differentiable Lyapunov function, and a comparison principal for this time derivative, which yields conditions for exponential convergence or convergence in finite time. By means of Lyapunov-like approach, a general result is proved on global exponential convergence toward a unique equilibrium point of the neural network solutions, Moreover, new results on global convergence in finite time are established, which are applicable to neuron activations with jump discontinuities, or neuron activations modelled by means of continuous (non-Lipschitz) Holder functions.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.