skip to main content
Language:
Search Limited to: Search Limited to: Resource type Show Results with: Show Results with: Search type Index

Regularisation of neural networks by enforcing Lipschitz continuity

Machine learning, 2021-02, Vol.110 (2), p.393-416 [Peer Reviewed Journal]

The Author(s) 2020 ;The Author(s) 2020. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. ;ISSN: 0885-6125 ;EISSN: 1573-0565 ;DOI: 10.1007/s10994-020-05929-w

Full text available

Citations Cited by
  • Title:
    Regularisation of neural networks by enforcing Lipschitz continuity
  • Author: Gouk, Henry ; Frank, Eibe ; Pfahringer, Bernhard ; Cree, Michael J.
  • Subjects: Artificial Intelligence ; Computation ; Computer Science ; Control ; Machine Learning ; Mathematical models ; Mechatronics ; Natural Language Processing (NLP) ; Neural networks ; Norms ; Optimization ; Regularization ; Robotics ; Simulation and Modeling ; Training ; Upper bounds
  • Is Part Of: Machine learning, 2021-02, Vol.110 (2), p.393-416
  • Description: We investigate the effect of explicitly enforcing the Lipschitz continuity of neural networks with respect to their inputs. To this end, we provide a simple technique for computing an upper bound to the Lipschitz constant—for multiple p -norms—of a feed forward neural network composed of commonly used layer types. Our technique is then used to formulate training a neural network with a bounded Lipschitz constant as a constrained optimisation problem that can be solved using projected stochastic gradient methods. Our evaluation study shows that the performance of the resulting models exceeds that of models trained with other common regularisers. We also provide evidence that the hyperparameters are intuitive to tune, demonstrate how the choice of norm for computing the Lipschitz constant impacts the resulting model, and show that the performance gains provided by our method are particularly noticeable when only a small amount of training data is available.
  • Publisher: New York: Springer US
  • Language: English
  • Identifier: ISSN: 0885-6125
    EISSN: 1573-0565
    DOI: 10.1007/s10994-020-05929-w
  • Source: SpringerOpen
    ProQuest Central
Back to results list
INSPIRE LIBRARY - TON DUC THANG UNIVERSITY (84-028) 37 755 057 Feedback
19 Nguyen Huu Tho St. Dist.7, HCM thuvien@tdtu.edu.vn

Copyright © 2017 INSPIRE LIBRARY - TON DUC THANG UNIVERSITY

Searching Remote Databases, Please Wait