Language:

Knowledge Distillation: A Survey

International journal of computer vision, 2021-06, Vol.129 (6), p.1789-1819 [Peer Reviewed Journal]

Crown 2021 ;COPYRIGHT 2021 Springer ;Crown 2021. ;ISSN: 0920-5691 ;EISSN: 1573-1405 ;DOI: 10.1007/s11263-021-01453-z

Full text available

Citations Cited by

Actions
1. Add to My Research
2. Remove from My Research
3. E-mail
4. Print
5. Permalink
6. Citation
7. EasyBib
8. EndNote
9. RefWorks
10. Delicious
11. Export RIS
12. Export BibTeX

Title:
Knowledge Distillation: A Survey
Author: Gou, Jianping ; Yu, Baosheng ; Maybank, Stephen J. ; Tao, Dacheng
Subjects: Algorithms ; Artificial Intelligence ; Artificial neural networks ; Computational linguistics ; Computer Imaging ; Computer Science ; Computer vision ; Deep learning ; Distillation ; Electronic devices ; Embedded systems ; Image Processing and Computer Vision ; Knowledge ; Language processing ; Machine learning ; Machine vision ; Natural language interfaces ; Neural networks ; Pattern Recognition ; Pattern Recognition and Graphics ; Students ; Success ; Survey Papers ; Surveys ; Teachers ; Vision
Is Part Of: International journal of computer vision, 2021-06, Vol.129 (6), p.1789-1819
Description: In recent years, deep neural networks have been successful in both industry and academia, especially for computer vision tasks. The great success of deep learning is mainly due to its scalability to encode large-scale data and to maneuver billions of model parameters. However, it is a challenge to deploy these cumbersome deep models on devices with limited resources, e.g., mobile phones and embedded devices, not only because of the high computational complexity but also the large storage requirements. To this end, a variety of model compression and acceleration techniques have been developed. As a representative type of model compression and acceleration, knowledge distillation effectively learns a small student model from a large teacher model. It has received rapid increasing attention from the community. This paper provides a comprehensive survey of knowledge distillation from the perspectives of knowledge categories, training schemes, teacher–student architecture, distillation algorithms, performance comparison and applications. Furthermore, challenges in knowledge distillation are briefly reviewed and comments on future research are discussed and forwarded.
Publisher: New York: Springer US
Language: English
Identifier: ISSN: 0920-5691
EISSN: 1573-1405
DOI: 10.1007/s11263-021-01453-z
Source: ProQuest Central

Back to results list


INSPIRE LIBRARY - TON DUC THANG UNIVERSITY	(84-028) 37 755 057	Feedback
19 Nguyen Huu Tho St. Dist.7, HCM	thuvien@tdtu.edu.vn	Feedback

Knowledge Distillation: A Survey

Crown 2021 ;COPYRIGHT 2021 Springer ;Crown 2021. ;ISSN: 0920-5691 ;EISSN: 1573-1405 ;DOI: 10.1007/s11263-021-01453-z

Searching Remote Databases, Please Wait