Language:

Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study

The lancet oncology, 2019-07, Vol.20 (7), p.938-947 [Peer Reviewed Journal]

2019 Elsevier Ltd ;Copyright © 2019 Elsevier Ltd. All rights reserved. ;2019. Elsevier Ltd ;Distributed under a Creative Commons Attribution 4.0 International License ;ISSN: 1470-2045 ;ISSN: 1474-5488 ;EISSN: 1474-5488 ;DOI: 10.1016/S1470-2045(19)30333-X ;PMID: 31201137

Full text available

Citations Cited by

Actions
1. Add to My Research
2. Remove from My Research
3. E-mail
4. Print
5. Permalink
6. Citation
7. EasyBib
8. EndNote
9. RefWorks
10. Delicious
11. Export RIS
12. Export BibTeX

Title:
Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study
Author: Tschandl, Philipp ; Codella, Noel ; Akay, Bengü Nisa ; Argenziano, Giuseppe ; Braun, Ralph P ; Cabo, Horacio ; Gutman, David ; Halpern, Allan ; Helba, Brian ; Hofmann-Wellenhof, Rainer ; Lallas, Aimilios ; Lapins, Jan ; Longo, Caterina ; Malvehy, Josep ; Marchetti, Michael A ; Marghoob, Ashfaq ; Menzies, Scott ; Oakley, Amanda ; Paoli, John ; Puig, Susana ; Rinner, Christoph ; Rosendahl, Cliff ; Scope, Alon ; Sinz, Christoph ; Soyer, H Peter ; Thomas, Luc ; Zalaudek, Iris ; Kittler, Harald
Subjects: Accuracy ; Adult ; Algorithms ; Artificial intelligence ; Automation ; Benign ; Bias ; Bowen's disease ; Cancer ; Dermatologi och venereologi ; Dermatology ; Dermatology and Venereal Diseases ; Dermoscopy ; Experts ; FDA approval ; Female ; Humans ; Internet ; Keratosis ; Learning algorithms ; Lichen planus ; Life Sciences ; Machine Learning ; Male ; Medicin och hälsovetenskap ; Melanoma ; Neural networks ; Nevus ; Oncology ; Pigmentation Disorders - pathology ; Reproducibility of Results ; Retrospective Studies ; Skin cancer ; Skin diseases ; Skin Neoplasms - pathology ; Stockholders ; Studies
Is Part Of: The lancet oncology, 2019-07, Vol.20 (7), p.938-947
Description: Whether machine-learning algorithms can diagnose all pigmented skin lesions as accurately as human experts is unclear. The aim of this study was to compare the diagnostic accuracy of state-of-the-art machine-learning algorithms with human readers for all clinically relevant types of benign and malignant pigmented skin lesions. For this open, web-based, international, diagnostic study, human readers were asked to diagnose dermatoscopic images selected randomly in 30-image batches from a test set of 1511 images. The diagnoses from human readers were compared with those of 139 algorithms created by 77 machine-learning labs, who participated in the International Skin Imaging Collaboration 2018 challenge and received a training set of 10 015 images in advance. The ground truth of each lesion fell into one of seven predefined disease categories: intraepithelial carcinoma including actinic keratoses and Bowen's disease; basal cell carcinoma; benign keratinocytic lesions including solar lentigo, seborrheic keratosis and lichen planus-like keratosis; dermatofibroma; melanoma; melanocytic nevus; and vascular lesions. The two main outcomes were the differences in the number of correct specific diagnoses per batch between all human readers and the top three algorithms, and between human experts and the top three algorithms. Between Aug 4, 2018, and Sept 30, 2018, 511 human readers from 63 countries had at least one attempt in the reader study. 283 (55·4%) of 511 human readers were board-certified dermatologists, 118 (23·1%) were dermatology residents, and 83 (16·2%) were general practitioners. When comparing all human readers with all machine-learning algorithms, the algorithms achieved a mean of 2·01 (95% CI 1·97 to 2·04; p<0·0001) more correct diagnoses (17·91 [SD 3·42] vs 19·92 [4·27]). 27 human experts with more than 10 years of experience achieved a mean of 18·78 (SD 3·15) correct answers, compared with 25·43 (1·95) correct answers for the top three machine algorithms (mean difference 6·65, 95% CI 6·06–7·25; p<0·0001). The difference between human experts and the top three algorithms was significantly lower for images in the test set that were collected from sources not included in the training set (human underperformance of 11·4%, 95% CI 9·9–12·9 vs 3·6%, 0·8–6·3; p<0·0001). State-of-the-art machine-learning classifiers outperformed human experts in the diagnosis of pigmented skin lesions and should have a more important role in clinical practice. However, a possible limitation of these algorithms is their decreased performance for out-of-distribution images, which should be addressed in future research. None.
Publisher: England: Elsevier Ltd
Language: English
Identifier: ISSN: 1470-2045
ISSN: 1474-5488
EISSN: 1474-5488
DOI: 10.1016/S1470-2045(19)30333-X
PMID: 31201137
Source: MEDLINE
SWEPUB Freely available online
ProQuest Central

Back to results list


INSPIRE LIBRARY - TON DUC THANG UNIVERSITY	(84-028) 37 755 057	Feedback
19 Nguyen Huu Tho St. Dist.7, HCM	thuvien@tdtu.edu.vn	Feedback

Comparison of the accuracy of human readers versus machine-learning algorithms for pigmented skin lesion classification: an open, web-based, international, diagnostic study

2019 Elsevier Ltd ;Copyright © 2019 Elsevier Ltd. All rights reserved. ;2019. Elsevier Ltd ;Distributed under a Creative Commons Attribution 4.0 International License ;ISSN: 1470-2045 ;ISSN: 1474-5488 ;EISSN: 1474-5488 ;DOI: 10.1016/S1470-2045(19)30333-X ;PMID: 31201137

Searching Remote Databases, Please Wait