skip to main content
Language:
Search Limited to: Search Limited to: Resource type Show Results with: Show Results with: Search type Index

Deep learning based assistive technology on audio visual speech recognition for hearing impaired

International journal of cognitive computing in engineering, 2022-06, Vol.3, p.24-30 [Peer Reviewed Journal]

2022 ;ISSN: 2666-3074 ;EISSN: 2666-3074 ;DOI: 10.1016/j.ijcce.2022.01.003

Full text available

Citations Cited by
  • Title:
    Deep learning based assistive technology on audio visual speech recognition for hearing impaired
  • Author: Kumar, L Ashok ; Renuka, D Karthika ; Rose, S Lovelyn ; Shunmuga priya, M C ; Wartana, I Made
  • Subjects: CNN ; Deep learning ; Lip reading ; RNN-GRU ; Speech recognition ; Visual speech recognition
  • Is Part Of: International journal of cognitive computing in engineering, 2022-06, Vol.3, p.24-30
  • Description: •Assistive technology on speech to text conversion for hearing impaired students.•Performance of the system boost using multi modal features.•Lip reading architecture capture visual speech recognition.•Deep learning based framework is incorporated to lower word error rate. Assistive technology would be an immense benefit for hearing impaired people by using Audio Visual Speech Recognition (AVSR). Around 466 million people worldwide suffer from hearing loss. Hearing impaired student rely on lip reading for understanding the speech. Lack of trained sign language facilitators and high cost of assistive devices are some of the major challenges faced by hearing impaired students. In this work, we have identified a visual speech recognition technique using cutting edge deep learning models. Moreover, the existing VSR techniques are erroneous. Hence to address the gaps identified, we propose a novel technique by fusion the results from audio and visual speech. This study proposes a new deep learning based audio visual speech recognition model for efficient lip reading. In this paper, an effort has been made to improve the performance of the system significantly by achieving a lowered word error rate of about 6.59% for ASR system and accuracy of about 95% using lip reading model.
  • Publisher: Elsevier B.V
  • Language: English
  • Identifier: ISSN: 2666-3074
    EISSN: 2666-3074
    DOI: 10.1016/j.ijcce.2022.01.003
  • Source: DOAJ Directory of Open Access Journals
Back to results list
INSPIRE LIBRARY - TON DUC THANG UNIVERSITY (84-028) 37 755 057 Feedback
19 Nguyen Huu Tho St. Dist.7, HCM thuvien@tdtu.edu.vn

Copyright © 2017 INSPIRE LIBRARY - TON DUC THANG UNIVERSITY

Searching Remote Databases, Please Wait