skip to main content
Language:
Search Limited to: Search Limited to: Resource type Show Results with: Show Results with: Search type Index

Decoding lip language using triboelectric sensors with deep learning

Nature communications, 2022-03, Vol.13 (1), p.1401-1401, Article 1401 [Peer Reviewed Journal]

2022. The Author(s). ;The Author(s) 2022. This work is published under http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. ;The Author(s) 2022 ;ISSN: 2041-1723 ;EISSN: 2041-1723 ;DOI: 10.1038/s41467-022-29083-0 ;PMID: 35301313

Full text available

Citations Cited by
  • Title:
    Decoding lip language using triboelectric sensors with deep learning
  • Author: Lu, Yijia ; Tian, Han ; Cheng, Jia ; Zhu, Fei ; Liu, Bin ; Wei, Shanshan ; Ji, Linhong ; Wang, Zhong Lin
  • Subjects: Communication ; Deep Learning ; Directional control ; Electric contacts ; Electrical properties ; Flexible components ; Humans ; Language ; Language translation ; Lip ; Neural networks ; Prototypes ; Recurrent neural networks ; Sensors ; Speech ; Translation systems ; Voice ; Voice communication
  • Is Part Of: Nature communications, 2022-03, Vol.13 (1), p.1401-1401, Article 1401
  • Description: Lip language is an effective method of voice-off communication in daily life for people with vocal cord lesions and laryngeal and lingual injuries without occupying the hands. Collection and interpretation of lip language is challenging. Here, we propose the concept of a novel lip-language decoding system with self-powered, low-cost, contact and flexible triboelectric sensors and a well-trained dilated recurrent neural network model based on prototype learning. The structural principle and electrical properties of the flexible sensors are measured and analysed. Lip motions for selected vowels, words, phrases, silent speech and voice speech are collected and compared. The prototype learning model reaches a test accuracy of 94.5% in training 20 classes with 100 samples each. The applications, such as identity recognition to unlock a gate, directional control of a toy car and lip-motion to speech conversion, work well and demonstrate great feasibility and potential. Our work presents a promising way to help people lacking a voice live a convenient life with barrier-free communication and boost their happiness, enriches the diversity of lip-language translation systems and will have potential value in many applications.
  • Publisher: England: Nature Publishing Group
  • Language: English
  • Identifier: ISSN: 2041-1723
    EISSN: 2041-1723
    DOI: 10.1038/s41467-022-29083-0
    PMID: 35301313
  • Source: MEDLINE
    PubMed Central
    Coronavirus Research Database
    ProQuest Central
    DOAJ Directory of Open Access Journals
Back to results list
INSPIRE LIBRARY - TON DUC THANG UNIVERSITY (84-028) 37 755 057 Feedback
19 Nguyen Huu Tho St. Dist.7, HCM thuvien@tdtu.edu.vn

Copyright © 2017 INSPIRE LIBRARY - TON DUC THANG UNIVERSITY

Searching Remote Databases, Please Wait