Deep learning and collaborative training for reducing communication barriers with deaf people
Authors: 
Manuel Vázquez-Enríquez, Pablo Losada-Rodríguez, Mateo González-Cid, José L. Alba-CastroAuthors Info & Claims
Pages 289 - 292
Abstract
Building bridges to facilitate communication between deaf and hearing people is a pending issue that can no longer be postponed in a technologically advanced society. In this work, we propose a multiplatform service that allows gradually building a collaborative platform to bring the interests of deaf and hearing people closer together. The proposal consists of a client-server architecture with two main parts: a multiplatform interface for video capture and playback, and a sign language recognizer system composed of a skeleton data extractor and a Graph Convolutional Neural Network. The result is a proof-of-concept of the first thematic dictionary accessible through sign language.
References
[1]
Danielle Bragg, Oscar Koller, Mary Bellard, Larwan Berke, Patrick Boudreault, Annelies Braffort, Naomi Caselli, Matt Huenerfauth, Hernisa Kacorri, Tessa Verhoef, Christian Vogler, and Meredith Ringel Morris. 2019. Sign Language Recognition, Generation, and Translation: An Interdisciplinary Perspective. In The 21st International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS 2019, Pittsburgh, PA, USA, October 28-30, 2019, Jeffrey P. Bigham, Shiri Azenkot, and Shaun K. Kane (Eds.). ACM, 16--31.
[2]
Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 1 (2021), 172--186. https://doi.org/10.1109/TPAMI.2019.2929257
[3]
Helen Cooper, Brian Holt, and Richard Bowden. 2011. Sign Language Recognition. Springer London, London, 539--562.
[4]
Laura Docío-Fernández, José Luis Alba-Castro, Soledad Torres-Guijarro, Eduardo Rodríguez-Banga, Manuel Rey-Area, Ania Pérez-Pérez, Sonia Rico-Alonso, and Carmen García-Mateo. 2020. LSE_UVIGO: A Multi-source Database for Spanish Sign Language Recognition. In Proceedings of the LREC2020: 9th Workshop on the Representation and Processing of Sign Languages: Sign Language Resources in the Service of the Language Community, Technological Challenges and Application Perspectives. 45--52.
[5]
Ziyu Liu, Hongwen Zhang, Zhenghao Chen, Zhiyong Wang, and Wanli Ouyang. 2020. Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognition. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2020, Seattle, WA, USA, June 13-19, 2020. IEEE, 140--149. https://doi.org/10.1109/CVPR42600.2020.00022
[6]
Razieh Rastgoo, Kourosh Kiani, and Sergio Escalera. 2021. Sign Language Recognition: A Deep Survey. Expert Syst. Appl. 164 (2021), 113794. https://doi.org/10.1016/j.eswa.2020.113794
[7]
Ozge Mercanoglu Sincan, Julio C. S. Jacques Junior, Sergio Escalera, and Hacer Yalim Keles. 2021. ChaLearn LAP Large Scale Signer Independent Isolated Sign Language Recognition Challenge: Design, Results and Future Research. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
[8]
Ozge Mercanoglu Sincan and Hacer Yalim Keles. 2020. AUTSL: A Large Scale Multi-Modal Turkish Sign Language Dataset and Baseline Methods. IEEE Access 8 (2020), 181340--181355. https://doi.org/10.1109/ACCESS.2020.3028072
[9]
Manuel Vázquez-Enríquez, Jose L. Alba-Castro, Laura Docío-Fernández, and Eduardo Rodríguez-Banga. 2021. Isolated Sign Language Recognition with Multi-Scale Spatial-Temporal Graph Convolutional Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops.
[10]
Sijie Yan, Yuanjun Xiong, and Dahua Lin. 2018. Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition. CoRR abs/1801.07455 (2018). arXiv:1801.07455 http://arxiv.org/abs/1801.07455
Index Terms
- Deep learning and collaborative training for reducing communication barriers with deaf people
Recommendations
Communication system between deaf-blind people and non-disabled people using body-braille and infrared communication
CCNC'10: Proceedings of the 7th IEEE conference on Consumer communications and networking conferenceThere are several barriers to communicate between deaf-blind people and non-disabled people. The communication method of deaf-blind people is totally different from the language of non-disabled people, so they can't communicate with each other directly. ...
ProSign Everywhere - Addressing Communication Empowerment Goals for Deaf People
GoodTechs '19: Proceedings of the 5th EAI International Conference on Smart Objects and Technologies for Social GoodSign language is recognized as one of the most popular means of communication for deaf people. Unfortunately, the knowledge of sign language by hearing people is not widespread and this may pose unpleasant barriers between deaf and their surrounding ...
Facilitating the communication with deaf people: Building a largest Saudi sign language dataset
AbstractRecently, several countries have been trying hard to facilitate the integration of disabled people into their societies by ensuring equal opportunities through ease of access to social services, daily human necessities, and the labor market. ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
September 2021
345 pages
ISBN:9781450384780
DOI:10.1145/3462203
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 09 September 2021
Check for updates
Badges
Best Short Paper
Author Tags
Qualifiers
- Short-paper
- Research
- Refereed limited
Conference
GoodIT '21
Sponsor:
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 131Total Downloads
- Downloads (Last 12 months)9
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Nov 2024
Other Metrics
Citations
View Options
Get Access
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in