DSpace Repository

Multimodal Machine Learning for Sign Language Prediction

Show simple item record

dc.contributor.author KHALAFAOUI, Yassèr
dc.contributor.author GROZAVU, Nistor
dc.contributor.author BASARAB, Matei
dc.contributor.author ROGOVSCHI, Nicoleta
dc.date.accessioned 2023-11-14T11:51:23Z
dc.date.available 2023-11-14T11:51:23Z
dc.date.issued 2023
dc.identifier.citation KHALAFAOUI, Yassèr, GROZAVU, Nistor, BASARAB, Matei et al. Multimodal Machine Learning for Sign Language Prediction. In: 6th International Conference on Nanotechnologies and Biomedical Engineering: proc. of ICNBME-2023, 20–23, 2023, Chisinau, vol. 2: Biomedical Engineering and New Technologies for Diagnosis, Treatment, and Rehabilitation, 2023, p. 236-242. ISBN 978-3-031-42781-7. e-ISBN 978-3-031-42782-4. en_US
dc.identifier.isbn 978-3-031-42781-7
dc.identifier.isbn 978-3-031-42782-4
dc.identifier.uri https://doi.org/10.1007/978-3-031-42782-4_26
dc.identifier.uri http://repository.utm.md/handle/5014/24796
dc.description Acces full text - https://doi.org/10.1007/978-3-031-42782-4_26 en_US
dc.description.abstract Numerous applications, including translation tools, interpreting services, video remote interpreting, human-computer interaction, online hand tracking of human communication in desktop settings, real-time multi-person recognition systems, games, virtual reality settings, robot controls, and natural language communications, benefit from sign language recognition advantages. Multimodal data contains information from different sources such as video, sensors, electrocardiograms (ECGs), while emotions refer to the non-verbal cues that accompany language use, such as facial expressions and body posture. Integrating these additional sources of information helps to better understand the user’s intent, which improves the performance of the sign language recognition model. To build such a model, a set of multimodal data and emotions must be collected. This data set should be differentiated and cover different individual/isolated signs, emotions and body gestures. The model is designed to integrate multimodal data and emotions, which would involve combining different machine and deep learning algorithms adapted to different types of data. In addition, the model will need to be trained to recognize the different emotions that accompany sign language. Once the model is trained, it can be tested on the test dataset to assess its performance and also plan for a test on real data (with signing people). In this paper we propose a study to use the multi-modal machine learning for sign recognition language. en_US
dc.language.iso en en_US
dc.publisher Springer Nature Switzerland en_US
dc.rights Attribution-NonCommercial-NoDerivs 3.0 United States *
dc.rights.uri http://creativecommons.org/licenses/by-nc-nd/3.0/us/ *
dc.subject sign language en_US
dc.subject multimodal data en_US
dc.subject multimodal machine learning (MML) en_US
dc.title Multimodal Machine Learning for Sign Language Prediction en_US
dc.type Article en_US


Files in this item

The following license files are associated with this item:

This item appears in the following Collection(s)

  • 2023
    6th International Conference on Nanotechnologies and Biomedical Engineering, September 20–23, 2023, Chisinau, Moldova

Show simple item record

Attribution-NonCommercial-NoDerivs 3.0 United States Except where otherwise noted, this item's license is described as Attribution-NonCommercial-NoDerivs 3.0 United States

Search DSpace


Advanced Search

Browse

My Account