Recognizing British Sign Language using deep learning: a contactless and privacy-preserving approach

Hira Hameed, Muhammad Usman, Ahsen Tahir, Kashif Ahmad, Amir Hussain, Muhammad Ali Imran, Qammer H. Abbasi

Research output: Contribution to journalArticlepeer-review

1 Downloads (Pure)

Abstract

Sign language is utilized by deaf-mute to communicate through hand movements, body postures, and facial emotions. The motions in sign language comprise a range of distinct hand and finger articulations that are occasionally synchronized with the head, face, and body. Automatic sign language recognition (SLR) is a highly challenging area and still remains in its infancy compared with speech recognition after almost three decades of research. Current wearable and vision-based systems for SLR are intrusive and suffer from the limitations of ambient lighting and privacy concerns. To the best of our knowledge, our work proposes the first contactless British sign language (BSL) recognition system using radar and deep learning (DL) algorithms. Our proposed system extracts the 2-D spatiotemporal features from the radar data and applies the state-of-the-art DL models to classify spatiotemporal features from BSL signs to different verbs and emotions, such as Help, Drink, Eat, Happy, Hate, and Sad. We collected and annotated a large-scale benchmark BSL dataset covering 15 different types of BSL signs. Our proposed system demonstrates highest classification performance with a multiclass accuracy of up to 90.07% at a distance of 141 cm from the subject using the VGGNet model.
Original languageEnglish
Pages (from-to)1-9
Number of pages9
JournalIEEE Transactions on Computational Social Systems
DOIs
Publication statusPublished - 13 Oct 2022

Keywords

  • British sign language (BSL)
  • contactless monitoring
  • deep learning (DL)
  • micro-Doppler signatures
  • radio frequency (RF) sensing
  • Radio frequency
  • Gesture recognition
  • Radar
  • Assistive technologies
  • Data collection
  • Radar antennas
  • Sensors

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Social Sciences (miscellaneous)
  • Modelling and Simulation

Fingerprint

Dive into the research topics of 'Recognizing British Sign Language using deep learning: a contactless and privacy-preserving approach'. Together they form a unique fingerprint.

Cite this