research-article

Position and Orientation Agnostic Gesture Recognition Using WiFi

Authors:

Aditya Virmani,

Muhammad ShahzadAuthors Info & Claims

MobiSys '17: Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services

Pages 252 - 264

https://doi.org/10.1145/3081333.3081340

Published: 16 June 2017 Publication History

Abstract

WiFi based gesture recognition systems have recently proliferated due to the ubiquitous availability of WiFi in almost every modern building. The key limitation of existing WiFi based gesture recognition systems is that they require the user to be in the same configuration (i.e., at the same position and in same orientation) when performing gestures at runtime as when providing training samples, which significantly restricts their practical usability. In this paper, we propose a WiFi based gesture recognition system, namely WiAG, which recognizes the gestures of the user irrespective of his/her configuration. The key idea behind WiAG is that it first requests the user to provide training samples for all gestures in only one configuration and then automatically generates virtual samples for all gestures in all possible configurations by applying our novel translation function on the training samples. Next, for each configuration, it generates a classification model using virtual samples corresponding to that configuration. To recognize gestures of a user at runtime, as soon as the user performs a gesture, WiAG first automatically estimates the configuration of the user and then evaluates the gesture against the classification model corresponding to that estimated configuration. Our evaluation results show that when user's configuration is not the same at runtime as at the time of providing training samples, WiAG significantly improves the gesture recognition accuracy from just 51.4% to 91.4%.

References

[1]

Allure Smart Thermostat. https://www.ure-energy.com/.

[2]

Honeywell Lyric Thermostat. https://wifithermostat.com/Products/Lyric/.

[3]

Honeywell Smart Thermostat. https://wifithermostat.com/Products/WiFiSmartThermostat/.

[4]

Insteon LED Bulbs. https://www.insteon.com/led-bulbs/.

[5]

LG Smart Appliances. https://www.lg.com/us/discover/smartthinq/thinq.

[6]

Nest Thermostat. https://nest.com/thermostat/meet-nest-thermostat/.

[7]

Philips Hue. https://www2.meethue.com/en-us/.

[8]

Smart Home. https://www.smarthome.com/.

[9]

Smoothing Spline Matlab. https://www.mathworks.com/help/curvefit/smoothing-splines.html.

[10]

Whirlpool Smart Appliances. https://www.whirlpool.com/smart-appliances/.

[11]

WiFi Plug. https://www.wifiplug.co.uk/.

[12]

Abdelnasser, H., Youssef, M., and Harras, K. A. WiGest: A Ubiquitous WiFi-based Gesture Recognition System, May 2015.

[13]

Adib, F., Hsu, C. Y., Mao, H., Katabi, D., and Durand, F. Capturing the Human Figure Through a Wall. ACM Trans. Graph. 34, 6 (Oct. 2015).

Digital Library

[14]

Adib, F., Kabelac, Z., and Katabi, D. Multi-person Localization via RF Body Reflections. In Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation (Berkeley, CA, USA, 2015), NSDI'15, USENIX Association, pp. 279--292.

Digital Library

[15]

Adib, F., Kabelac, Z., Katabi, D., and Miller, R. C. 3D Tracking via Body Radio Reflections. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (Berkeley, CA, USA, 2014), NSDI'14, USENIX Association, pp. 317--329.

Digital Library

[16]

Adib, F., and Katabi, D. See Through Walls with WiFi! SIGCOMM Comput. Commun. Rev. 43, 4 (Aug. 2013), 75--86.

Digital Library

[17]

Adib, F., Mao, H., Kabelac, Z., Katabi, D., and Miller, R. C. Smart Homes That Monitor Breathing and Heart Rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (New York, NY, USA, 2015), CHI '15, ACM, pp. 837--846.

Digital Library

[18]

Aggarwal, J. K., and Michael, S. R. Human activity analysis: A review. ACM Computing Surveys 43, 3 (2011).

Digital Library

[19]

Ali, K., Liu, A. X., Wang, W., and Shahzad, M. Keystroke Recognition Using WiFi Signals. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (New York, NY, USA, 2015), MobiCom '15, ACM, pp. 90--102.

Digital Library

[20]

Burrus, C. S., Gopinath, R. A., and Guo, H. Introduction to wavelets and wavelet transforms: a primer.

[21]

Cheung, G. K., Kanade, T., Bouguet, J.-Y., and Holler, M. A real time system for robust 3d voxel reconstruction of human motions. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on (2000), vol. 2, IEEE, pp. 714--720.

[22]

Ertin, E., Stohs, N., Kumar, S., Raij, A., al'Absi, M., and Shah, S. AutoSense: unobtrusively wearable sensor suite for inferring the onset, causality, and consequences of stress in the field. In Proceedings of ACM Sensys (2011).

Digital Library

[23]

Halperin, D., Hu, W., Sheth, A., and Wetherall, D. Tool release: Gathering 802.11n traces with channel state information. ACM SIGCOMM CCR 41, 1 (Jan. 2011), 53.

Digital Library

[24]

Han, C., Wu, K., Wang, Y., and Ni, L. M. Wifall: Device-free fall detection by wireless networks. In Proceedings of IEEE INFOCOM (2014), pp. 271--279.

[25]

Herda, L., Fua, P., Pl\"ankers, R., Boulic, R., and Thalmann, D. Skeleton-based motion capture for robust reconstruction of human motion. In Computer Animation 2000. Proceedings (2000), IEEE, pp. 77--83.

Digital Library

[26]

Huang, D., Nandakumar, R., and Gollakota, S. Feasibility and limits of wi-fi imaging. In Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems (2014), ACM, pp. 266--279.

Digital Library

[27]

Kellogg, B., Talla, V., and Gollakota, S. Bringing gesture recognition to all devices. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (Berkeley, CA, USA, 2014), NSDI'14, USENIX Association, pp. 303--316.

Digital Library

[28]

Lang, M., Guo, H., Odegard, J. E., Burrus, C. S., and Wells, R. O. Noise reduction using an undecimated discrete wavelet transform. IEEE Signal Processing Letters 3, 1 (1996), 10--12.

[29]

Lemmey, T., Vonog, S., and Surin, N. System architecture and methods for distributed multi-sensor gesture processing, Aug. 15 2011. US Patent App. 13/210,370.

[30]

Lyonnet, B., Ioana, C., and Amin, M. G. Human gait classification using microdoppler time-frequency signal representations. In 2010 IEEE Radar Conference (2010), IEEE, pp. 915--919.

[31]

Moeslund, T. B., Hilton, A., and Krüger, V. A survey of advances in vision-based human motion capture and analysis. Computer vision and image understanding 104, 2 (2006), 90--126.

Digital Library

[32]

Ocak, H. Automatic detection of epileptic seizures in eeg using discrete wavelet transform and approximate entropy. Expert Systems with Applications 36, 2 (2009), 2027--2036.

Digital Library

[33]

Oprisescu, S., Rasche, C., and Su, B. Automatic static hand gesture recognition using tof cameras. In Signal Processing Conference (EUSIPCO), 2012 Proceedings of the 20th European (2012), IEEE, pp. 2748--2751.

[34]

Park, T., Lee, J., Hwang, I., Yoo, C., Nachman, L., and Song, J. E-gesture: a collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices. In Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems (2011), ACM, pp. 260--273.

Digital Library

[35]

Pu, Q., Gupta, S., Gollakota, S., and Patel, S. Whole-home gesture recognition using wireless signals. In Proceedings of the 19th Annual International Conference on Mobile Computing & Networking (New York, NY, USA, 2013), MobiCom '13, ACM, pp. 27--38.

Digital Library

[36]

Seco, F., Jiménez, A. R., and Zampella, F. Joint estimation of indoor position and orientation from rf signal strength measurements. In Indoor Positioning and Indoor Navigation (IPIN), 2013 International Conference on (2013), IEEE, pp. 1--8.

[37]

Sen, S., Lee, J., Kim, K.-H., and Congdon, P. Avoiding multipath to revive inbuilding wifi localization. In Proceeding of the 11th annual international conference on Mobile systems, applications, and services (2013), ACM, pp. 249--262.

Digital Library

[38]

Shotton, J., Sharp, T., Kipman, A., Fitzgibbon, A., Finocchio, M., Blake, A., Cook, M., and Moore, R. Real-time human pose recognition in parts from single depth images. Communications of the ACM 56, 1 (2013), 116--124.

Digital Library

[39]

Singh, G., Nelson, A., Robucci, R., Patel, C., and Banerjee, N. Inviz: Low-power personalized gesture recognition using wearable textile capacitive sensor arrays. In Pervasive Computing and Communications (PerCom), 2015 IEEE International Conference on (2015), IEEE, pp. 198--206.

[40]

Sun, L., Sen, S., Koutsonikolas, D., and Kim, K. H. WiDraw: Enabling Hands-free Drawing in the Air on Commodity WiFi Devices. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (New York, NY, USA, 2015), MobiCom '15, ACM, pp. 77--89.

Digital Library

[41]

Tzanetakis, G., Essl, G., and Cook, P. Audio analysis using the discrete wavelet transform. In Proc. Conf. in Acoustics and Music Theory Applications (2001).

[42]

Van Dorp, P., and Groen, F. Feature-based human motion parameter estimation with radar. IET Radar, Sonar & Navigation 2, 2 (2008), 135--145.

[43]

Wang, G., Zou, Y., Zhou, Z., Wu, K., and Ni, L. M. We can hear you with wi-fi! In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (New York, NY, USA, 2014), MobiCom '14, ACM, pp. 593--604.

Digital Library

[44]

Wang, W., Liu, A. X., and Shahzad, M. Gait recognition using wifi signals. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing (2016), ACM, pp. 363--373.

Digital Library

[45]

Wang, W., Liu, A. X., Shahzad, M., Ling, K., and Lu, S. Understanding and modeling of wifi signal based human activity recognition. In Proceedings of the 21st Annual International Conference on Mobile Computing and Networking (2015), ACM, pp. 65--76.

Digital Library

[46]

Wang, Y., Liu, J., Chen, Y., Gruteser, M., Yang, J., and Liu, H. E-eyes: Device-free Location-oriented Activity Identification Using Fine-grained WiFi Signatures. In Proceedings of the 20th Annual International Conference on Mobile Computing and Networking (New York, NY, USA, 2014), MobiCom '14, ACM, pp. 617--628.

Digital Library

[47]

Xi, W., Zhao, J., Li, X.-Y., Zhao, K., Tang, S., Liu, X., and Jiang, Z. Electronic frog eye: Counting crowd using WiFi. In Proceedings of IEEE INFOCOM (2014).

[48]

Yang, Z., Zhou, Z., and Liu, Y. From rssi to csi: Indoor localization via channel response. ACM Computing Surveys (CSUR) 46, 2 (2013), 25.

Digital Library

[49]

Yatani, K., and Truong, K. N. Bodyscope: a wearable acoustic sensor for activity recognition. In Proceedings of ACM UbiComp (2012), pp. 341--350.

Digital Library

Cited By

Wang SWang XJiang WMiao CCao QWang HSun KXue HSu L(2024)Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic SignalsACM Transactions on Sensor Networks10.1145/367712220:5(1-32)Online publication date: 26-Aug-2024
https://dl.acm.org/doi/10.1145/3677122
Li XWang HChen ZJiang ZLuo JOkoshi TKo JLiKamWa R(2024)UWB-Fi: Pushing Wi-Fi towards Ultra-wideband for Fine-Granularity SensingProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661889(42-55)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661889
Liu YYu AWang LGuo BLi YYi EZhang D(2024)UniFiProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314297:4(1-29)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631429
Show More Cited By

Index Terms

Position and Orientation Agnostic Gesture Recognition Using WiFi
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction techniques
      1. Gestural input
2. Networks
  1. Network types
    1. Mobile networks
    2. Wireless access networks

Recommendations

Multi-User Gesture Recognition Using WiFi
MobiSys '18: Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services

WiFi based gesture recognition has received significant attention over the past few years. However, the key limitation of prior WiFi based gesture recognition systems is that they cannot recognize the gestures of multiple users performing them ...
Augmenting User Identification with WiFi Based Gesture Recognition

Over the last few years, researchers have proposed several WiFi based gesture recognition systems that can recognize predefined gestures performed by users at runtime. As most environments are inhabited by multiple users, the true potential of WiFi based ...
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)

Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '17: Proceedings of the 15th Annual International Conference on Mobile Systems, Applications, and Services

June 2017

520 pages

ISBN:9781450349284

DOI:10.1145/3081333

General Chairs:
Tanzeem Choudhury
Cornell University, USA
,
Steve Ko
University at Buffalo, USA
,
Program Chairs:
Andrew Campbell
Dartmouth College, USA
,
Deepak Ganesan
University of Massachusetts, USA

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 June 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MobiSys'17

Sponsor:

SIGMOBILE

MobiSys'17: The 15th Annual International Conference on Mobile Systems, Applications, and Services

June 19 - 23, 2017

New York, Niagara Falls, USA

Acceptance Rates

MobiSys '17 Paper Acceptance Rate 34 of 188 submissions, 18%;

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

206
Total Citations
View Citations
1,826
Total Downloads

Downloads (Last 12 months)190
Downloads (Last 6 weeks)12

Reflects downloads up to 24 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang SWang XJiang WMiao CCao QWang HSun KXue HSu L(2024)Towards Smartphone-based 3D Hand Pose Reconstruction Using Acoustic SignalsACM Transactions on Sensor Networks10.1145/367712220:5(1-32)Online publication date: 26-Aug-2024
https://dl.acm.org/doi/10.1145/3677122
Li XWang HChen ZJiang ZLuo JOkoshi TKo JLiKamWa R(2024)UWB-Fi: Pushing Wi-Fi towards Ultra-wideband for Fine-Granularity SensingProceedings of the 22nd Annual International Conference on Mobile Systems, Applications and Services10.1145/3643832.3661889(42-55)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3643832.3661889
Liu YYu AWang LGuo BLi YYi EZhang D(2024)UniFiProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36314297:4(1-29)Online publication date: 12-Jan-2024
https://dl.acm.org/doi/10.1145/3631429
Liao ZLuo ZHuang QZhang LWu FZhang QChen G(2024)Gesture Recognition Using Visible Light on Mobile DevicesIEEE/ACM Transactions on Networking10.1109/TNET.2024.336999632:4(2920-2935)Online publication date: Aug-2024
https://doi.org/10.1109/TNET.2024.3369996
Chen YYu JKong LZhu YTang F(2024)Sensing Human Gait for Environment-Independent user Authentication using Commodity RFID DevicesIEEE Transactions on Mobile Computing10.1109/TMC.2023.3318753(1-13)Online publication date: 2024
https://doi.org/10.1109/TMC.2023.3318753
Zhang YCheng AChen BWang YJia L(2024)A Location-independent Human Activity Recognition Method Based on CSI: System, Architecture, ImplementationIEEE Transactions on Mobile Computing10.1109/TMC.2023.3296987(1-14)Online publication date: 2024
https://doi.org/10.1109/TMC.2023.3296987
Chen XZou YLi CXiao W(2024)A Deep Learning Based Lightweight Human Activity Recognition System Using Reconstructed WiFi CSIIEEE Transactions on Human-Machine Systems10.1109/THMS.2023.334869454:1(68-78)Online publication date: Feb-2024
https://doi.org/10.1109/THMS.2023.3348694
Chen ZHu CZheng TCao HYang YChu YJiang HLuo J(2024)ISAC-Fi: Enabling Full-Fledged Monostatic Sensing Over Wi-Fi CommunicationIEEE Journal of Selected Areas in Sensors10.1109/JSAS.2024.34432481(139-153)Online publication date: 2024
https://doi.org/10.1109/JSAS.2024.3443248
Smida BWichman RKolodziej KSuraweera HRiihonen TSabharwal A(2024)In-Band Full-Duplex: The Physical LayerProceedings of the IEEE10.1109/JPROC.2024.3366768112:5(433-462)Online publication date: May-2024
https://doi.org/10.1109/JPROC.2024.3366768
Wu JWang JDai TGao QPan M(2024)Trajectory Features-Based Robust Device-Free Gesture Recognition Using mmWave SignalsIEEE Internet of Things Journal10.1109/JIOT.2024.336043411:10(18123-18135)Online publication date: 15-May-2024
https://doi.org/10.1109/JIOT.2024.3360434
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents