skip to main content
research-article
Open access

Detecting State Changes of Indoor Everyday Objects using Wi-Fi Channel State Information

Published: 11 September 2017 Publication History

Abstract

Detecting the events of indoor everyday objects such as door or window open/close events has been actively studied to implement such applications as intrusion detection, adaptive HVAC control, and monitoring an independently living elderly person. This study proposes a method for detecting the events and states of indoor everyday objects such as doors and windows without using distributed sensors attached to the objects. In this study, we achieve practical and unobtrusive event detection using a commodity Wi-Fi access point and a computer equipped with a commodity Wi-Fi module. Specifically, we detect the events using Wi-Fi channel state information (CSI), which describes how a signal propagates from a transmitter to a receiver, and is affected by such events. To handle CSI data that consists of the mixed effects of multiple indoor objects in an environment of interest, we employ independent component analysis to separate the events caused by the objects. The decomposed data are then fed into our event classifier based on convolutional and recurrent neural networks to automatically extract features from CSI data, as it is difficult to intuitively design features to be extracted from the CSI data. Moreover, we correct the neural network estimates by incorporating knowledge about the state transitions of an object using hidden Markov models. For example, because the “open” event of a door occurs only when the door is in a “closed” state. We correct impossible state transitions estimated by the neural network based on this knowledge.

Supplementary Material

ohara (ohara.zip)
Supplemental movie, appendix, image and software files for, Detecting State Changes of Indoor Everyday Objects using Wi-Fi Channel State Information

References

[1]
Heba Abdel-Nasser, Reham Samir, Ibrahim Sabek, and Moustafa Youssef. 2013. MonoPHY: Mono-stream-based device-free WLAN localization via physical layer information. In IEEE Wireless Communications and Networking Conference (WCNC 2013). 4546--4551.
[2]
Heba Abdelnasser, Khaled A Harras, and Moustafa Youssef. 2015. UbiBreathe: A ubiquitous non-invasive WiFi-based breathing estimator. In the 16th ACM International Symposium on Mobile Ad Hoc Networking and Computing. 277--286.
[3]
Fadel Adib, Zach Kabelac, Dina Katabi, and Robert C Miller. 2014. 3D tracking via body radio reflections. In the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI 14). 317--329.
[4]
Kamran Ali, Alex Xiao Liu, Wei Wang, and Muhammad Shahzad. 2015. Keystroke recognition using WiFi signals. In the 21st Annual International Conference on Mobile Computing and Networking (MobiCom 2015). 90--102.
[5]
Chris Beckmann, Sunny Consolvo, and Anthony LaMarca. 2004. Some assembly required: Supporting end-user sensor installation in domestic ubiquitous computing environments. In International Conference on Ubiquitous Computing (UbiComp 2004). 107--124.
[6]
Yoshua Bengio, Patrice Simard, and Paolo Frasconi. 1994. Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks 5, 2 (1994), 157--166.
[7]
Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5--32.
[8]
Bradford Campbell and Prabal Dutta. 2014. An energy-harvesting sensor architecture and toolkit for building monitoring and event detection. In the 1st ACM Conference on Embedded Systems for Energy-Efficient Buildings. 100--109.
[9]
Olivier Chapelle, Bernhard Schölkopf, and Alexander Zien. Semi-supervised learning. MIT press Cambridge, MA.
[10]
Yogita Chapre, Aleksandar Ignjatovic, Aruna Seneviratne, and Sanjay Jha. 2015. CSI-MIMO: An efficient Wi-Fi fingerprinting using Channel State Information with MIMO. Pervasive and Mobile Computing 23 (2015), 89--103.
[11]
Ricardo Chavarriaga, Hesam Sagha, Alberto Calatroni, Sundara Tejaswi Digumarti, Gerhard Tröster, José del R Millán, and Daniel Roggen. 2013. The Opportunity challenge: A benchmark database for on-body sensor-based activity recognition. Pattern Recognition Letters 34, 15 (2013), 2033--2042.
[12]
Dan Ciregan, Ueli Meier, and Jürgen Schmidhuber. 2012. Multi-column deep neural networks for image classification. In Computer Vision and Pattern Recognition (CVPR 2012). 3642--3649.
[13]
David Cohn, Les Atlas, and Richard Ladner. 1994. Improving generalization with active learning. Machine Learning 15, 2 (1994), 201--221.
[14]
Pierre Comon. 1994. Independent component analysis, a new concept? Signal Processing 36, 3 (1994), 287--314.
[15]
Felix A Gers, Jürgen Schmidhuber, and Fred Cummins. 2000. Learning to forget: Continual prediction with LSTM. Neural Computation 12, 10 (2000), 2451--2471.
[16]
Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In the 38th International Conference on Acoustics, Speech and Signal Processing (ICASSP 2013). 6645--6649.
[17]
Yanying Gu, Anthony Lo, and Ignas Niemegeers. 2009. A survey of indoor positioning systems for wireless personal networks. IEEE Communications Surveys 8 Tutorials 11, 1 (2009), 13--32.
[18]
Daniel Halperin, Wenjun Hu, Anmol Sheth, and David Wetherall. 2011. Tool release: Gathering 802.11 n traces with channel state information. ACM SIGCOMM Computer Communication Review 41, 1 (2011), 53--53.
[19]
Chunmei Han, Kaishun Wu, Yuxi Wang, and Lionel M Ni. 2014. WiFall: Device-free fall detection by wireless networks. In IEEE Conference on Computer Communications (INFOCOM 2014). 271--279.
[20]
Aapo Hyvärinen. 1999. Fast and robust fixed-point algorithms for independent component analysis. IEEE Transactions on Neural Networks 10, 3 (1999), 626--634.
[21]
Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015).
[22]
Matthew J. Johnson and Alan S. Willsky. 2013. Bayesian nonparametric hidden semi-Markov models. Journal of Machine Learning Research 14, 1 (2013), 673--701.
[23]
Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014).
[24]
Palanivel A Kodeswaran, Ravi Kokku, Sayandeep Sen, and Mudhakar Srivatsa. 2016. Idea: A system for efficient failure management in smart IoT environments. In the 14th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys 2016). 43--56.
[25]
Anthony LaMarca, Yatin Chawathe, Sunny Consolvo, Jeffrey Hightower, Ian Smith, James Scott, Timothy Sohn, James Howard, Jeff Hughes, Fred Potter, and others. 2005. Place lab: Device positioning using radio beacons in the wild. In International Conference on Pervasive Computing (Pervasive 2005). 116--133.
[26]
Hui Liu, Houshang Darabi, Pat Banerjee, and Jing Liu. 2007. Survey of wireless indoor positioning techniques and systems. IEEE Transactions on Systems, Man, and Cybernetics, Part C: Applications and Reviews 37, 6 (2007), 1067--1080.
[27]
Xuefeng Liu, Jiannong Cao, Shaojie Tang, and Jiaqi Wen. 2014. Wi-Sleep: Contactless sleep monitoring via WiFi signals. In IEEE Real-Time Systems Symposium (RTSS 2014). 346--355.
[28]
Kazuya Ohara, Takuya Maekawa, Yasue Kishino, Yoshinari Shirai, and Futoshi Naya. 2015. Transferring positioning model for device-free passive indoor localization. In the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015). 885--896.
[29]
Francisco Javier Ordóñez and Daniel Roggen. 2016. Deep convolutional and LSTM recurrent neural networks for multimodal wearable activity recognition. Sensors 16, 1 (2016), 115.
[30]
Shwetak N Patel, Matthew S Reynolds, and Gregory D Abowd. 2008. Detecting human movement by differential air pressure sensing in HVAC system ductwork: An exploration in infrastructure mediated sensing. In International Conference on Pervasive Computing (Pervasive 2008). 1--18.
[31]
Anindya S Paul, Eric A Wan, Fatema Adenwala, Erich Schafermeyer, Nick Preiser, Jeffrey Kaye, and Peter G Jacobs. 2014. MobileRF: A robust device-free tracking system based on a hybrid neural network HMM classifier. In the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2014). 159--170.
[32]
Matthai Philipose, Kenneth P Fishkin, Mike Perkowitz, Donald J Patterson, Dieter Fox, Henry Kautz, and Dirk Hähnel. 2004. Inferring activities from interactions with objects. IEEE Pervasive Computing 3, 4 (2004), 50--57.
[33]
Qifan Pu, Sidhant Gupta, Shyamnath Gollakota, and Shwetak Patel. 2013. Whole-home gesture recognition using wireless signals. In the 19th annual International Conference on Mobile Computing and Networking (MobiCom 2013). 27--38.
[34]
Lawrence R Rabiner. 1989. A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE 77, 2 (1989), 257--286.
[35]
Moustafa Seifeldin, Ahmed Saeed, Ahmed E Kosba, Amr El-Keyi, and Moustafa Youssef. 2012. Nuzzer: A large-scale device-free passive localization system for wireless environments. IEEE Transactions on Mobile Computing 12, 7 (2012), 1321--1334.
[36]
Souvik Sen, Božidar Radunovic, Romit Roy Choudhury, and Tom Minka. 2012. You are facing the Mona Lisa: Spot localization using PHY layer information. In the 10th International Conference on Mobile Systems, Applications, and Services. 183--196.
[37]
Shuyu Shi, Stephan Sigg, and Yusheng Ji. 2012. Passive detection of situations from ambient FM-radio signals. In the 2012 ACM Conference on Ubiquitous Computing (UbiComp 2012). 1049--1053.
[38]
Stephan Sigg, Ulf Blanke, and Gerhard Tröster. 2014. The telepathic phone: Frictionless activity recognition from WiFi-RSSI. In 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom 2014). 148--155.
[39]
Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research 15, 1 (2014), 1929--1958.
[40]
Emmanuel Munguia Tapia, Stephen S Intille, and Kent Larson. 2004. Activity recognition in the home using simple and ubiquitous sensors. In International Conference on Pervasive Computing (Pervasive 2004). 158--175.
[41]
Tim Van Kasteren, Athanasios Noulas, Gwenn Englebienne, and Ben Kröse. 2008. Accurate activity recognition in a home setting. In the 10th International Conference on Ubiquitous Computing (Ubicomp 2008). 1--9.
[42]
Wei Wang, Alex X Liu, Muhammad Shahzad, Kang Ling, and Sanglu Lu. 2015. Understanding and modeling of WiFi signal based human activity recognition. In the 21st annual International Conference on Mobile Computing and Networking (MobiCom 2015). 65--76.
[43]
Xuyu Wang, Lingjun Gao, and Shiwen Mao. 2015. PhaseFi: Phase fingerprinting for indoor localization with a deep learning approach. In IEEE Global Communications Conference (GlobeCom 2015). 1--6.
[44]
Xuyu Wang, Lingjun Gao, Shiwen Mao, and Santosh Pandey. 2015. DeepFi: Deep learning for indoor fingerprinting using channel state information. In IEEE Wireless Communications and Networking Conference (WCNC 2015). 1666--1671.
[45]
Yi Wang, Xinli Jiang, Rongyu Cao, and Xiyang Wang. 2015. Robust indoor human activity recognition using wireless signals. Sensors 15, 7 (2015), 17195--17208.
[46]
Yan Wang, Jian Liu, Yingying Chen, Marco Gruteser, Jie Yang, and Hongbo Liu. 2014. E-eyes: Device-free location-oriented activity identification using fine-grained WiFi signatures. In the 20th annual International Conference on Mobile Computing and Networking (MobiCom 2014). 617--628.
[47]
Kaishun Wu, Jiang Xiao, Youwen Yi, Dihu Chen, Xiaonan Luo, and Lionel M Ni. 2013. CSI-based indoor localization. IEEE Transactions on Parallel and Distributed Systems 24, 7 (2013), 1300--1309.
[48]
Muchen Wu, Parth H Pathak, and Prasant Mohapatra. 2015. Monitoring building door events using barometer sensor in smartphones. In the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp 2015). 319--323.
[49]
Chenren Xu, Bernhard Firner, Yanyong Zhang, Richard Howard, Jun Li, and Xiaodong Lin. 2012. Improving RF-based device-free passive localization in cluttered indoor environments through probabilistic classification methods. In International Conference on Information Processing in Sensor Networks (IPSN 2012). 209--220.
[50]
Jian Bo Yang, Minh Nhut Nguyen, Phyo Phyo San, Xiao Li Li, and Shonali Krishnaswamy. 2015. Deep convolutional neural networks on multichannel time series for human activity recognition. In the 24th International Joint Conference on Artificial Intelligence (IJCAI 2015). 25--31.
[51]
Moustafa Youssef, Matthew Mah, and Ashok Agrawala. 2007. Challenges: Device-free passive localization for wireless environments. In the 13th annual ACM International Conference on Mobile Computing and Networking (MobiCom 2007). 222--229.
[52]
Ming Zeng, Le T Nguyen, Bo Yu, Ole J Mengshoel, Jiang Zhu, Pang Wu, and Joy Zhang. 2014. Convolutional neural networks for human activity recognition using mobile sensors. In 6th International Conference on Mobile Computing, Applications and Services (MobiCASE 2014). 197--205.
[53]
Daqing Zhang, Hao Wang, Yasha Wang, and Junyi Ma. 2015. Anti-fall: A non-intrusive and real-time fall detector leveraging CSI from commodity WiFi devices. In International Conference on Smart homes and health Telematics (ICOST 2015). 181--193.
[54]
Chunting Zhou, Chonglin Sun, Zhiyuan Liu, and Francis Lau. 2015. A C-LSTM neural network for text classification. arXiv preprint arXiv:1511.08630 (2015).

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 1, Issue 3
September 2017
2023 pages
EISSN:2474-9567
DOI:10.1145/3139486
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 September 2017
Accepted: 01 July 2017
Revised: 01 May 2017
Received: 01 February 2017
Published in IMWUT Volume 1, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Indoor context recognition
  2. Wi-Fi channel state information
  3. deep neural network
  4. open/close event
  5. pattern recognition

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • JST CREST JPMJCR15E2, JSPS KAKENHI

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)178
  • Downloads (Last 6 weeks)29
Reflects downloads up to 24 Sep 2024

Other Metrics

Citations

Cited By

View all
  • (2023)SIDAProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109197:3(1-24)Online publication date: 27-Sep-2023
  • (2023)AttFLProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36109177:3(1-31)Online publication date: 27-Sep-2023
  • (2023)Automated Face-To-Face Conversation Detection on a Commodity Smartwatch with Acoustic SensingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108827:3(1-29)Online publication date: 27-Sep-2023
  • (2023)SignRingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108817:3(1-29)Online publication date: 27-Sep-2023
  • (2023)Interaction HarvestingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36108807:3(1-31)Online publication date: 27-Sep-2023
  • (2023)Contactless WiFi Sensing and Monitoring for Future Healthcare - Emerging Trends, Challenges, and OpportunitiesIEEE Reviews in Biomedical Engineering10.1109/RBME.2022.315681016(171-191)Online publication date: 2023
  • (2023)Real-Time Identification of Rogue WiFi Connections in the WildIEEE Internet of Things Journal10.1109/JIOT.2022.322368210:7(6042-6058)Online publication date: 1-Apr-2023
  • (2023)Battery-Free Wireless Sensor Networks: A Comprehensive SurveyIEEE Internet of Things Journal10.1109/JIOT.2022.322238610:6(5543-5570)Online publication date: 15-Mar-2023
  • (2023)WiFi CSI Based Passive Human Activity Recognition Method Using BLSTM-CNN2023 IEEE 23rd International Conference on Communication Technology (ICCT)10.1109/ICCT59356.2023.10419414(210-215)Online publication date: 20-Oct-2023
  • (2023)Efficient Sub-Carrier Relationship Extraction for Human Activity Recognition via EEGNet in Wireless Sensing2023 13th International Conference on Computer and Knowledge Engineering (ICCKE)10.1109/ICCKE60553.2023.10326282(259-264)Online publication date: 1-Nov-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Get Access

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media