HARrad

Introduction

Many smart home applications rely on indoor human activity recognition. This challenge is currently primarily tackled by employing video camera sensors. However, the use of such sensors is characterized by fundamental technical deficiencies in an indoor environment, often also resulting in a breach of privacy. In contrast, a radar sensor resolves most of these flaws and maintains privacy in particular.

We investigated a novel approach towards automatic indoor human activity recognition, feeding high-dimensional radar and video camera sensor data into several deep neural networks. Furthermore, we explored the efficacy of sensor fusion to provide a solution in less than ideal circumstances. To that end, we constructed and publish two data sets that consist of 2347 and 1505 samples distributed over six different types of gestures and events, respectively.[1] As described in [1], we employ six different deep neural networks to classify different activities based on a radar and video camera sensor. A complete overview of this study can be found in our published paper at:

https://link.springer.com/article/10.1007/s00521-019-04408-1.

The code linked to this paper can be found at https://github.com/baptist/harrad. More information on our team can be found at http://www.idlab.ugent.be and http://www.sumo.intec.ugent.be/.

In case you find the data set useful, please cite the accompanying paper [1].

[1] Vandersmissen, Baptist, et al. "Indoor human activity recognition using high-dimensional sensors and deep neural networks." Neural Computing and Applications (2019): 1-15.

Summary

To facilitate further research, we developed and release the Human Activity Recognition with a Radar (HARrad) data set. HARrad was retrieved using a Frequency Modulated Continuous Wave (FMCW) radar with a center frequency of 77GHz. In total the data set consists of 37 h5py files containing the raw FMCW radar data.

Table 1 lists the details per recording subject. In total the data sets contain 3852 activities, taking on average 2.56 s per activity,  subdivided in 1505 event-related activities and 2347 gesture-related activities. Our data sets thus contain a total of 4.32 hours of effectively annotated activity data distributed over 12 classes.

Table 1: Details of radar files per subject.

The labeling of the data is provided in the activities.csv file. More precisely, this file contains the description of each activity sample (per line) in the form of:

<filename>,<start index>,<stop index>,<activity class>,<subject>

Furthermore. We also provide the random stratified splits (RS) used in the paper [1]. In Table 2, we listed the number of samples per activity class and Table 3 shows a more detailed distribution of all samples per subject and activity class.

Table 2: The following table lists the number of samples and the corresponding average duration of each activity for both data sets.

Table 3: Number of recorded events and gestures per subject Si , with i ∈ {1 . . . 9}.

Download