Human Activity Recognition (HAR) represents an important task for many healthcare applications. From the perspective of developing patient-specific solutions, it is clear how the use of artificial intelligence enhances the potential of HAR. The present work settles its roots in the context of early-diagnosis of neurodevelopmental disorders in children (Autism Spectrum Disorder, ASD) and in the evaluation of their motor skills. In this paper, we present an artificial intelligence-based approach for fine-grained HAR which relies on dead-reckoning applied to data collected through inertial measurement units (IMUs). This approach has been applied on a dataset collected through IMU-embedded toys in order to validate its feasibility in the inference of infants fine-grained motor skills. The proposed solution’s workflow starts from the estimation of the orientation and position of solid objects through dead-reckoning exploiting Kalman filters and moves to the extraction of informative features, which are then used to feed a Temporal Convolutional Network (TCN). The achieved training average accuracy of 89% highlights how such a non-intrusive approach reaches great performances on HAR tasks, even overcoming the limitations of most of the works already present in literature, based on wearable sensors and/or computer vision techniques. The presented work and achieved results represent a solid base for IoT-based systems aiming at supporting clinicians in the early diagnosis of ASD in children.
✎ Paper

Incorporating machine and deep learning methodologies into wearable devices has enhanced the capacity to accurately recognize human activity, thus enabling a range of applications including healthcare monitoring and fitness tracking. However, machine and deep learning can be costly in terms of the computational resources and energy consumption required. In this work, we study how a feature selection decision impacts the energy consumption of an ESP32 wearable device by evaluating the best trade-off between classification performance and energy expenditure. Experimental results, conducted on publicly available datasets, demonstrate that the best trade-off between energy consumption and accuracy is reached by selecting between 20 and 25 features, with an accuracy ranging between 73.56% and 87.44%, and an energy consumption between 2340.945 𝜇J and 3759.270 𝜇J.
✎ Paper

As a consequence of demographic shifts, the proportion of the older population is growing at an accelerated pace, resulting in a notable decline in cognitive and motor functions. This study examines the potential of wearables to monitor activities of daily living (ADL) and identify changes in behavior, thereby enabling early intervention to maintain independence of the person. Eight dynamic ADLs were analyzed using data collected from eight subjects who were wearing a sensor belt that includes an accelerator and a gyroscope. The data were preprocessed and employed to train and evaluate two distinct types of classifiers: a deep-learning and several machine learning approaches. Two data splits were considered: a subject-specific model, which utilized data from all subjects for training and testing, and a generalized model, which excluded one subject from the training set for validation. The subject-specific approach yielded high accuracies of 99.5% and 99.8% for the classification network and the best support vector machine, respectively. The generalized approach yielded accuracies of 77.8% for the convolutional neural network and 77.6% for the best support vector machine. While the classification network demonstrated marginally superior results, the support vector machine required significantly less training time, suggesting its potential suitability for practical applications.
✎ Paper

Cognitive fatigue may have significant results if not intervened in factories, automobiles, and office environments. Development of a system for monitoring cognitive fatigue in real-life settings using unobtrusive wearable devices can help to minimize health problems, and work and car accidents. Photoplethysmography (PPG) sensors offer an unobtrusive way to track changes in heart rate and heart rate variability (HRV), which are indicative of cognitive fatigue levels. In this study, we propose a personalized PPG normalization technique to reduce inter-subject variability and enhance the performance of machine learning algorithms in classifying cognitive fatigue. The best-performing model, a Random Forest Classifier, achieved an accuracy of 80.5% in binary classification and demonstrated robust performance in regression tasks as well. The study highlights the potential of PPG-based wearables for non-obtrusive, long-term monitoring of cognitive fatigue, which could aid in preventing health issues associated with chronic fatigue.
✎ Paper

We are witnessing an increasing interest in vision-language models (VLMs) as reflected in the impressive results across a large spectrum of tasks. In this context, we introduce in this paper a novel architecture that exploits the capabilities of VLMs for facial expression recognition in driving environment to enhance road safety. We present an approach called CLIVP-FER, which uses the Contrastive Language-Image Pretraining (CLIP) and combines both visual and textual data to overcome the environmental challenges and ambiguities in facial expression interpretation. In addition, we apply average pooling to improve the accuracy and the computational efficiency. The proposed approach is thoroughly evaluated on a benchmark driving dataset called KMU-FED. The experiments showed superior performance compared to state-of-the-art methods, achieving an average accuracy of 97.36%. Cross-database evaluation is also provided showing good generalization abilities. The ablation study gives more insights into the performance of our proposed architecture. The obtained results are interesting and confirm the capabilities of vision-language models in vision tasks, demonstrating their promising applications in efficient driver assistance and intervention systems. We are making the code of this work publicly available for research purposes at https://github.com/Ibtissam-SAADI/CLIVP-FER.
✎ Paper

The contactless detection of vital parameters in humans and animals is crucial for ensuring health. While sensors that touch the body or are invasive can already accurately capture parameters such as body temperature, heart rate, or respiratory rate, contactless systems are still under development. An interesting technological approach involves the use of thermal cameras for capturing vital data, as they can operate unobtrusively even in complete darkness and from a greater distance. Unfortunately, the section of measurements affects the recorded temperature; the absolute temperature is imprecise, and calibration is very complex and should be redone periodically. This paper presents a simple yet effective solution for enhancing thermal imaging. The solution is a temperature-controlled Peltier element with a feedback loop that provides an exact reference point in the desired temperature range. This might improve the detection of fever and infections and support further vital data recognition in humans and animals or condition monitoring in machines using thermal cameras.
✎ Paper

This paper examines the use of computer vision algorithms to estimate aspects of the psychosocial work environment using CCTV footage. We present a proof of concept for a methodology that detects and tracks people in video footage and estimates interactions between customers and employees by estimating their poses and calculating the duration of their encounters. We propose a pipeline that combines existing object detection and tracking algorithms (YOLOv8 and DeepSORT) with pose estimation algorithms (BlazePose) to estimate the number of customers and employees in the footage as well as the duration of their encounters. We use a simple rule-based approach to classify the interactions as positive, neutral or negative based on three different criteria: distance, duration and pose. The proposed methodology is tested on a small dataset of CCTV footage. While the data is quite limited in particular with respect to the quality of the footage, we have chosen this case as it represents a typical setting where the method could be applied. The results show that the object detection and tracking part of the pipeline has a reasonable performance on the dataset with a high degree of recall and reasonable accuracy. At this stage, the pose estimation is still limited to fully detect the type of interactions due to difficulties in tracking employees in the footage. We conclude that the method is a promising alternative to self-reported measures of the psychosocial work environment and could be used in future studies to obtain external observations of the work environment.
✎ Paper

In this paper, we introduce a new type of bionic AI that enhances decision-making unpredictability by incorporating responses from a living fly. Traditional AI systems, while reliable and predictable, lack nuanced and sometimes unseasoned decision-making seen in humans. Our approach uses a fly’s varied reactions, to tune an AI agent in the game of Gobang. Through a study, we compare the performances of different strategies on altering AI agents and found a bionic AI agent to outperform human as well as conventional and white-noise enhanced AI agents. We contribute a new methodology for creating a bionic random function and strategies to enhance conventional AI agents ultimately improving unpredictability.
✎ Paper

In this paper we present SurfSole, an untethered mobile system combining a smart sole prototype and mobile app to achieve real-time surface identification. We solely rely on the technology of capacitive sensing while choosing a neural network approach for classification. We evaluated different machine learning models with different layer architectures of 3-4 layers with 32, 64, 128, and 256 filters. The theoretical overall accuracy reaches from 74.85% up to 87.11%. While we retrieve data with 40Hz with a single window of 120 data points, we have a real-time detection delay of 3s.
✎ Paper

In this paper, we introduce a proof-of-concept multi-sensory footwear prototype with artificial intelligence. We equipped a shoe with a force sensitive pressure sensor, an accelerometer, and a gyroscope in order to detect human activity. We trained and compared several models, which are two types of convolutional neural networks (CNN) and a conventional support vector machine (SVM). The system's accuracy in identifying activities like standing, sitting, walking, running, and jumping was evaluated, and scored highest using a MobileNet CNN with 83.33% accuracy. With this work, we demonstrate that a somewhat robust real-time activity recognition is feasible with prototypical hardware.
✎ Paper

This work presents a novel concept for achieving optimal coverage of an unspecified room using 2D-LiDAR (Light Detection and Ranging) sensors. The primary goal is to maximize coverage with the fewest possible sensors. We present an algorithm that determines the ideal locations for these sensors, which are all mounted on the floor by the walls. By dividing the room into a grid with adjustable cell sizes (e.g., 10x10 cm), the algorithm marks all grid cells detected by each potential sensor location. This process is repeated for all possible locations. Based on the resulting coverage map, the algorithm calculates the minimum number of required sensors and their optimal positions. An application case for this approach is movement and fall detection using 2D-LiDAR.
✎ Paper

Stroke is one of the major causes of disability worldwide, and patients with residual neurological deficits are recommended to undergo rehabilitation. Besides baseline medical conditions, many other factors play a role in rehabilitation success, including patient engagement and medical complications. Wearable technology allows objective and continuous monitoring of body functions and behavior. Furthermore, wearable sensors could detect the risk of adverse events such as falls, infection and mood disorders. In the past, we gained experience using wearable sensors for stroke gait analysis. For example, we have evaluated algorithms to quantify gait patterns using inertial measurement units (IMUs) data and demonstrated that we can quantify and visualize changes in gait patterns in both healthy and stroke populations. As a next step, we aim to combine multiple wearable sensors and data modalities to further assess patient performance in rehabiliation. This planned study aims to investigate the efficacy of wearable devices, including, continuous glucose monitoring (CGM), and smartwatch devices. We will not only investigate the potential of wearable devices in quantifying the improvement in neurological function and reducing complications during the early stroke rehabilitation process, but also assess the effects of using wearables in improving patient engagement and motivation during early stroke rehabilitation.
✎ Paper

Mental and physical health are interlinked. While there are many objective measurement methods and diagnostic tools for physical disorders, the factors influencing mental health are wide-ranging and very subjective. As a result, there is a need for an objective method for assessing stress and stress resilience. The main parameter used today is heart rate vari- ability, which unfortunately varies according to age, person and physical health. In this paper, we analyze the effects of respiratory sinus arrhythmia (RSA) on stress and promote RSA as a possible new and advanced parameter for stress resilience. In an evaluation with 20 subjects, we demonstrate RSA variation and define an RSA parameter that can be as- sessed remotely without physical contact using computer vision and a standard webcam.
✎ Paper

The increasing popularity of conducting studies in real-life settings, known as "studies in-the-wild," is a valuable addition to the traditional controlled clinical trials. These studies enable the observation of long-term effects and account for the complex influences of everyday life. Body-worn sensors facilitate the continuous and unobtrusive collection of motion data in its user’s natural, everyday life environment. However, studies in-the-wild require careful planning regarding equipment usability, accessibility, and the creation of efficient study protocols to maximize the quality and output of the collected data. This paper presents insights from our recent study on compulsive handwashing, highlighting the challenges and strategies in study design, implementation, and label acquisition in order to perform supervised machine learning. We present approaches as well as the benefits and limitations of annotating data retrospectively so that participants are impacted minimally during the study. Finally, we list our learning and insights for upcoming studies of that kind.
✎ Paper

Hand washing is a crucial part of personal hygiene. Hand washing detection is a relevant topic for wearable sensing with applications in the medical and professional fields. Hand washing detection can be used to aid workers in complying with hygiene rules. Hand washing detection using body-worn IMU-based sensor systems has been shown to be a feasible approach, although, for some reported results, the specificity of the detection was low, leading to a high rate of false positives. In this work, we present a novel, open-source prototype device that additionally includes a humidity, temperature, and barometric sensor. We contribute a benchmark dataset of 10 participants and 43 hand-washing events and perform an evaluation of the sensors’ benefits. Added to that, we outline the usefulness of the additional sensor in both the annotation pipeline and the machine learning models. By visual inspection, we show that especially the humidity sensor registers a strong increase in the relative humidity during a hand-washing activity. A machine learning analysis on our data shows that distinct features benefiting from such relative humidity patterns remain to be identified.
✎ Paper

This paper presents the Equimetrics data capture system. The primary objective is to apply HAR principles to enhance the understanding and optimization of equestrian performance. By integrating data from strategically placed sensors on the rider’s body and the horse’s limbs, the system provides a comprehensive view of their interactions. Preliminary data collection has demonstrated the system’s ability to accurately classify various equestrian activities, such as walking, trotting, cantering, and jumping, while also detecting subtle changes in rider posture and horse movement. The system leverages open-source hardware and software to offer a cost-effective alternative to traditional motion capture technologies, making it accessible for researchers and trainers. The Equimetrics system represents a significant advancement in equestrian performance analysis, providing objective, data-driven insights that can be used to enhance training and competition outcomes. The system has been made available at [1].
✎ Paper

Like skateboarding acceleration in sur2ing on a hydrofoil with muscle power is achieved by a constant sinusoidal motion. Both are challenging sports to begin with because learning the complex up and down movements takes time, skill and re2lexion. The interplay of rotating joints and applying forces at the right time is hard to perceive, understand and to transfer into muscle memory. Since the motions in skateboarding on pump tracks and hydrofoil pumping are similar, we are comparing both motion sequences with inertial measurement units and 3D pose estimation. We postulate that learning the physically challenging and expensive hydrofoil pumping can be improved and accelerated by training with skateboards. Therefore, we are capturing forces with inertial measurement units and validate them with 3D pose estimation. Finally, we are comparing and visualizing the motions and forces of the boards and the skeleton.
✎ Paper

In the age of smart buildings and ubiquitous computing, accurately recognizing whether a user is taking the stairs or the elevator is beneficial for encouraging health and wellness. Encouraging the use of stairs over lifts can help boost physical activity among building residents, which can lead to better cardiovascular health, weight management, and general fitness. By precisely tracking and boosting stair usage, building management systems can give users health insights and motivating reminders, encourage a healthy lifestyle and lower the risk of sedentary-related health problems. This research describes a new and exploratory dataset for stair and lift usage to examine the patterns and behaviors related with using stairs and lifts. For this dataset, we collected data from 20 participants as they used stairs and lifts in a variety of scenarios. This preliminary dataset intends to provide insights and demonstrate the practicality of our choice of data collection, which will be used to inform future larger-scale studies. The newly collected dataset was used to train and test a Random Forest machine learning model. The results show that our method is highly accurate at classifying stair and lift operations with an accuracy of 87.61% and a multiclass weighted F1-score of 87.56% over 8-second time-windows. Furthermore, we investigate the effect of various sensor kinds and data attributes on the model’s performance. Our findings show that combining movement and pressure sensors yields a viable solution for real-time activity detection in smart environments. This research helps to produce intelligent systems capable of improving the functioning and responsiveness of modern infrastructure.
✎ Paper

This paper presents a novel approach to exercise repetition analysis using the YOLOv8-pose model and Dynamic Time Warping (DTW) techniques applied to the InfiniteRep dataset. Our research addresses the challenges of accurate pose estimation and tracking in dynamic camera environments and with varying occlusions in synthetic datasets. By integrating YOLOv8’s pose detection capabilities with the temporal analysis strength of DTW, we propose a method that significantly improves the detection and classification of exercise repetitions across diverse conditions. We demonstrate the effectiveness of this approach through rigorous experiments that test various scenarios, including changes in camera angles and exercise complexity. Our results indicate notable improvements in both the accuracy and robustness of exercise recognition, suggesting promising applications in sports science and personal fitness coaching.
✎ Paper