This work presents a fall detection system based on artificial intelligence. The system incorporates miniature wearable devices for fall detection. Fall detection is achieved by integrating a three-axis gyroscope and a three-axis accelerometer. The system gathers the differential data collected by the gyroscope and accelerometer, applies artificial intelligence algorithms for model training and constructs an effective model for fall detection. To provide easy wearing and effective position detection, it is designed as a small device attached to the user’s waist. Experiment results have shown that the accuracy of the proposed fall detection model is up to 98%, demonstrating the effectiveness of the model in real-life fall detection.
The current reduction in birth rate and extended life expectancy have resulted in the ageing of the population, which has become a global issue. According to [
Falling is an important health issue for the senior population as it may cause serious harm to the elderly [
We propose a system based on artificial intelligence for fall detection to reduce manual cost and provide early warnings. The system’s goal is to detect the current positions of personnel within a monitored range and determine if a falling event has occurred. The current behaviours of the personnel are returned to the monitoring system for centralized record and management to establish the conditions of indoor activities of the personnel. The proposed system integrates a three-axis gyroscope, a three-axis accelerometer, as well as a Bluetooth module for wireless communication to design a waist-attached, miniature fall detection device. The device collects information from the gyroscope and accelerometer for analysis to deduce continuous signals representing the human body postures. According to the posture and signal relationship, artificial intelligence is used to construct a highly accurate model. Information associated with falling is transmitted by the waist-attached, miniature fall detection device
The current fall detection system implementation methods can be divided into two categories: fixed fall detection system and non-fixed fall detection system. The fixed fall detection system [
In 2014, Wang et al. [
In 2017, De Miguel et al. [
In 2018, Shahzad et al. [
In the same year, Ichwana et al. [
In 2020, Nooruddin et al. [
In the same year, Clemente et al. [
Nowadays, artificial intelligence is widely used in various systems. This research uses TensorFlow as the framework for building a neural network (NN) and uses three different neural network architectures to build a fall detection model. The purpose of the neural network is to find the optimal weight value and deviation, which are obtained when the input data passes through the function in the framework. Therefore, an activation function is added after the hidden layer of the neural network to facilitate the finding of the desired result and calculate the suitable value to build a predictive model for fall detection. This work uses the multi-layer perceptron (MLP), recurrent neural network (RNN) and long short-term memory (LSTM) models to compare their advantages and disadvantages as shown in
Neural network | Advantages | Disadvantages |
---|---|---|
MLP | Simple structure |
Difficult to select the number of nodes. |
RNN | Able to handle continuous signals | Suffers from the vanishing gradient or exploding gradient problems. |
LSTM | Continuous processing |
Suffers from the exploding gradient problem. |
The MLP is a kind of feedforward neural network with three layers: an input layer, a hidden layer and an output layer. The input layer is the current material to be learned. The hidden layer is the feature node that needs to be learned and each node is a neuron with a nonlinear activation function so that its output meets the required results. The output layer is the category that needs to be learned. Each layer in the middle is a fully connected layer. The multi-layer perceptron was widely used in the 1980s in applications such as speech recognition, image recognition, machine translation and so on, and achieved good results in classification problems.
The early neural network architecture had no time concept and labeling learning for sequence signals. In 1982, Hopfield [
Jordan [
The LSTM is a recurrent neural network. Since the RNN architecture is unsuitable for very long time series, Hochreiter et al. [
The LSTM architecture consists of three layers, the forget gate, the input gate and new input, and the output gate. The forget gate determines which old memories to retain, the input gate and the new input determine which values need to be added and the output gate determines which values to output.
The activation function in neural networks mainly uses nonlinear equations to solve nonlinear problems. When the activation function is not used, the neural network is a linear combination of operations. As both the hidden layer and the output layer are the input results of the upper layer, they are calculated with the weight value and the deviation, and the calculation result is regarded as the output of the layer so that the output and the input have a linear relationship. If a nonlinear activation function is not used, the learned model cannot solve a nonlinear problem. In this work, we have investigated 4 different activation functions for learning the framework and selected suitable functions through experiments to implement the fall detection model.
According to
According to
The function is calculated according to
The function is provided in
This research incorporates a three-axis gyroscope, a three-axis acceleration sensor, as well as the Bluetooth radio frequency technology, Raspberry Pi system, management terminal interface program, cloud database and other related components and technology to implement the smart fall detection system. The system also uses artificial intelligence algorithms to improve the accuracy of fall detection.
In this work, we propose an efficient solution for the monitoring center to confirm the status of a user wearing the device
The system architecture is shown in
As the device worn by the user obtains the six-axis signal, the smart fall detection algorithm is used to calculate the current walking state of the user and information packets are continuously sent to the information receiving device through the Bluetooth broadcast mode. The information receiver calculates the RSSI value of the received packet and selects the information receiver with the best RSSI value for data transmission. When the information receiver receives the relevant signal, it will be sent to the management computer through the wired network. The signal will be analyzed through the smart fall detection model to find the current walking status of the user and the user status will be displayed on the interface program. The management computer will also store the analysis data on the cloud database.
The wearing device uses the Texas Instruments CC2541 Bluetooth wireless chip. The core of the chip is an 8051 microcontroller and a six-axis sensing module (MPU-6050). The fall detection algorithm analyzes the signals generated during the user’s actions to determine if a fall has occurred. The power source of the device is a 3.7 V, 1000 mAh, polymer lithium battery and its working endurance is about 30 days.
When the user attaches the device to the waist and turns the power on, the power level is automatically detected, which allows the management staff to confirm if the power of the device is sufficient and check the status of the wearing device. The smart fall detection model on the wearing device starts to continuously detect for a fall event and uses the information packet to send the detected state to the information receiver for transmission. The information receiver analyzes whether or not the received packet needs to be forwarded to the management computer and discard the packets sent by the non-wearing device.
The status information of the correct packet is sent to the management computer through the TCP/IP Socket, the content of the information packet is analyzed through the computer interface program and the result is displayed on the graphical user interface. The management personnel is able to check the status information of all users in the field through the computer interface program. When a person accidentally falls or activates the emergency button, the displayed user marker will be marked in different colors and an alarm page will be displayed, allowing the management staff to quickly proceed to the location and deal with the situation.
To solve the difference between the fall and non-fall states of a user, we use a neural network to build a detection model and find an optimal algorithm to solve the problem. The development software for building the model is Spyder and the model is implemented using the Python programming language and Tensorflow framework.
The proposed system uses a variety of neural network models to find a model that consumes fewer resources and is highly accurate. The established model is programmed in the C/C++ language to reproduce the algorithm and apply it to smart fall detection.
To capture the complete fall action, all 6 signals from the six axes are used and every 80 times is taken as a complete action. The once time is 10 ms. A total of 110 fall actions and 130 other actions are captured as the training material.
In this work, 3 different neural network models have been chosen for comparison with respect to the accurate determination of the differences between each action. The three models are the multi-layer perceptron (MLP), recurrent neural network (RNN) and long short-term memory (LSTM) models. When building the model, 160 motion signals are used for training, and 80 signals are used for testing the trained model. The input layer is 6 * 80 groups of fall signal values and the output layer is 2 to indicate if a fall action has been detected. After modifying the number of nodes, the number of layers, the activation function of the intermediate hidden layer, and considering the resource constraints of the controller, it is impossible to store too many model weights. The system uses the Tensorflow framework to train a model that meets the demands and select a model with low resource consumption and high accuracy out of the three chosen models.
The architecture diagram of the MLP fall model is shown in
The architecture diagram of the RNN fall model is shown in
The architecture diagram of the LSTM fall model is shown in
Corresponding correction methods and discussions are proposed according to the test data recorded in the smart fall detection model from training and testing. The proposed system is evaluated with respect to the model test data collection, model architecture accuracy and resource consumption.
The system adopts a non-fixed fall detection system. We install the wearable device on the test person. The accelerometer sensor on the wearable device will detect the falling state of the test person at any time and determine whether a fall has occurred. Since the acceleration stability is different when acting in different positions. This experiment will install the wearable device on the chest, wrist and waist to test the stability of the impact.
In
To capture the complete fall action, the 6 signals of the six axes are acquired once every 10 ms and every 80 acquisitions constitute a complete action, as shown in
The six-axis signal is captured every 10 ms and if the weight value is too large, it cannot be burned into the microcontroller. Therefore, the calculation time of the model must be within 10 ms and the resources consumed by the number of node weights must also be met. The learning effect varies according to the hidden layer, the number of nodes and the activation function. In this work, we have selected several model architectures with good results in training and testing. The final test model does not show a large difference between the training and testing accuracies, which means that the model has high reliability.
Since each layer of the MLP architecture is a fully connected layer with 480 input nodes, where each node in the first layer has 480 weight values and 1 deviation. As a result, the number of weight values is very large and the microcontroller cannot load the resources. In the end, the algorithm is not implemented in the fall detection model and the resource consumption is not calculated.
Judge action | Number of layers | Number of nodes | Dropout | Activation function |
---|---|---|---|---|
A fall action is detected | 1 layer | 6 | 0.7 | ReLU |
A fall action is detected | 1 layer | 9 | 0.7 | ReLU |
A fall action is detected | 2 layers | 1st layer 4 |
0.7 | ReLU |
A fall action is detected | 2 layers | 1st layer 6 |
0.7 | ReLU |
Judge action | Number of weights | Training |
Testing |
Calculated |
---|---|---|---|---|
A fall action is detected | 2886 hidden layers |
88 | 86 | 4.1 |
A fall action is detected | 4329 hidden layers 20 output layers | 92 | 90 | 5.6 |
A fall action is detected | 1954 hidden layers |
93 | 92 | 3.3 |
A fall action is detected | 2949 hidden layers 20 output layers | 95 | 94 | 4.3 |
Since the trained RNN model has better results, the model is divided into two architectures to judge a fall action and other six kinds of actions, to facilitate better understandings of the user’s current actions and prevent a user from performing dangerous actions, such as a jump. If a user’s action before the fall can be detected, the rescue can be carried out more effectively.
The number of weights on a single node in the hidden layer of the RNN is the number of input nodes plus the number of nodes in the hidden layer. For example, if the input value of the hidden layer is 6 and the number of hidden nodes is 12, then each node of the layer has 18 weight values and 1 deviation value.
Judge action | Number of layers | Number of nodes | Dropout | Activation |
---|---|---|---|---|
A fall action is detected | 1 layer | 6 | 0.7 | ReLU |
A fall action is detected | 1 layer | 9 | 0.7 | ReLU |
Six actions | 2 layers | 1st layer 12 |
0.7 | ReLU |
Six actions | 2 layers | 1st layer 18 |
0.7 | ReLU |
Judge action | Total number |
Training |
Testing |
---|---|---|---|
A fall action is detected | 6 hidden nodes |
96 | 95 |
A fall action is detected | 9 hidden nodes; |
99 | 98 |
Six actions | 1st layer 12 hidden nodes |
96 | 95 |
Six actions | 1st layer 18 hidden nodes |
97 | 93 |
Since the trained LSTM model has better results, it has two architectures like the RNN model. Each layer of the LSTM model has 4 sets of nodes to determine the memory and output values. The number of hidden layer nodes will be 4 times the setting and the weight value of each node is the same as the RNN, which is the input plus the recursion number. For example, if the hidden layer is set to 12 nodes, the input value is 6, which means that the layer has 48 nodes, and each node has 18 weight values and a deviation value.
Architecture | Number of weights | One-time calculation |
---|---|---|
1-layer RNN hidden layer with 6 nodes |
78 hidden layers |
4.8 |
1-layer RNN hidden layer with 9 nodes |
144 hidden layers; |
5.8 |
Judge action | Number of layers | Number of |
Dropout | Activation |
---|---|---|---|---|
A fall action is detected | 1 layer | 6 | 0.7 | sigmoid |
A fall action is detected | 1 layer | 9 | 0.7 | sigmoid |
Six actions | 2 layers | 1st layer 9 |
0.7 | sigmoid |
Six actions | 2 layers | 1st layer 12 |
0.7 | sigmoid |
Judge action | Total number of nodes | Training |
Testing |
---|---|---|---|
A fall action is detected | 24 (4 times 6) hidden nodes |
97 | 94 |
A fall action is detected | 36 (4 times 9) hidden nodes |
98 | 94 |
Six actions | 1st layer 36 hidden nodes |
95 | 94 |
Six actions | 1st layer 48 hidden nodes; |
97 | 95 |
Architecture | Number of weights | One-time calculation |
---|---|---|
2-layers LSTM hidden layer with 84 nodes |
1632 hidden layers |
20 |
2-layers RNN hidden layer with 120 nodes |
3144 hidden layers |
30 |
Comparison |
Model | Number of |
Number of |
Time |
Accuracy |
---|---|---|---|---|---|
MLP fall architecture | A fall action is detected | 1st layer 6 hidden nodes; |
2969 | 4.3 | 94.5 |
RNN fall architecture | A fall action is detected | 9 hidden nodes; |
164 | 5.8 | 98.5 |
LSTM fall architecture | Six actions | 1st layer 18 hidden nodes |
3258 | 30 | 96 |
In this work, we have proposed a smart fall detection system that uses artificial intelligence to implement fall detection to improve the accuracy of human fall detection. The proposed system uses a variety of different neural network models to learn a low-resource and high-accuracy algorithm. The system provides the users with a convenient management interface program so that the management staff can quickly and efficiently handle accidental situations. The experimental results have demonstrated that the accuracy of fall detection can be up to 98%.
At present, artificial intelligence is widely used to facilitate systems that require large amounts of data and noise processing. Still, the model architecture often requires too many resources to be written into the microcontroller, such as the LSTM algorithm used in this research. In the future, if the processing resources of the hardware microcontroller become sufficient, artificial intelligence can be effectively applied to various signal-related systems.