2020
Kyritsis, Konstantinos; Diou, Christos; Delopoulos, Anastasios
A Data Driven End-to-end Approach for In-the-wild Monitoring of Eating Behavior Using Smartwatches Journal Article
In: IEEE Journal of Biomedical and Health Informatics, 2020.
Abstract | Links | BibTeX | Tags: Feature extraction, Informatics, Monitoring, Obesity, Sensors, Tools, Wrist
@article{kyritsis2020data,
title = {A Data Driven End-to-end Approach for In-the-wild Monitoring of Eating Behavior Using Smartwatches},
author = {Konstantinos Kyritsis and Christos Diou and Anastasios Delopoulos},
doi = {10.1109/JBHI.2020.2984907},
year = {2020},
date = {2020-01-01},
journal = {IEEE Journal of Biomedical and Health Informatics},
publisher = {IEEE},
abstract = {The increased worldwide prevalence of obesity has sparked the interest of the scientific community towards tools that objectively and automatically monitor eating behavior. Despite the study of obesity being in the spotlight, such tools can also be used to study eating disorders (e.g. anorexia nervosa) or provide a personalized monitoring platform for patients or athletes. This paper presents a complete framework towards the automated i) modeling of in-meal eating behavior and ii) temporal localization of meals, from raw inertial data collected in-the-wild using commercially available smartwatches. Initially, we present an end-to-end Neural Network which detects food intake events (i.e. bites). The proposed network uses both convolutional and recurrent layers that are trained simultaneously. Subsequently, we show how the distribution of the detected bites throughout the day can be used to estimate the start and end points of meals, using signal processing algorithms. We perform extensive evaluation on each framework part individually. Leave-one-subject-out (LOSO) evaluation shows that our bite detection approach outperforms four state-of-the-art algorithms towards the detection of bites during the course of a meal (0.923 F1 score). Furthermore, LOSO and held-out set experiments regarding the estimation of meal start/end points reveal that the proposed approach outperforms a relevant approach found in the literature (Jaccard Index of 0.820 and 0.821 for the LOSO and held-out experiments, respectively). Experiments are performed using our publicly available FIC and the newly introduced FreeFIC datasets.},
keywords = {Feature extraction, Informatics, Monitoring, Obesity, Sensors, Tools, Wrist},
pubstate = {published},
tppubtype = {article}
}
The increased worldwide prevalence of obesity has sparked the interest of the scientific community towards tools that objectively and automatically monitor eating behavior. Despite the study of obesity being in the spotlight, such tools can also be used to study eating disorders (e.g. anorexia nervosa) or provide a personalized monitoring platform for patients or athletes. This paper presents a complete framework towards the automated i) modeling of in-meal eating behavior and ii) temporal localization of meals, from raw inertial data collected in-the-wild using commercially available smartwatches. Initially, we present an end-to-end Neural Network which detects food intake events (i.e. bites). The proposed network uses both convolutional and recurrent layers that are trained simultaneously. Subsequently, we show how the distribution of the detected bites throughout the day can be used to estimate the start and end points of meals, using signal processing algorithms. We perform extensive evaluation on each framework part individually. Leave-one-subject-out (LOSO) evaluation shows that our bite detection approach outperforms four state-of-the-art algorithms towards the detection of bites during the course of a meal (0.923 F1 score). Furthermore, LOSO and held-out set experiments regarding the estimation of meal start/end points reveal that the proposed approach outperforms a relevant approach found in the literature (Jaccard Index of 0.820 and 0.821 for the LOSO and held-out experiments, respectively). Experiments are performed using our publicly available FIC and the newly introduced FreeFIC datasets.
Fagerberg, Petter; Klingelhoefer, Lisa; Bottai, Matteo; Langlet, Billy; Kyritsis, Konstantinos; Rotter, Eva; Reichmann, Heinz; Falkenburger, Björn; Delopoulos, Anastasios; Ioakimidis, Ioannis
In: Nutrients, 12 (7), pp. 2109, 2020.
Abstract | Links | BibTeX | Tags: eating behavior, energy intake, food, malnutrition, Monitoring, neurodegenerative diseases, Parkinson’s disease, weight loss
@article{fagerberg2020lower,
title = {Lower Energy Intake among Advanced vs. Early Parkinson’s Disease Patients and Healthy Controls in a Clinical Lunch Setting: A Cross-Sectional Study},
author = {Petter Fagerberg and Lisa Klingelhoefer and Matteo Bottai and Billy Langlet and Konstantinos Kyritsis and Eva Rotter and Heinz Reichmann and Björn Falkenburger and Anastasios Delopoulos and Ioannis Ioakimidis},
doi = {https://doi.org/10.3390/nu12072109},
year = {2020},
date = {2020-01-01},
journal = {Nutrients},
volume = {12},
number = {7},
pages = {2109},
publisher = {Multidisciplinary Digital Publishing Institute},
abstract = {Unintentional weight loss has been observed among Parkinson’s disease (PD) patients. Changes in energy intake (EI) and eating behavior, potentially caused by fine motor dysfunction and eating-related symptoms, might contribute to this. The primary aim of this study was to investigate differences in objectively measured EI between groups of healthy controls (HC), early (ESPD) and advanced stage PD patients (ASPD) during a standardized lunch in a clinical setting. The secondary aim was to identify clinical features and eating behavior abnormalities that explain EI differences. All participants (n = 23 HC, n = 20 ESPD, and n = 21 ASPD) went through clinical evaluations and were eating a standardized meal (200 g sausages, 400 g potato salad, 200 g apple purée and 500 mL water) in front of two video cameras. Participants ate freely, and the food was weighed pre- and post-meal to calculate EI (kcal). Multiple linear regression was used to explain group differences in EI. ASPD had a significantly lower EI vs. HC (−162 kcal, p < 0.05) and vs. ESPD (−203 kcal, p < 0.01) when controlling for sex. The number of spoonfuls, eating problems, dysphagia and upper extremity tremor could explain most (86%) of the lower EI vs. HC, while the first three could explain ~50% vs. ESPD. Food component intake analysis revealed significantly lower potato salad and sausage intakes among ASPD vs. both HC and ESPD, while water intake was lower vs. HC. EI is an important clinical target for PD patients with an increased risk of weight loss. Our results suggest that interventions targeting upper extremity tremor, spoonfuls, dysphagia and eating problems might be clinically useful in the prevention of unintentional weight loss in PD. Since EI was lower in ASPD, EI might be a useful marker of disease progression in PD. },
keywords = {eating behavior, energy intake, food, malnutrition, Monitoring, neurodegenerative diseases, Parkinson’s disease, weight loss},
pubstate = {published},
tppubtype = {article}
}
Unintentional weight loss has been observed among Parkinson’s disease (PD) patients. Changes in energy intake (EI) and eating behavior, potentially caused by fine motor dysfunction and eating-related symptoms, might contribute to this. The primary aim of this study was to investigate differences in objectively measured EI between groups of healthy controls (HC), early (ESPD) and advanced stage PD patients (ASPD) during a standardized lunch in a clinical setting. The secondary aim was to identify clinical features and eating behavior abnormalities that explain EI differences. All participants (n = 23 HC, n = 20 ESPD, and n = 21 ASPD) went through clinical evaluations and were eating a standardized meal (200 g sausages, 400 g potato salad, 200 g apple purée and 500 mL water) in front of two video cameras. Participants ate freely, and the food was weighed pre- and post-meal to calculate EI (kcal). Multiple linear regression was used to explain group differences in EI. ASPD had a significantly lower EI vs. HC (−162 kcal, p < 0.05) and vs. ESPD (−203 kcal, p < 0.01) when controlling for sex. The number of spoonfuls, eating problems, dysphagia and upper extremity tremor could explain most (86%) of the lower EI vs. HC, while the first three could explain ~50% vs. ESPD. Food component intake analysis revealed significantly lower potato salad and sausage intakes among ASPD vs. both HC and ESPD, while water intake was lower vs. HC. EI is an important clinical target for PD patients with an increased risk of weight loss. Our results suggest that interventions targeting upper extremity tremor, spoonfuls, dysphagia and eating problems might be clinically useful in the prevention of unintentional weight loss in PD. Since EI was lower in ASPD, EI might be a useful marker of disease progression in PD.
2019
Kyritsis, Konstantinos; Diou, Christos; Delopoulos, Anastasios
Modeling wrist micromovements to measure in-meal eating behavior from inertial sensor data Journal Article
In: IEEE journal of biomedical and health informatics, 23 (6), pp. 2325–2334, 2019.
Abstract | Links | BibTeX | Tags: Acceleration, Feature extraction, Informatics, Monitoring, Mouth, Obesity, Wrist
@article{kyritsis2019modeling,
title = {Modeling wrist micromovements to measure in-meal eating behavior from inertial sensor data},
author = {Konstantinos Kyritsis and Christos Diou and Anastasios Delopoulos},
doi = {10.1109/JBHI.2019.2892011},
year = {2019},
date = {2019-01-01},
journal = {IEEE journal of biomedical and health informatics},
volume = {23},
number = {6},
pages = {2325--2334},
publisher = {IEEE},
abstract = {Overweight and obesity are both associated with in-meal eating parameters such as eating speed. Recently, the plethora of available wearable devices in the market ignited the interest of both the scientific community and the industry toward unobtrusive solutions for eating behavior monitoring. In this paper, we present an algorithm for automatically detecting the in-meal food intake cycles using the inertial signals (acceleration and orientation velocity) from an off-the-shelf smartwatch. We use five specific wrist micromovements to model the series of actions leading to and following an intake event (i.e., bite). Food intake detection is performed in two steps. In the first step, we process windows of raw sensor streams and estimate their micromovement probability distributions by means of a convolutional neural network. In the second step, we use a long short-term memory network to capture the temporal evolution and classify sequences of windows as food intake cycles. Evaluation is performed using a challenging dataset of 21 meals from 12 subjects. In our experiments, we compare the performance of our algorithm against three state-of-theart approaches, where our approach achieves the highest F1 detection score (0.913 in the leave-one-subject-out experiment). The dataset used in the experiments is available at https://mug.ee.auth.gr/intake-cycle-detection/.},
keywords = {Acceleration, Feature extraction, Informatics, Monitoring, Mouth, Obesity, Wrist},
pubstate = {published},
tppubtype = {article}
}
Overweight and obesity are both associated with in-meal eating parameters such as eating speed. Recently, the plethora of available wearable devices in the market ignited the interest of both the scientific community and the industry toward unobtrusive solutions for eating behavior monitoring. In this paper, we present an algorithm for automatically detecting the in-meal food intake cycles using the inertial signals (acceleration and orientation velocity) from an off-the-shelf smartwatch. We use five specific wrist micromovements to model the series of actions leading to and following an intake event (i.e., bite). Food intake detection is performed in two steps. In the first step, we process windows of raw sensor streams and estimate their micromovement probability distributions by means of a convolutional neural network. In the second step, we use a long short-term memory network to capture the temporal evolution and classify sequences of windows as food intake cycles. Evaluation is performed using a challenging dataset of 21 meals from 12 subjects. In our experiments, we compare the performance of our algorithm against three state-of-theart approaches, where our approach achieves the highest F1 detection score (0.913 in the leave-one-subject-out experiment). The dataset used in the experiments is available at https://mug.ee.auth.gr/intake-cycle-detection/.
Kyritsis, Konstantinos; Diou, Christos; Delopoulos, Anastasios
Detecting meals in the wild using the inertial data of a typical smartwatch Inproceedings
In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4229–4232, IEEE 2019.
Abstract | Links | BibTeX | Tags: Accelerometers, Artificial neural networks, Detection algorithms, Feature extraction, Gyroscopes, Lenses, Monitoring
@inproceedings{kyritsis2019detecting,
title = {Detecting meals in the wild using the inertial data of a typical smartwatch},
author = {Konstantinos Kyritsis and Christos Diou and Anastasios Delopoulos},
doi = {10.1109/EMBC.2019.8857275},
year = {2019},
date = {2019-01-01},
booktitle = {2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)},
pages = {4229--4232},
organization = {IEEE},
abstract = {Automated and objective monitoring of eating behavior has received the attention of both the research community and the industry over the past few years. In this paper we present a method for automatically detecting meals in free living conditions, using the inertial data (acceleration and orientation velocity) from commercially available smartwatches. The proposed method operates in two steps. In the first step we process the raw inertial signals using an End-to-End Neural Network with the purpose of detecting the bite events throughout the recording. During the next step, we process the resulting bite detections using signal processing algorithms to obtain the final meal start and end timestamp estimates. Evaluation results obtained from our Leave One Subject Out experiments using our publicly available FIC and FreeFIC datasets, exhibit encouraging results by achieving an F1/Average Jaccard Index of 0.894/0.804.},
keywords = {Accelerometers, Artificial neural networks, Detection algorithms, Feature extraction, Gyroscopes, Lenses, Monitoring},
pubstate = {published},
tppubtype = {inproceedings}
}
Automated and objective monitoring of eating behavior has received the attention of both the research community and the industry over the past few years. In this paper we present a method for automatically detecting meals in free living conditions, using the inertial data (acceleration and orientation velocity) from commercially available smartwatches. The proposed method operates in two steps. In the first step we process the raw inertial signals using an End-to-End Neural Network with the purpose of detecting the bite events throughout the recording. During the next step, we process the resulting bite detections using signal processing algorithms to obtain the final meal start and end timestamp estimates. Evaluation results obtained from our Leave One Subject Out experiments using our publicly available FIC and FreeFIC datasets, exhibit encouraging results by achieving an F1/Average Jaccard Index of 0.894/0.804.