Enhancing Food Intake Tracking in Long-term Care With Automated Food Imaging and Nutrient Intake Tracking (AFINI-T) Technology: Validation and Feasibility Assessment

Kaylen Pfisterer; Robert Amelard; Jennifer Boger; Heather Keller; Audrey Chung; Alexander Wong

doi:10.2196/37590

Enhancing Food Intake Tracking in Long-term Care With Automated Food Imaging and Nutrient Intake Tracking (AFINI-T) Technology: Validation and Feasibility Assessment

JMIR Aging. 2022 Nov 17;5(4):e37590. doi: 10.2196/37590.

Authors

Kaylen Pfisterer^#^{1

2

3

4}, Robert Amelard^#^{3

5}, Jennifer Boger^{1

3

5}, Heather Keller^{3

6}, Audrey Chung^{1

2}, Alexander Wong^{1

2

3}

Affiliations

¹ Department of Systems Design Engineering, University of Waterloo, Waterloo, ON, Canada.
² Waterloo AI Institute, Waterloo, ON, Canada.
³ Schlegel-University of Waterloo Institute for Aging, Waterloo, ON, Canada.
⁴ Centre for Digital Therapeutics, Techna Institute, University Health Network, Toronto, ON, Canada.
⁵ KITE-Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada.
⁶ Department of Kinesiology and Health Sciences, University of Waterloo, Waterloo, ON, Canada.

^# Contributed equally.

PMID: 36394940
PMCID: PMC9716425
DOI: 10.2196/37590

Abstract

Background: Half of long-term care (LTC) residents are malnourished, leading to increased hospitalization, mortality, and morbidity, with low quality of life. Current tracking methods are subjective and time-consuming.

Objective: This paper presented the automated food imaging and nutrient intake tracking technology designed for LTC.

Methods: A needs assessment was conducted with 21 participating staff across 12 LTC and retirement homes. We created 2 simulated LTC intake data sets comprising modified (664/1039, 63.91% plates) and regular (375/1039, 36.09% plates) texture foods. Overhead red-green-blue-depth images of plated foods were acquired, and foods were segmented using a pretrained food segmentation network. We trained a novel convolutional autoencoder food feature extractor network using an augmented UNIMIB2016 food data set. A meal-specific food classifier was appended to the feature extractor and tested on our simulated LTC food intake data sets. Food intake (percentage) was estimated as the differential volume between classified full portion and leftover plates.

Results: The needs assessment yielded 13 nutrients of interest, requirement for objectivity and repeatability, and account for real-world environmental constraints. For 12 meal scenarios with up to 15 classes each, the top-1 classification accuracy was 88.9%, with mean intake error of -0.4 (SD 36.7) mL. Nutrient intake estimation by volume was strongly linearly correlated with nutrient estimates from mass (r²=0.92-0.99), with good agreement between methods (σ=-2.7 to -0.01; 0 within each of the limits of agreement).

Conclusions: The automated food imaging and nutrient intake tracking approach is a deep learning-powered computational nutrient sensing system that appears to be feasible (validated accuracy against gold-standard weighed food method, positive end user engagement) and may provide a novel means for more accurate and objective tracking of LTC residents' food intake to support and prevent malnutrition tracking strategies.

Keywords: aging; automated nutrient intake; collaborative design; convolutional neural network; deep learning; depth imaging; food classification; food intake; food segmentation; long-term care.

©Kaylen Pfisterer, Robert Amelard, Jennifer Boger, Heather Keller, Audrey Chung, Alexander Wong. Originally published in JMIR Aging (https://aging.jmir.org), 17.11.2022.