Billions of people use fitness trackers, smartwatches, and all sorts of hand-worn devices that track fitness. According to some estimates, over 1 in 5 Americans use such a device — and the figure is even higher in other countries. But most people wrongly assume these trackers are very precise. In fact, the mean error of commercial smartwatches that use heart rate is close to 40%. Now, researchers have found a new way to reduce that error: by using a thermal camera.

Humans obtain calories from the food and drinks they consume, which are essential for providing the energy required for all bodily functions and activities. This energy comes from the macronutrients found in food which the body breaks down during nutrition.

Keeping track of calories

Smartwatches estimate the number of calories burned by using a combination of data inputs and algorithms. They typically monitor the wearer’s physical activity level through built-in sensors that detect movement and heart rate. By analyzing this data, along with the user’s personal information (such as weight, height, age, and gender), smartwatches can estimate total energy expenditure.

But it’s still just an estimate.

If you use a smartwatch that keeps track of your fitness, you probably make decisions based on that estimate. Mayank Goel, an associate professor at Carnegie Mellon University, says you’re probably using imprecise data.

“When people see these numbers, they make changes in their behavior and that can be troublesome if the numbers are wrong,” Goel said. “That is a huge problem,” he added.

In a medical context, researchers can monitor calories accurately with a calorimeter. A calorimeter uses a combination of heart rate and respiration tracking (especially looking at CO2 in exhaled breaths). But there’s no solution to measuring CO2 with a simple wearable device. So instead, Goel looked at something else: a thermal camera.

A camera in your phone

A thermal camera could assist in calorie estimation by measuring the heat emitted from the body. Body heat reflects metabolic rate changes due to physical activity or the thermic effect of food. When you engage in physical activity or after consuming a meal, your body’s metabolic rate increases to convert food into energy, producing heat in the process. A thermal camera, by detecting variations in body temperature associated with these metabolic changes, could potentially provide data on how much energy the body is expending.

This information, combined with other data such as the type and duration of activity or meal composition, could enhance the accuracy of calorie burn estimates. While working on a different project, another Cornell information sciences major Maite Sadeh noted that a thermal camera can detect exhalations of hot air. Goel and colleagues then recognised that exhalation results in evaporation around the lips and nostrils — and both these signals can be captured by a thermal camera.

“We lose the ability to measure the concentration of oxygen and CO2, but we gained temperature measurements,” Goel said. 

The combination of existing smartwatch data, combined with thermal data and analyzed with machine learning, enabled the researchers to develop a better system for calculating calories. They asked 54 participants to cycle or run for 15 minutes. When compared to a clinical calorimeter, the new system had an error of only 5.8%.

It gets even better. The thermal camera used in the study isn’t anything special — it’s already available for $45 or less and can be integrated into smartphones. Researchers are working on incorporating even lower-cost cameras.

Towards better decisions

This technology shows great promise, but there are still significant limitations. For starters, environmental conditions, such as ambient temperature and humidity, might affect the accuracy of thermal imaging in capturing metabolic rate changes and temperature readings. The methodology also assumes that changes in body temperature due to physical activity or food intake are uniform and significant enough to be accurately captured by thermal cameras. This may not always be the case due to individual physiological differences.

Lastly, they must consider the system’s performance outside of controlled settings. It’s effectiveness in real-world scenarios where activities are more varied and unpredictable, remains to be thoroughly tested. These challenges highlight the need for further research and development to refine this innovative approach to enhancing the accuracy of calorie burn estimates in fitness trackers.

Nevertheless, this is a promising new method for accurate and reliable calorie burn estimation, bridging the gap between the rudimentary estimates provided by current fitness trackers and the precise measurements achievable in clinical settings. The innovative use of thermal imaging can bring us one step forward to improved health outcomes, more tailored fitness routines, and a better understanding of how everyday activities and nutrition affect our bodies.

The study has been published in the Proceedings of the ACM Mobile, Wearable and Ubiquitous Technologies and will be presented in October at the UbiComp 2024 conference in Melbourne, Australia.