Exclusive: Inside the Apple Watch’s dance tracking algorithms

When Apple said at this year’s Worldwide Developers Conference (WWDC) that it can tell you how many calories you spent while dancing, it piqued our interest. Traditional workouts are repetitive, something algorithms excel at tracking. Compared to that, a dance is as erratic as a Twitter trend.

Apple added four new workout types to the Activity app – Core Training, Dance, Functional Strength Training and Cooldown.

With watchOS 7, you can pick any of these workouts and get the calorie credit for them instead of having to pick “Other” and getting a generic calorie count. The Apple Watch does a lot of things, but picking the Other option because we can’t quite tell which workout covers what we’re doing is something we’re all guilty of.

If your workout is picked accurately, the Apple Watch tracks what you are doing accurately too. “The Workout app on Apple Watch is one of the most used apps, and each of the 20 top-level workout types uses a smartly-tuned algorithm that understands the workout you are doing to give you the most accurate metrics for each one,” Julz Arney from Apple’s Fitness Technologies team told HT Tech.

The Dance workout has been tested and taught with four dance styles – Bollywood, cardio dance, hip-hop and Latin. Apple is specifically targeting dance in a workout setting, not social partner dancing, ballroom dancing or choreography.

“Creating an algorithm to give accurate calorie credit for dance was a real challenge for a few reasons,” Arney said. “Movements in dance aren’t always repetitive or predictable like they are in walking and running, so what the sensors can detect strictly from the wrist is not necessarily the whole story,” she added. For dance, the watch needs to detect how your “whole body is moving”.

The key difference is the lack of repetitive motion. Walking, running, swimming or even lifting weights have a single, repetitive motion signature, like the arm swing, that is more straightforward for the sensors to detect. For instance, your arm swings faster the faster you walk. Dance moves are made up of any combination of arm and leg movements.

Apple had to specifically look at dance moves that use just the arms, just the lower body and moves that use the entire body, separately. The algorithms use data like speed, intensity and pattern of motion coming from the accelerometer and gyroscope on the watch to determine something called “work rate”. This is what determines how much work you have actually done during a session.

For example, when you are swimming, the Workout app looks at the gyroscope and accelerometer data to identify the stroke style and estimate the stroke rate – that’s the work rate. For something like indoor cycling, where your body isn’t moving through space and there is very little movement at the wrist, it uses heart rate only. For dance, both the work rate and the heart rate had to be used together to predict intensity and get accurate calorie credit by using both the heart rate sensor and the motion sensors.

So how exactly did Apple track this?

To find out exactly how hard you work when you dance, Apple put Apple put mobile metabolic carts on dancers back. These track your body’s metabolism while you’re performing an activity. They’re often used by athletes and even for health purposes at times, to detect breathing response etc.

This allowed the company to measure true calorie expenditure in a dance workout. To allow the research team to understand how movement at the wrist related to movement at the hip, the dancers had a phone attached to their hip. Additionally, the dancers wore a watch on their wrist and a heart rate monitor chest strap.

Collecting heart rate data allowed researchers to compare the calorie credit from heart rate data against the true calorie burn from the backpacks. This helped create an algorithmic model for the Watch.

Apple is using “advanced sensor fusion”, which combines data from both the accelerometer and the gyroscope, to detect the difference between dancing with just your arms, just your lower body, or when you dance with your entire body. The heart rate data is added to double check, so that you don’t get extra calorie credit for simply moving your arms, or get too little for moving your whole body.

The whole point of picking the right workout type is vital to get the right calorie credit and close those circles. But most importantly, it lets you know if and how you need to change your workouts to ace your fitness goals.