Pietro Grandinetti
This is a short article that illustrates some things I discovered while working on Garmin’s workout data.
My readers know that I am working on an application that computes what I feel is missing in the Garmin Connect app, and what I think shouldn’t be a paid feature in other apps. For instance, distance covered in each HR zone. Or post-workout manual interval analysis. Or the percentiles of all metrics.
I have run into an interesting behavior of the recording feature of the watch. There are two options available in most Garmin watches: Smart Recording and Every 1 Second Recording. The former is the default, and, if selected, the watch will save on file only some of the data points that it receives. It will still receive data from sensors every second, but only some are written down and stored in the activity file. The Every 1 Second mode, instead, stores them all.
The Smart Recording feature is supposed to save storage space and, in minor part, the battery of the device. It doesn’t have a good reputation online though, and I also don’t recommend it, especially as a result of the analysis illustrated in the remainder of this article.
Before moving on, here are some pages I found useful to understand more about these features:
Running data analysis with Smart Recording
I am working on my app using files from my own activities. For instance, an activity on May 30 that was a AnT on-road estimation test: 4.1 km warm-up Z1/Z2; activation routine of approximately 20 minutes, while the watch was paused; 20 minutes time trial (all out); 4.4 km slow jogging home.
This type of workout present some tricky edge cases when working with the data:
- The total workout time is 1:15:47 (total seconds: 3,707).
- However, the first recorded point was at 06:06:14 AM and the last was at 07:32:51 AM. Hence there are 5,197 seconds between the first and the last recorded points, much more than the effective workout time. This is due to the watch being paused during the activation routine.
On top of that, I had my watch on with the Smart Recording mode. That was a lucky mistake: It’s not a good feature to use, but if I hadn’t then I wouldn’t have found out what follows.
I converted the Garmin TCX file into a spreadsheet with my app, and analyzed it with some formulas in Google Sheets.
I immediately spotted something odd. The Garmin Connect app gives 154 as average HR over the entire workout. But when I compute the average of the HR column in the spreadsheet I get 157.13.
A difference of 3 bpm for the average value is a lot. Also, and most important, this is raw data analysis, hence there shouldn’t be any difference at all!
I figured the problem is caused by the Smart Recording feature. The watch computes the overall workout statistics, like the HR average, using data from every second. But then it writes onto the file only some of the data points, selecting them in a “smart” way.
The result is not smart at all: I get a spreadsheet of values that don’t match what the Garmin Connect app says.
Hence the main takeaway message here is the following: Disable the Smart Recording mode. Use Every 1 second instead.
Anyway, I wanted to get my app working even for a workout recorded _Smart_ly, so I started playing with the data to figure out the interpolation that Smart Recording does and see if I could reconstruct the correct values.
In the beginning I thought, Maybe it writes onto the file an approximation of the values that it discards. Like a weighted average. For instance, if the first data point is at 00:00 with HR equal to 140, and the next is at 00:05 with HR equal to 145, maybe—I thought—that second value means that during the missing 4 seconds the average HR was 145.
So I modified my app to compute the weighted average of the HR column, where the weights are the number of missing seconds between two recorded data points.
However, with so doing I got a value of 151.07, which is far from the 154 that Garmin Connect shows.
I went back and read everything I could find about Smart Recording online. Garmin doesn’t share technical details, but according to their site, the written data points are actual data points. No fancy interpolation or other stuff. The missing data points are just discarded, not interpolated.
Back I was at square one, but reading a bit more there’s a note in their site that says that a data point is recorded in Smart mode when it deviates noticeably from the previous recorded point. In other words, if anything has changed considerably, not only HR, also GPS coordinates, pace, etc.
Reading between the lines, that means that the missing data points would have been similar, if not equal, to the last recorded points.
So I modified my app to interpolate the missing points equal to the last available data point, and finally got an average HR equal to 154.33—exactly equal to what Garmin Connect says when taken as a whole integer.
I then tested it with a few more activity files, some with Smart Recording mode and some other with Every 1 Second mode, and it seems to work well in all cases.
There was one last pitfall. When the watch is paused, like I did on that during activation routines, the interpolation should not be done. That would add, for example, 1,200 data points (for 20 minutes) with HR value equal to the one recorded before pausing the watch. That would send the average HR through the roof.
As a side note, I think stopping the watch is a mistake in general. I should have kept it on even during the activation routine. I could have just made it another lap later. In my app I can discard timeframes from the analysis.
I analyzed my own data more carefully and noticed that Smart Recording never waits more than 6 seconds to write down one more data point. So I changed my app to not do any interpolation if the distance in seconds between two recorded points is more than 8 seconds, because that very likely means that the watch was paused. It’s not bulletproof, but it seems to work well for now.
With all of that said, it’s better to keep the watch on Every 1 Second.