At my recent trip to Lake George I was able to collect some interesting data regarding my new Garmin 255 watch.
I had 15 sessions and for each one I wore my Garmin 255 underhanded on my left wrist, the Mini Motion on my left upper arm and the ESP unit just below the elbow on my right arm.

Mini Motion set to log at 5 hz. Results reported from GPSSpeedreader
ESP logger set to 5 hz (This unit has been approved for posting to GPSTC).Results reported from GPSSpeedreader.
Garmin 255 set to use all constellations with multi frequency turned on. Results reported from Garmin Connect using the latest V4 App

To keep things simple I have only looked at the peak 2 sec speed recorded in each session. This is well known to be the most difficult category to measure accurately.

Here are the results: All data in Knots.

The Motion-Garmin column shows that in all but two cases the Motion gives a higher result. This is to be expected as the Motion is sampling at 5 times a second so should always find a higher peak.
The Motion-ESP result has a similar number of positive and negative results as they are both sampling at 5hz

In the graph below I have sorted the differences by size. If the Garmin line was lowered by 0.05 knots the range of errors would be practically the same. Perhaps this 0.05 kts can be put down to the advantage of 5hz data over 1 hz data.


Conclusions:
The new Garmin 255 watch when set up to use all constellations and multi frequency can produce very accurate results. Much better than I would have believed possible. I would often find my old Fenix 5 to be 0.2 to 0.5 kts away from the Motion.
I highly recommend the Garmin 255, the screen is significantly larger than the 245 (Music) and the accuracy has massively improved. On sale they can be had for under Au$400

Of course being accurate is only half the requirement, proving it is another thing. Garmin watches still do not report accuracy data.

Hi
Is this the Garmin model you are referring to
Thanks TK

Yes that is the one. They were selling them at $379 before Christmas. But $449 is still a good deal.

Select to expand quote

tbwonder said..
Yes that is the one. They were selling them at $379 before Christmas. But $449 is still a good deal.

Thanks for that
I might line one up for my next birthday

Keep an eye on the Garmin site as they have regular sales on most models including the expensive Fenix series. I got my 745 for half price.

Interesting data. So is it your impression that the 255 seems more accurate than older Garmin watches?

The most common problem with spikes is linked to crashes - anything from just a few knots to > 100 knots. That's with all kinds of u-blox units. The spikes are generally not a problem, since they can easily be filtered using the error estimates from the u-blox chips.

It would be interesting to see how the watch does in deep water crashes. Lake George data from expert sailors are a bit limited for this, since crashes where the GPS gets submerged for a while are rare. Wingfoil sessions from learners with lots of crashes are a lot better for this .

One question about the 255 (and other Garmins) is how waterproof the watch is. According to Garmin's web site, the 5 ATM rating is less than what they suggest for "high speed watersports" (10 ATM). How well do the 5 ATM rated Garmins hold up when crashing often and/or at high speed?

Select to expand quote

boardsurfr said..
Interesting data. So is it your impression that the 255 seems more accurate than older Garmin watches?

I had a quick glance at the data for one of the sessions (Garmin, Motion, ESP). It's definitely the best results that I've seen from any Garmin.

+1 for everything you said about spikes.

Did a bit of digging on the Garmin site. The Garmin 255 watch seems to be the lowest cost multi-band GNSS watch they have, at about $250 US, vs. $450+ US for other models. That's pretty relevant for those of us who can't just drop $700+ on a more expensive model.

Andrew's data indicate that the accuracy can be better than for other watches he reported on in the past. The differences in this data set are similar what is typical for using two u-blox GPS units (Motions and/or ESP). This looks quite promising, and call for reproducing the test.

The question is if the multi-band GNSS is responsible for the better accuracy. The claims on the Garmin website regarding better positional accuracy are just small (2 m vs. 3 m), and I did not see any statements about speed accuracy. However, the use of two frequencies should allow for better identification of "bad" satellite signals, for example from reflection. This should then improve speed accuracy. It may well also help in the worst-case scenario of spikes due to crashes - having data from two frequencies should make it pretty obvious if a signal is heavily distorted because the watch is in the water. Other watches like the Fenix that use multi-band data do not seem to show the same improvement in speed accuracy (although the folks at Coros may debate that). But the actual results will depend a lot on the hardware and firmware used, and perhaps Garmin has made some improvements with the 255. More data and analysis should tell.

Select to expand quote

boardsurfr said..
One question about the 255 (and other Garmins) is how waterproof the watch is. According to Garmin's web site, the 5 ATM rating is less than what they suggest for "high speed watersports" (10 ATM). How well do the 5 ATM rated Garmins hold up when crashing often and/or at high speed?

I have been using Garmin watches of various types (including the base level types with lower water depth rating) for many years and I use them for windsurfing and other watersports. I may not go as fast as some but I have had significant catapults. I have never had a Garmin watch ever get water ingress or any other related issue. The watch I used the longest was the Vivoactive 3 which was used for 6 years and still worked as well the day I stopped using it as when I got it (including great battery life). I just upgraded to a 965 due to more features and OLED display and my aging eyes struggling to read the small ambient backlit LED screen on the VA3 in my lap swimming in a dark indoor pool. I still use it from time to time,

Yes I believe the Garmin 255 is much better than older Garmin watches. Is that because of the multi frequency? Or some other improvement?

I have a friend who wings and has a 255 so I will try and get some tracks from him.

Regarding waterproofing, I didn't bother to look at the specs for the 255 vs Fenix 7.
Garmin watches are used by most of my friends and have been for the last 10 years or so. I have never heard of a single failure due to water ingress. I am on my 4th Garmin watch since 2009. They all still work. Although the one at the bottom of the lake probably has a flat battery by now.

Yes the 255 is the cheapest multi frequency watch on the market.

Andrew, do you know if the 255 uses different filtering for different activities? My wife bought one after I told her about your report. Based on the advice in your documentation to "get to know your watch", she made a "Wing" activity which she based on the SUP activity on the watch. She used it today in a speedsurfing session, and the results were a bit surprising - top 2 seconds were 1 knot lower than what 2 ESP32 boom units showed (which were in close agreement), 5x10 was about 1/2 knot lower, alphas about a 1/2 to 1 knot higher:

The speeds for the top 2 second run show evidence of heavy filtering in the Garmin data (blue curve):


One peculiar behavior is the sudden speed drop at the end of a run - it often drops from > 10 knots to 0 in one second, which is clearly a data processing artifact. The ESPs, but also the positional speeds from the 255, show deceleration over several seconds instead - here's an example:

Most stops look quite similar. This was in choppy, hip deep water (no weeds to stop the chop ).

So the question is if this filtering behavior of the watch can be change, or perhaps differs for different activities. For a GPS worn on the wrist, it would make sense to use heavier filters for activities with a lot of arm movement (swimming, running, SUP), and lighter filters for other activities (like biking or windsurfing).

The multi-second delay in acceleration and deceleration phases relative to the ESP data is an indicator that should be present even in smooth-water sessions.

No I am not aware of different filtering for different activities. But perhaps there is?
I think I based my activities on the standard "Run" activity.

First thing to check is that you have the satellites setting set to "All + Multi-band". This setting is activity dependent.

All my data shows that the Garmin (Blue) lags the Motion/ESP Devices by about 2 seconds. This is a little odd, but as it is consistent it doesn't cause any trouble. I do not see the issue you report of the sudden drop from 10 kts to zero.

The delay, which indicates how much filtering was done, looks quite different for your data. In the one session I have, the delay is consistently about 8 seconds for increasing speeds, 10 seconds for peaks, and 5 seconds for speed drops (satellite settings were for all systems + multiband in both activity and in the generals settings).
In the example below, the selected region in 10.4 seconds:

It is also quite obvious that there is a lot more smoothing going on than in your data. There are two possible reasons that come to mind: (1) different filtering for different activities, and (2) auto-adjusting filters, depending on the actual data. The data from the chop session have a lot more ups & downs than your data. Your instructions state to base activities on "other", so I'll check if that makes a difference in a test drive, compared to the app that's based on SUP. I have to drive to the car dealer to swap rental cars today, anyway. The van is now in for repairs (wheel bearing) for the 13th day, and they still cannot tell me when it will be done. Doing the test makes this drive a bit less pointless...

Mystery solved! Here's data from a test drive with a "SUP" activity:




It shows a lot of smoothing in the blue 255 traces, while the 2 ESP traces are almost identical. The selected region is 8.4 seconds long, similar to what we saw on the water.

Just basing the activity on "Other" gives a very different image:


The delay is closer to the 2 seconds that Andrew reports. Peaks and troughs are reproduced quite well. It certainly makes sense to adjust the smoothing based on the activity for a GPS watch. Activities like running, swimming, or SUPing where the arm moves forward and backward would otherwise show speed varying from close to 0 to twice the "real" speed. Runners want to know the speed they are running, not the speed their arm moves, so they need filters long enough that multiple arm movements can be evened out.

So for using the Garmin watches, it is very important which activity is used to record. Activities that involve a lot of arm movements (like SUP, rowing, walking, etc) should be avoided. "Other-other" seems to work well enough, but it's possible that there are activities that may be even better suited. For GPSTC, the numbers to look at are alphas and 2 seconds.

There's an old post on the Garmin forum that describes filtering differences based on activity, and states cycling has little or no filtering, while running has a lot of filtering: forums.garmin.com/developer/connect-iq/f/discussion/1421/getactivityinfo-current-speed-filtering

Seems that Garmin has not changed the filtering much in the last 9 years. Based on the old forum post above, we tried biking, and this is what the speeds look like:

Smoothing seems to be either absent or minimal; the observed delay is in the range of 0.2-0.4 seconds. All peaks and valleys seen in the ESP data are nicely reproduced. Just glancing at the data, the 2 second speeds reported by the Garmin seem to be within about 0.05 knots of those reported by the ESPs, and differences between the ESPs are similar to the differences to the watch. Quite encouraging, can't wait to get some windy days to do more tests.

That's a good find Peter.
Before we all rush to change our Activity base to "Bike" rather than "Other", it would be worth doing some testing to prove that this actually gives better results.
The results I posted at the start of this post all have the 2 second delay. It is possible that this filtering has actually helped.

I certainly agree that more testing is a good idea. A filter that seems to have a time constant near 2 seconds should be quite appropriate for 2 second (and longer) results. The effect on "max speed" may be larger, but one could argue that appropriately filtered data are at least as likely to be correct as unfiltered data which contain more noise. Testing should include some sessions in real chop, not just on Lake George or Albany on what's called "choppy" there.

Besides 2 seconds, the category to look is alpha, since filtering can smooth out dips, which can be quite real in case of alphas. The effect was quite noticeable when using a heavily filtered activity setting (SUP). The main problem is that the effect led to higher alphas. Errors that understate speeds are easier to tolerate from a competition perspective than errors that overstate speeds. It's easy to speculate what the outcome is, but data from tests are what really matters.

Just checked now and the 255 watch is on special for $404 on amazon.AU

Select to expand quote

boardsurfr said..
Smoothing seems to be either absent or minimal; the observed delay is in the range of 0.2-0.4 seconds.

This is excellent testing and observation.

I have to correct the delay numbers for the 255 - it's closer to 2 seconds in chop (with activities based on biking). Filtering does indeed seem to be minimal, though. Here's a short section from a windsurf session in chop, with the data from the 255 (in blue) moved by 2 seconds compared to 2 boom-mounted ESP loggers:

The 255 reproduces all features very nicely. Top speeds in all categories, including 2 seconds and alpha, are very close (~ 0.1 knots). The steps in the blue trace above are an artifact of drawing 1 Hz and 5 Hz traces in the same graph (the 1-second values are reused 4 times in the graph); that's something I plan to fix.
The data look very promising, better than anything I've seen from watches or phones so far. Dual frequency GPS can use the second frequency to eliminate spikes and give better accuracy, so they definitely deserve a closer explanation. I've seen examples where the u-blox GPS data gave fake (dead-reckoning-like) data in crashes that did not trigger the filters, while the 255 showed speeds going down right away:


Interestingly, the u-blox fantasy extends to both the speeds and the positions. It happens for about 2 seconds before a filter set to 1 knot sAcc would trigger. The u-blox position data then show a big (22 m) jump backwards when satellite reception is recovered. This is an example where dual-band GPS data are actually more accurate than single-band data with error estimates and filtering. We'll need to examine more comparison data, but this is promising indeed.

Great stuff Peter!
I think this may set the cat amongst the pigeons.

I think most of the pigeons flew away some time ago

So following Andrew's installation directions of using "other" activity as a base for uploading GPSTC V4 data field is still the way to go?

I have continued to use the "Other" activity. I have just looked at the comparisons for my 16 foil sessions in March when I wore a Mini Motion and the 255.
Just looking at the 2 Sec peak:
8 of the results are within 0.05 knots.
4 within 0.1 and the final 4 between 0.1 and 0.17

We've had just a few windsurf sessions comparing the 255 to ESP loggers, and never saw any large deviations with the 255. The biggest deviation we noticed was a ~ 0.5 knot difference in a 2-second speed between the two boom ESP loggers. That's more than usual, but not unheard of.

This is very different from some Coros data a team mate produced during the same session. The Coros data understated the 2-second top speed by about a knot or a knot and a half. This was clearly due to heavy filtering in the Coros data; the top speed was in a gust on choppy water, so it really was just a short peak, with lower speeds before and after. The delay seen in the Coros data was in the 4 or 5 second range, so the filter just flattened out the top speed. The Coros has plenty of big claims, but the Garmin 255 has the better accuracy, at least if used with activities that don't filter much.

Select to expand quote

boardsurfr said..
This is very different from some Coros data a team mate produced during the same session. The Coros data understated the 2-second top speed by about a knot or a knot and a half. This was clearly due to heavy filtering in the Coros data; the top speed was in a gust on choppy water, so it really was just a short peak, with lower speeds before and after. The delay seen in the Coros data was in the 4 or 5 second range, so the filter just flattened out the top speed. The Coros has plenty of big claims, but the Garmin 255 has the better accuracy, at least if used with activities that don't filter much.

The above clearly shows very promising results from the 255 which is great news.

Can you confirm which model of COROS was being used? The Sony-based VERTIX and APEX Pro will often understate the 2-second top speeds, whereas the Airoha-based VERTIX 2, APEX 2, APEX 2 Pro will usually be more in line with the motion. As an aside, issues that I've observed in the Sony-based COROS watches (excessive smoothing, repeated speeds, huge spikes when signal is lost) also affect Garmin's using the same family of chipset (e.g. Fenix 6). The Airoha models (same family as newer Garmin watches) have generally been performing a lot better, but not as well as I've seen in the charts above.

I'll look forward to a future version of speedreader which removes the step artefacts. :)

Select to expand quote

K888 said..
Can you confirm which model of COROS was being used? The Sony-based VERTIX and APEX Pro will often understate the 2-second top speeds, whereas the Airoha-based VERTIX 2, APEX 2, APEX 2 Pro will usually be more in line with the motion.

According to the FIT file contents, the watch is a Coros Apex Pro.Interesting to hear it's largely the chip, and that the Sony chip shares the "huge spikes when signal is lost" problem with the u-blox chips.

Select to expand quote

K888 said..
I'll look forward to a future version of speedreader which removes the step artefacts. :)

The GPS analysis software historically has used interpolation to get better results for 1 hz data (I'm certain GPSResults did, and I think there were several options in GPSAR). I'm tempted to simply interpolate 1 hz data so they look like 5 hz data. In theory, interpolation should improve accuracy (which is why GPSResults etc did it), at least when the speed change at the start and end of a run differs significantly.

I may have been a little sloppy with my wording regarding the Sony chipset. That family is very prone to spikes when resuming logging after loss of signal (e.g. submersion after a fall / coming ashore). Combined with the tendency for the exact same speed to be reported for several seconds, these spikes often appear in the 2s rankings on GP3S because they slip through the filters (namely the acceleration filter because the speed is repeated). The charts below show a 50+ knot spike whilst I was de-rigging, seconds putting my arm inside a metal trailor.



I've seen the exact same behavior (spikes typically up to 60-something knots and repeating speeds) from the Garmin Fenix 6 which uses the same chipset family, both a wing foiler coming ashore and a windsurfer during a crash. The Sony spikes are somewhat different in nature to what I've seen on the u-blox. They tend to be short-lived, followed by another period without logging which is a little different to the u-blox thing where the speed is locked and the position changes are uniform. As an aside, I'm 99% sure that COROS aren't logging the number of satellites correctly. It looks to me like they are logging the number of visible GPS satellites (not even all of the GNSS constellations) from GSV instead of the number of active satellites from GSA. I've got lots of tracks that include an obvious loss of signal (followed by a spike) but the satellite numbers never drop during that period. The number of satellites being reported never exceeds what is possible from GPS alone, which is one of the things that leads me to say what I did about GSA. Whatever is being logged, it is misleading to say the least.

IIRC, GPSResults used to use cubic-spline interpolation when calculating 10s results from 1 Hz devices (perhaps it still does). You could use cubic-spline interploation for your visuals, or just use linear interpolation to make the steps disappear. Personally, I'd probably just implement linear interpolation as that's essentially what we see when the data is loaded in isolation (regardless of frequency) and makes it easier to interpret the raw data.

Interesting stuff, I was about to order until I saw the 255 was only 5atm water resistant. I guess if you keep it covered high speed crashes shouldn't be an issue.
www.garmin.com/en-AU/legal/waterrating-definitions/

Select to expand quote

airsail said..
Interesting stuff, I was about to order until I saw the 255 was only 5atm water resistant. I guess if you keep it covered high speed crashes shouldn't be an issue.

We had the same concern before ordering the 255, but others who had been using various Garmins for years (including watches rated 5 m) assured us that they never had problems. So far, so good, but only a dozen or so sessions so far. Note that Garmin cautions against pressing buttons under water.

Garmin has been selling watches that are frequently used in water (e.g. triathlons) for many years, and they seem to handle warranty claims decently. This is very different from the Locosys watch. Locosys had zero experience with watches, and apparently used outside sources for the watch housing and armband. The Locosys watch definitely did get water inside if you ever pressed a button under water, and eventually even if you just pressed with wet hands, which then later stopped the watches from working ... if you did not loose them because the band broke first. The handling of warranty claims by Locosys for the watch was ridiculous, and plainly illegal in many countries. The claims regarding water resistance may have been similar to the Garmin claims, but they overpromised, while Garmin appears to be quite conservative in their claims.