I happened to drive around in my van to test a new $30 phone for use with GPSLogit, and ended up coming to some insights about the spikes in 5 Hz GW-52 data. Bottom line is that there is a lot of additional noise when recording at 5 Hz when compared to 1 Hz data from GT-31s. Not all of the speed variations we see at the 200 ms level is real - maybe very little of it is. So the actual improvement in accuracy that is achieved by recording at 5 Hz rather than 1 Hz is definitely smaller than the theoretically possible factor of 2.2. I attached an image that illustrates the measurement noise in 5 Hz data.





The full report is at boardsurfr.blogspot.com/2016/07/dead-toys-and-lots-of-noise.html (scroll down a bit to get to the GW-52 stuff).

Precisely why I asked locosys to display 1hz average max speed instead of 5hz instantaneous max speed. I'm not interested in a noise spike, that gives you a very false impression of what the 2s average will be when you crunch the numbers.
Looks like this isn't going to happen.

What interest me is the nature of the noise.
At one stage I was testing 2 GW52s with them side by side the two signals showed very similar noise, in phase with each other. When I had one unit on my head and one on my upper arm, the signals looked similar, but the phase between them varied, about 80% was 180deg out of phase.

Select to expand quote

decrepit said..
Precisely why I asked locosys to display 1hz average max speed instead of 5hz instantaneous max speed. I'm not interested in a noise spike, that gives you a very false impression of what the 2s average will be when you crunch the numbers.
Looks like this isn't going to happen.

What interest me is the nature of the noise.
At one stage I was testing 2 GW52s with them side by side the two signals showed very similar noise, in phase with each other. When I had one unit on my head and one on my upper arm, the signals looked similar, but the phase between them varied, about 80% was 180deg out of phase.

If you want 2-sec, why not ask for 2-sec? ie: The point of max-speed is specifically to have the max-speed.

In any case, it can be turned off.

Select to expand quote

mathew said..
>>>>

If you want 2-sec, why not ask for 2-sec? ie: The point of max-speed is specifically to have the max-speed.

In any case, it can be turned off.

But that's just the point it isn't max speed it's max noise. Well max speed plus noise I guess.

And I'm not with you on "it can be turned off". What can be turned off how?

It's not 'noise', it's DATA.

The speed errors are random and therefore more data leads to far better accuracy over many points.

We are not interested in 0.2 knots speeds. (we are looking at the Forest, not the individual trees)

We do not give much serious credibility to the accuracy of even 2 second speeds.

We are interested in longer periods of 10 seconds and more. In all situations, more data is far better.

The 1Hz data from a GT-31 is logically derived from an average of multiple readings over that second. We just don't know how many and the method used. Is it any wonder that agrees closely with 1 second averaged data from the GW-52, which is an average of the 5Hz data.

It is interesting to note that speed graph traces from the 10Hz Trimble survey grade GPS (cm accuracy) from Maquarrie Innovation and Sailrocket show the same sort of sawtooth appearance. It is, for all intents and purposes, irrelevant.

Select to expand quote

sailquik said..

It is interesting to note that speed graph traces from the 10Hz Trimble survey grade GPS (cm accuracy) from Maquarrie Innovation and Sailrocket show the same sort of sawtooth appearance. It is, for all intents and purposes, irrelevant.

No, it is not irrelevant. It's simple:
Measured data = real data + noise.

Measure at low frequency with filters => Measured data are close to real data.
Measure at higher frequency with fewer filters => more noise.

Assuming that higher data rates mean higher accuracy is simply wrong.

Theoretically, higher data rates could be better if because allow you to determine which kind of filters or averaging is used in the analysis to eliminate the noise. But that's simply not done in the current analysis software.

Everyone who has been disappointed by seeing his top speed drop between what the GW-52 shows and what you see in the software will have an idea what I am talking about.

As I have said many times before. The one calculation top speed figures given by ANY GPS are highly unreliable. (high error margin)

Even the 2 second figure calculated in software has a high error margin. Getting too puffed up about your 2 second figures from a 1Hz GPS is kidding yourself a bit. That is where the big benefits come from higher Hz.

On the matter of higher Hz I will agree to disagree since I have not seen any actual evidence to the contrary and plenty of evidence in support.

The level of noise in the sampling rate can be directly attributed to the quality of the antenna. The GW52 has a very cheap one and it's reflected in the noise created. The GT31's is a better patch type but the true level of noise is masked by filters and averaging over 1 second. Fitting a higher quality antenna is the key to improved accuracy, here's the result from a helix one fitted to a 18hz U-Blox 8 gps proto. The m/s figure is much smoother. N.B. the 16 sec. time differential is caused by the leap year correction in some gps units.

Yes, GW-52 antenna costs $0.25/0.50
The helix one is $15
Diference between cost optimized and result driven design...

Yes, it is quite possible that the antenna is a limiting factor that increases noise disproportionally at higher rates. But there are other possible factors, too; for example, electromagnetic interference can create problems when going from 1 Hz to 5 Hz (as seen with when trying to adopt the Flysight).

The important thing is to take a close look at the data to see if the underlying assumptions are valid. I did a few test drives with a ublox-7 based GPS dongle and the GW-52 today. Interestingly, the 1 Hz GW-52 data give a higher estimated accuracy than the 5 Hz data:


That's because the error estimates for the individual data points are about 3-5-fold higher in the 5 Hz data. At least in this example, the 1 Hz data are more accurate than the 5 Hz data for 10 second runs. From what I have seen so far, I would not be surprised if that is always the case. BTW, the 10 second segment I selected was from the same stretch of road, and two drives that were about 20 minutes apart.

For the ublox-7 GPS data, the result is quite different. Here, the error estimate is about 2x lower for the 5 Hz data than for the 1 Hz data (+- 0.3-0.5 for 1 Hz, 0.12-0.25 for 5 Hz). That's what would be expected if (a) measuring at the higher rate does not increase the measurement error, and (b) the 1 Hz data are sampled, not averaged (or filtered).

Note that in both cases, we rely on the error estimates given by the instruments. They are using different chips from different manufacturers, so it is very likely that the numbers are computed differently. One chip might use one standard deviation, the other one two or three. We're not comparing apples and oranges, but we may be comparing apples to sets of three apples.

But if we ignore that for the time being, it's cool to see that the sub-$20 USB dongle got similar accuracy as the GW-52. Now I just need to find a cheap phone with USB host mode support...

Select to expand quote

Roo said..
The level of noise in the sampling rate can be directly attributed to the quality of the antenna. The GW52 has a very cheap one and it's reflected in the noise created. The GT31's is a better patch type but the true level of noise is masked by filters and averaging over 1 second. Fitting a higher quality antenna is the key to improved accuracy, here's the result from a helix one fitted to a 18hz U-Blox 8 gps proto. The m/s figure is much smoother. N.B. the 16 sec. time differential is caused by the leap year correction in some gps units.

Interesting numbers, Roo. Looking at the speed change from point to point, it seems the error estimates for the 18 Hz data are quite high (very conservative). What are the accuracy estimates you get for 10 second runs? Have you compared two Ublox prototypes side by side?

Error estimate for 10 sec.

18hz U-Blox was 0.091 GT31 0.259 Run 1
18hz U-Blox was 0.078 GT31 0.533 Run 2
18hz U-Blox was 0.137 GT31 0.414 Run 3
18hz U-Blox was 0.081 GT31 0.268 Run 4

The U-Blox always has higher individual error estimates than the Sirf GPS chipsets. I think the reason is the U-Blox is a 3d estimate versus a 2d estimate for Sirf.

Very nice numbers, Roo. Roughly 4-fold better than the GT-31, in line with what would be expected from bumping the data rate up. I'd guess that a lot of the improvements between the Sirf3 and the ublox8 are necessary to go to an 18-fold higher data rate without adding measurement noise.

Select to expand quote

boardsurfr said..
Interestingly, the 1 Hz GW-52 data give a higher estimated accuracy than the 5 Hz data:

They don't appear to be the same runs. If they are, the speed figures are very different! Did you use two side by side GW-52's?

If so, the results should be the same (for the time period same sample set) as the 1Hz setting output is the average of the 5Hz calculation.

Select to expand quote

Roo said..
Error estimate for 10 sec.

18hz U-Blox was 0.091 GT31 0.259 Run 1
18hz U-Blox was 0.078 GT31 0.533 Run 2
18hz U-Blox was 0.137 GT31 0.414 Run 3
18hz U-Blox was 0.081 GT31 0.268 Run 4

The U-Blox always has higher individual error estimates than the Sirf GPS chipsets. I think the reason is the U-Blox is a 3d estimate versus a 2d estimate for Sirf.

That what I would expect from a well implemented, modern Ublox 18Hz device.

Can you point to any documentation to support your 2D v's 3D theory?

Pretty well covered in the U-Blox data sheets. VACC is the velocity accuracy estimate, velocity being calculated in 3 dimensions, x, y and z. The Sirf calulates SDOP, speed dilution of precision, speed SOG calculated from a scalar measurement.

Select to expand quote

sailquik said..

boardsurfr said..
Interestingly, the 1 Hz GW-52 data give a higher estimated accuracy than the 5 Hz data:

They don't appear to be the same runs. If they are, the speed figures are very different! Did you use two side by side GW-52's?

Yes, they are not from the same runs. I said that in my post. They were runs at the same spot, with the same GW-52, about 20 minutes apart.

Select to expand quote

sailquik said..

If so, the results should be the same (for the time period same sample set) as the 1Hz setting output is the average of the 5Hz calculation.

That seems like a reasonable expectation, so I looked into this some more. When I re-ran the 1 Hz data in GPSResults, I got accuracy estimates that were indeed closer to the 5 Hz data. Why did GPSResults give me different results today? That took me a while to figure out!

It turns out that GPSResults uses different defaults to calculate the error estimates for 1 Hz data and 5 Hz data. That is (a) mathematically questionable, and (b) completely hidden from anyone who uses the Mac version of GPSResults (and not obvious to Windows users, either). But what's worse, Mac users of GPSResults can't even change the settings, since the "Filter Settings" dialog is missing in the Mac version. I had played around with a bunch of different files, including some that had a mix of 1 Hz and 5 Hz data, and that must have somehow switch the setting on the Mac.

For 1 Hz data, GPSResults calculate the estimate error for all results (the +- column) from the average of the error estimates for the individual points. For 5 Hz data, it uses Gaussian error propagation by default (basically dividing the average error by the square root of the number of points). So if you have 1 Hz data where all 10 data points in a run have an accuracy of +- 0.5 knots, GPSResults will give the entire 10-second run an accuracy of +- 0.5 knots. But if you have 5 Hz data where each data point has this accuracy, GPSResults will give the run an accuracy of +- 0.1 knots, making the 5 Hz data appear five times more accurate. But if you actually use the same math for both Hz rates, then the 1 Hz data should have had an accuracy that is much closer (+- 0.16). Note than in reality, the error estimates for single points are much higher for 5 Hz data on the GW-52!

I have put some screen shots and a more detailed explanation on my blog at http://boardsurfr.blogspot.com/2016/07/more-noise-and-errors.html. For fun, here is the comparison of 1 Hz data and 5 Hz data when both are analyzed with the "Average" error propagation setting that is the default for 1 Hz data:


Using this method that GPSResults for some reason picks for 1 Hz data, the 5 Hz accuracy estimates look a lot worse than the 1 Hz estimates.

I think the error rate is due to GW-52 design, I checked files from the following units
- 1hz GT-31
- 10hz Ublox PAM-7, std Ublox antenna
- 18hz Ublox Max-M8q, helix antenna

All three devices have comparable +/- values when you check the values on the same runs.
GT-31/PAM-7 checked 2 tracks in NL : 0.2xx against 0.3xx
GT-31/MAX-8q check 2 tracks on La Franqui : 0.2xx against 0.3xx

Changing value from average to gaussian didn't change the AVG outcome from the 4 tracks.
Used GPS Results 6.147 Windows version.

I don't own a GW-52, can't compare the devices...
My guess is that the watch design was build for the small (chip)antenna and that the only added the big antenna without changing the device design.

Select to expand quote

boardsurfr said..
It turns out that GPSResults uses different defaults to calculate the error estimates for 1 Hz data and 5 Hz data. That is (a) mathematically questionable, and (b) completely hidden from anyone who uses the Mac version of GPSResults (and not obvious to Windows users, either). But what's worse, Mac users of GPSResults can't even change the settings, since the "Filter Settings" dialog is missing in the Mac version. I had played around with a bunch of different files, including some that had a mix of 1 Hz and 5 Hz data, and that must have somehow switch the setting on the Mac.

Must be some type of conspiracy....

Select to expand quote

mathew said..

boardsurfr said..
It turns out that GPSResults uses different defaults to calculate the error estimates for 1 Hz data and 5 Hz data. That is (a) mathematically questionable, and (b) completely hidden from anyone who uses the Mac version of GPSResults (and not obvious to Windows users, either). But what's worse, Mac users of GPSResults can't even change the settings, since the "Filter Settings" dialog is missing in the Mac version. I had played around with a bunch of different files, including some that had a mix of 1 Hz and 5 Hz data, and that must have somehow switch the setting on the Mac.

Must be some type of conspiracy....

Would you trust a muc user to be able o adjust anything properly

Select to expand quote

Roo said..
Pretty well covered in the U-Blox data sheets. VACC is the velocity accuracy estimate, velocity being calculated in 3 dimensions, x, y and z. The Sirf calulates SDOP, speed dilution of precision, speed SOG calculated from a scalar measurement.

Doppler speed in GPS are necessarily a product of 3D calculations. Can't be any other way. There is essentially no difference between the way Ublox measured Doppler speed and error and the way Sirf does it. They should be directly comparable.

Tom explained to me that we had the option to output 3D speed with the GT-31 but it was set to output 2D as we are only moving in 2D on the water and therefore make the calculation simpler. This was done to eliminate that variable and enhance accuracy.

The GW-52 is the same.

I just checked a couple tracks, but there is always a movement in 3 dimensions.

Lowest value in run 1 was 830.9 meter and highest 831.5 m (0.6 meter) 8 sats
Lowest value in run 2 was 830.8 meter and highest 831.9 m (0.9 meter) 9 sats
Lowest value in run 3 was 832.0 meter and highest 833.1 m (1.1 meter) 8 sats
Lowest value in run 4 was 829.5 meter and highest 831.6 m (2.1 meter) 8 sats
Lowest value in run 5 was 829.1 meter and highest 830.0 m (0.9 meter) 8 sats

Especially run 4 has a very high value, others values are the highest or the lowest when you look at the accuracy and speed +/- values.
I also checked the run in the overview and it is a run where I almost took of with my big slalom gear, that run had a lot of vertical movement anyway...
Calculations in 3D should be done when we use a new device.

Every time we use a GPS we are moving in 3D. We are moving in space relative to the satellites that we use to triangulate our position, to them we are not on a flat surface but moving in 3 directions. To dismiss this introduces an error into our calculations. The Sirf SDOP is speed dilution of precision which is based on speed over ground not 3D velocity that the U-Blox uses.

Select to expand quote

Roo said..
Every time we use a GPS we are moving in 3D. We are moving in space relative to the satellites that we use to triangulate our position, to them we are not on a flat surface but moving in 3 directions. To dismiss this introduces an error into our calculations. The Sirf SDOP is speed dilution of precision which is based on speed over ground not 3D velocity that the U-Blox uses.

Note was under the impression that "SDOP" was a name that we (the GPS community, specifically Tom) made up to describe the error [ for Doppler measurement ].

The Doppler speed measurement - by definition of GNSS calculations - is always the 3D instantaneous speed-over-ground measurement. There is some discussion around "fixing" one of the parameters in the calculation, to be at sea-level - but that really is only helpful if you have 3 satellites. Thus GNSS calculation producing Doppler-speed, also produces the error estimate. ie: implying it has units (m/s, knots, etc) as the speed-value.

xDOP... aka the TDOP, PDOP, GDOP, VDOP, HDOP, etc... are all unit less. They arn't in seconds, distance, etc. It is simply a scalar measure of error reduction.

I dont know enough about the UBLOX "velocity" - from the tec-notes, it appears it isn't a velocity at all, as the engine only produces speed-over-ground. [ But I may be mistaken on that - maybe we just dont see the direction property ]

As such, from my reading, the Ubox velocity error, and the Sirf SDOP are indeed the same. **

** that said, unless both companies open up their source-code, we will probably never know.

Select to expand quote

mathew said..

The Doppler speed measurement - by definition of GNSS calculations - is always the 3D instantaneous speed-over-ground measurement. There is some discussion around "fixing" one of the parameters in the calculation, to be at sea-level

The ublox chips actually have different navigation modes, including at least one ("sea") that fixes speed to 2D.

Select to expand quote

mathew said..

Thus GNSS calculation producing Doppler-speed, also produces the error estimate. ie: implying it has units (m/s, knots, etc) as the speed-value.

xDOP... aka the TDOP, PDOP, GDOP, VDOP, HDOP, etc... are all unit less.

Certainly true for HDOP etc., but not for SDOP. For ublox data, the units are not just implied - they are given in the receiver protocol specifications (for example cm/s for "Speed accuracy estimate" in NAV-VELECEF messages, according to the ublox7 specs). I have never seen a Sirf specification, so I'll have to take Tom Chalko's interpretation for it.

Select to expand quote

mathew said..

As such, from my reading, the Ubox velocity error, and the Sirf SDOP are indeed the same. **

** that said, unless both companies open up their source-code, we will probably never know.

Two questionable conclusions. A statement by the chip companies what they are trying to give us in the accuracy estimates would be a very good start, but I don't think we'll get that. But we can go and characterize the data that we are getting. That's not really easy, but can be done. Tom Chalko's original work indicated that the error estimates that Sirf chips give are higher than one or two standard deviations; in 81,493 data points, he did not find a single point were SDOP underestimated the actual error. If SDOP was 2 standard deviations, we'd expect about 2,000 points where SDOP is too low, so this points towards SDOP being > 4 SDs (a bit simplified). I have looked into this just a little bit, but my first impression for ublox data was that the ublox speed accuracy estimate is closer to 1 SD or 2 SD. If that is indeed the case, then the numbers are not the same.

But this analysis is a bit to simplistic. Tom's analysis of stationary (or, more accurately, constant speed) data did not take into account errors that can arise from speed changes at frequencies above the sampling rate (more or less - theory will add a factor of 2 here). Such changes could introduce "aliasing" errors, which is a reason why GPSResults does not use Gaussian error propagation for 1 Hz data (which are presumed to be GT31 data). To what extend that is still justified for 1 Hz GW-52 data is a different question. I am waiting for a few more GW-52s and ublox8 toys to look into this a bit more. Even if we did have a full specification of what SDOP and "Speed accuracy estimate" are supposed to be, and even if we did have the entire firmware source code and could understand it, we'd still need to do an empirical validation to see if the data we get are indeed what they are supposed to be.

Back from holiday and really bad wifi :)Here is a screenshot from GPSAR
Red GT-31
Yellow Gyro (Ublox-M8q@18hz, data comes from UBX and was saved to SBP via GPS Results)
Speed is around 27kts average, nothing spectacular

The highlighted part is 10sec.









Tracks can be made available on request :)

Is there a slight time displacement between the two tracks?

Two things immediately obvious.

1. The Doppler data is far better at high Hz (but we already knew that).

2. The time offset.

Select to expand quote

boardsurfr said..The ublox chips actually have different navigation modes, including at least one ("sea") that fixes speed to 2D.

That was available to Locosys from the SIRF chip as well, as I and Mat said in earlier posts, but probably only in the SDK so it's not something we can adjust. I had a feeling we had the choice in some settings. I vaguely remember a discussion with Tom about it when I experimented with the GPS for Skiing, (where one must have 3D speed) but it is not in the GT-31 menus. Maybe it was in a test firmware version for the GT-11?

Select to expand quote

sailquik said..
Two things immediately obvious.

1. The Doppler data is far better at high Hz (but we already knew that).

2. The time offset.

1. no comment :)

2. time offset is due to leap seconds in GPS satellites. Already send an email about it.

Select to expand quote

raymondw said..

2. time offset is due to leap seconds in GPS satellites. Already send an email about it.

GPS satellites dont use leap-seconds ... is that what you mean?

In any case, can you include the time-scale (horizontal axis) from these pics?