Select to expand quote
boardsurfr said..sailquik said..Roo said..
Here's some comparison measurements from today.
Just got back from a mission to the coast. Ran with GPSLogit on Alcatel One Touch Model 4037T Android 4.4.2 phone on left upper arm and 2 GT31 on right upper arm. I have included both doppler and trackpoint measurements as it gives you a good idea how clean the data stream is.
Max and 2 sec were all at the same time stamp of 8:43:50
Max: Phone 42.590 GT31-1 42.181 +/-0.272 GT31-2 42.259 +/-0.253 Trackpoint: Phone: 42.951 GT31-1 42.533 GT31-2 42.566
2 sec: Phone 42.492 GT31-1 42.143 +/-0.272 GT31-2 42.172 +/-0.253 Trackpoint: Phone: 42.685 GT31-1 42.400 GT31-2 42.358
5 x 10 sec: Phone 38.781 GT31-1 38.558 GT31-2 38.563 Trackpoint: Phone: 38.834 GT31-1 38.830 GT31-2 38.875
10 sec run Phone GT31-1 GT31-2
1 9:16:18 203.4m 39.540 202.5m 39.357 +/- 0.255 202.5m 39.356 +/- 0.253
2 9:43:54 201.7m 39.206 199.4m 38.765 +/- 0.289 199.4m 38.757 +/- 0.258
3 9:56:03 200.9m 39.054 199.5m 38.773 +/- 0.286 199.5m 38.782 +/- 0.284
4 9:21.07 196.8m 38.259 195.7m 38.035 +/- 0.291 195.7m 38.004 +/- 0.277
5 9:32:56 194.7m 37.849 194.8m 37.859 +/- 0.265 194.9m 37.877 +/- 0.254
All in all the phone acquits itself pretty well.
'Pretty well'???
Well, I guess so if you are measuring with a stretchy string!
I have to say that an
objective view of this 10 sec. run data is that
the phone is over reading constantly by up to half a knot. Especially since the data from the two GT-31's is consistently extremely close and within 0.01 and 0.03 Kts of each other on all the 10 second runs. (this is why they are so good!)
It can be observed that the Maximin 99% error range of the GT-31's is between 0.253 and 0.291 Kts. This says that we can be 99% sure that the speed is at least what the reading says.
There are different ways of looking at Roo's data. Four of the five 10-second numbers of the phone are within the range for the GT-31 speeds; only run #2 is outside of the GT-31 range, by 0.16 knots. To correctly compare the two numbers, however, we would need accuracy estimates for the phone numbers, too (which are not available). But it seems reasonable to assume similar numbers as for the GT-31 (+/- 0.25-0.3 knots); then, the speed ranges overlap for all 5 runs. In other words, the numbers are
not statistically different.
The next question is what the +/- numbers actually mean. I'd love to see an actual description. The "99% sure" refers to 3 standard deviations and normal-distributed random errors. Typically, however, +/- numbers give a single standard deviation. Even if the numbers would indicate 3 standard deviations, a non-normal distribution of the errors would reduce the probability to a number below 99%.
The fact that the two GT-31s that were both worn on the same arm give very similar numbers does
not prove that the numbers are accurate. Error sources in the GPS data, for example from atmospheric distortions, will be virtually identical for two devices right next to each other. An ideal test would involve one phone and one GT-31 on each arm.
That said, there does indeed appear to be a bias in the reported speeds between the phone and the GT-31s. But
we do not know for certain whether the phone overstates the speed, or the GT-31s understate it. Assigning the error to the phone is jumping to conclusions. It is at just as likely that the GT-31 underreports the doppler speeds (similar to the way speedometers in cars are allowed to over-report speed, but not under-report it). I have seen slower reported Doppler speeds in GT-31 data when compared to a 5 Hz ublox GPS that appeared to be more accurate (but this is just anecdotal). The 5x10 positional speed data, which are very similar for all three GPS units and higher than the doppler speeds, also point towards the GT-31s under-reporting speeds. Maybe Roo knows something about bias in the GT-31 firmware when reporting doppler speed.
This is just one example of a data set. But it is a significant difference in that
all the 10 second runs are
significantly faster. Of course you would have to study a lot more comparison runs from this particular combination to see if this is truly a consistent pattern, but if this was just random range within the GT-31 range, we would expect to see just as many variations
below the GT-31 readings in a large sample, and since there are none even close to that in this small sample, it is a big worry. As you said, that is a strong indicator of a bias.
On the other hand, most of the testing here has not shown that significance of bias, but some has. There are so many combinations of phones and GPs chips and and antenna and they way they are implemented that we are struggling to find and list those we can recommend. More Doppler data comparisons like that which Roo submitted would help a lot.
We don't compare GPS worn on different arms when doing testing for accuracy as we cannot guarantee that they are seeing the same number of satellites with the same reception. The body and head can easily block signals and the orientation of the units can be quite different and change the geometry of satellite reception. In fact, most of my and Tom Chalko's testing is done with the GPS worn on the top of the head. A lot of Toms testing of many scores of GPS, GT-11, GT-31 and more recently GW-52 was done with arrays of 10 or more mounted on the roof rack of his car. He actually claims, supported by a whole lot of Math that is beyond my grasp, that the GT-31 is far more accurate that the SDOP error values suggest, (by a factor 10) and that the GW-52 beats it by a significant margin again. (subject to ideal reception). We choose to take a very conservative view and are therefore very confident of it.
The trackpoint data is almost totally irrelevant. We saw a long time ago that it is subject to far greater error variation and almost always reports higher speed due to a number of known factors, including the inherent error of positioning and often lack of resolution in the positional data (grid Effect). To put it simply, the positional points not only vary in the line of travel, (which tend to cancel each other out) but to the side of the direction of travel. This sideways variation leads to a slight Zig Zagging of the plotted track which effectively causes over reporting of the speed. Some data from higher the Hz units is really bad. To use positional data for speed calculation we would need far higher positional accuracy than the consumer GPS chips can produce. Experiments with RTK and base station correction are promising to bring that level of accuracy in the not too distant future. Roos laudable attempts to post process GT-11 data apparently failed to get the desired results, I am guessing, due to the poor quality of the data the Chips were capable of providing. Since he did not share his methods and results with us, we don't really know.
As to the accuracy of the GT-31 and GW-52 using Doppler derived data. There are numerous scientific papers and studies going back 10 years or more that report the method and reasons why the Doppler derived data can be used for high accuracy speed calculation if implemented correctly. A massive amount of testing was done by Tom Chalko and others including Manfred Fuchs to confirm the speed accuracy of the GT-31 in particular. I personally took part in one of Manfred's tests at the Luderitz speed canal in 2013 comparing both GT-31 and Ublox10hz data against the official timing gates. In those tests the GPS's fared extremely well, (and the UBlox based test units even better) and the few very small variations from the official speed timing gates were pretty evenly spread slightly higher and lower, and, as I remember,
well within the reported error values of the GPS's tested.
But as Decepit has pointed out. There are still issues with data from some Phones that is not related to their inherent GPS accuracy. Spotting these errors when comparing them directly with a GT-31/GW-52 is not difficult, but when a sailor is using them as their only source of data, and they are inexperienced in recognising errors, we could end up with quite a significant amount of junk data, and that can easily upset a keen, close fought competition, even on GPS-TC.
We have no issues with new sailors who are finding their interest and just want to get involved at a fun level using phones with GPS-Logit to see their results and post. How we implant their use in the GPS-TC challenge competition is the question we are working on.
Roo
> 9 months ago, 31 Oct 2015
