Sounds like a bug in the start_logger script - did you add the proper indentation? Python is picky there..

Things you can try:
- Check the stdout.txt file for messages.
- Comment out the shutdown command in the "logging_done" script, and replace it with a print. You'll have to turn the Pi off using a keyboard or VNC.
- Run the start_logger script from the command line. You may see error messages that you don't see otherwise this way.

I did get a couple of windsurfing sessions in today to test the prototypes. First one in light wind on flat water, second one was a bit crazy - bump & jump with wind from 10-30 knots. Learned that I definitely have to modify the Android bluetooth logger app to automatically re-connect after crashes . Bluetooth does not work under water, and the HC-06 disconnects. No surprise, just had not gotten around to it.

Just glanced at the data, but they look very good so far. Here's an graph showing the top 10 10-second speeds for the two prototypes and the GW-60 I used as a control:


Numbers are within ~ +- 0.1 knots, usually closer:


The watch and the GPYes dongle had very similar +- numbers, the ublox 8 was a little better; it got ~18 sats vs. 10 for the two others, and was on top of my head. In the second session, I used two watches, one on each hand, and it's immediately obvious when I used an under grip (which I do only with my left hand) - the data look much worse:



The undergrip watch is the red line, the other watch the blue line. No surprise, but funny to see this come up again in this test.

So I confirmed that the wireless keyboard mouse wasn't the cause of excess current drain. Again samba connection vanished as soon as GPS detected with nothing else plugged in.
Now Samba doesn't work!!!!!!!
Apparently the nmbd daemon can't find a non loopback local ipv4 connection.

And Now our internet has gone very flaky, I'm writing this in one of it's brief on periods

Did you change the password on the Pi? These things are quite a security risk otherwise, since the well-known default gives you full root access to a Linux machine. When I have internet cameras running, someone tries to hijack them within less than 24 hours. The most common attacks just try default passwords.

For my earlier development work, I used a Raspberry Pi 3. It has the advantage of an ethernet port and several USB ports, and is a bit faster. You can simply swap the SD cards between the Zero and the 3. At times, the wired network comes in handy when the WiFi router acts funny.

Yep, changed password, one of the first things I did.

But strangely, sudo, doesn't ask for password, so I'm not sure how secure it is.

OH dear oh dear oh dear, got samba working again, turned out to be very easy, I just had to turn the wifi back on!!!!!!!!!!!!!!!!!!!!!!
No idea how it turned itself off.
I'm going to bed now, I may get round to trying more stuff tomorrow.

From the Arduino development side of things I finally played with the code to get data logging to SD card. It turns out it is easy, as long as you go with the option to pre-allocate the files.

The data logger creates 2Gig files before it actually does anything so that the SD card doesn't need to go searching and erasing disk blocks when the data logger needs a new one to store more data. So it removes that delay, which tends to be the thing that catches everyone out with data logging as they don't have much memory available to buffer the data while waiting for the SD card.

I ran the logger a few times, just with the default timestamp data that the person that wrote the sketch included, and it filled up the 2Gig file in something like 30mins. This is more than enough speed as I have calculated that two streams of 115200bps data would be flat out filling 2Gig of storage in an entire day.

Its nice when other people have done all the hard work!

Then I moved onto looking at UBX data streams. It seems someone has also done some nice work on that and also created a few youtube videos to talk about it. Good work! The first simple sketch is enough to identify any UBX sentences and he then moves onto parsing a few of them. He also provides another sketch that sends the initialisation strings to the M8N, so he does all the work there too.

So, hopefully in the next week I can insert the code that filters out streams of UBX data and log them to the data logger.

I am keen to see if this works similarly well on the slower Arduino processors just to see if its doable. Its not much of an advantage though as the price of these Teensy modules is so cheap that it doesn't give any great benefit other than maybe the power consumption.

Select to expand quote

decrepit said..
Think I may have a solution to my intermittent problems. Files that don't open, u-center crashing on some saves and occasional error messages on saving.
Originally I connected the module and converter with wires, but that was awkward to move around, so I decided using the posts would be better.
I had trouble cleaning the small holes after removing the wires, (my solder sucker is at least a 1/4 century old) so I overheated the terminals.
Then the posts supplied with the converter, were too thick to go in the converter holes! I filed them down, and thought they went in easily enough, but when I had a close look, The whole track of RX had lifted, and the round connection at the end broken off, The TX was a bit better only the copper hole had lifted and was alongside the post. I endeavored to connect the RX track to the post with a piece of wire. With glasses and magnifier it seemed I'd succeeded, but thinking of the faults, a noisy/intermittent RX signal would explain it.



Aren't phone camera's great, I get a much better look at what's going on through the lens of my phone than any other way.

So you can see the discontinuity in the R and X letters, this is the lifted track. You can also see the wire I've added sticking out slightly to the left. It looks like my connection here is no where near as I thought it was.

Not sure I've got the skills, eye sight and steady hands to improve this much, but I guess I'd better try.
maybe if I scrape the paint off the track I can get a better soldered connection to it.

Ha, I had to deal with these modules again yesterday. I forgot how small the holes were and now I can understand why you had problems with them. I think if I started from scratch again with those, I would buy some small protoboards and join the GPS module to those with some hookup wire, and then solder the leads to the protoboard.

All the time I was thinking, 'I hope I don't end up overheating the traces on these things'. Any long term solution is definitely going to involve soldering them to another PCB as using wire is just problematic.

While I was preparing for bed last night, I was pondering how the wifi turned it self off, then had a light bulb moment.

Everything is working as it's supposed too, it's just suffering from an attack of silly old fart syndrome.
So what did I do wrong?
well all the clues are here.
wind the clock back a week or so, or a few pages in this thread. Peter advises me to comment out the blue tooth, wifi and display turn off instructions, so I can see what's happening.
So being a good little linux nerd, I call the file "old" and comment it out
Then after I can't get the dual gps mod working I comment that out, and uncomment out the original and delete the "old"
Have you got it???

That was my mistake, completely forgetting the commented out turn off section.
It's only taken me 3 days to remember, and that's with the help of turned off wifi.
The Pi wasn't crashing, it's just turned itself invisible. So I only thought it had crashed.

So there you go Fangy, have I reached my alloted number of mistakes? Cause from here on in I wouldn't mind a relatively clear run!

Mind you I haven't tested this theory yet, the wind is blowing and I feel like a sail.

Sorry Decrepit, I noticed on page 876 of the Federal Budget press release;
" ...recommending lifting of all Silly Old Fart mistake threshold limits...blah blah...to be implemented immediately...blah, blah.."
So, Good News! It turns out, you can keep wracking them up with no issues or tax implications. So keep at, as I only have a few more weeks and then I will be entering the fray, and I want a nice clear run myself.

Had a fun sailing session today, with the bluetooth GPS on the helmet and a GW60 on each arm. I noticed while sailing that one of the watches had missed one of my better runs, and discovered at home that it had dropped lots of points during a ~2-3 minute period, including 50 seconds where it did not record anything. And that was the "good" watch - the other one with the broken glass had water drops on the inside of the glass after the session, but it recorded ok. The data from the bluetooth ublox 8 GPS again looked great. Details and some numbers at boardsurfr.blogspot.com/2018/05/summer-speed-and-gps-results.html

Great result Peter, I think you have had a win!

So my theory is correct, and now I know what's happening I've started some more tests, and discovered another problem.
The 2,200mah battery hasn't got enough oomph to run the wifi, hdmi and GPSdongle, I only get files in the bytes when I try this. A 3/4 charged 4,000mah batter gives me files in the kb and a fully charged 4,000mah battery stays up as long as I've left it on so far.

So I've reverted to the original python start script and turned wifi and hdmi off when gps is detected.

The file from the module on the little battery is corrupt, I'm just seeing if the bigger battery is an improvement. Next trick is a long extension cord, to see what that does. It may be that my soldering isn't the culprit.

Back later if I get any meaning full results.

nup, file still corrupt, so I'll give the extension cord a go, but I think it's my soldering.

Peter if it is my soldering, the chances of me recovering the usb/serial converter are extremely low. Instead of buying another converter, I think I'll go your way with a blue tooth module.
My Samsung is 4 years old now, could be a good excuse to replace it.

I have gotten some data with the Pi with a GPS module connected directly over the pins, without USB. Unfortunately, getting this to work is a bit confusing, since the Pi Zero W uses the UART pins for other stuff (bluetooth and debugging). The instructions on how to get it to work on the web are mostly outdated, incomplete, and/or not very clear. It took me a few hours to get to work, and since I tried so many things, I'm not sure what exactly the steps were that made it work.

There's also work that needs to be done on the software side. The Rxtx library I used can be made to work, but it requires creating a symbolic link to discover the UART. Stopping logging is also different, not as easy as just pulling a plug and getting an exception that tells the program we're done. Finally, autobauding is highly desirable or even essential for GPS modules that forget baud settings. I tried to work on that yesterday, but discovered that the Rxtx causes some rather bad crashes on my Mac. I'll have to look into alternatives. All that may take a few weeks, so the bluetooth module may be your best bet.

Select to expand quote

decrepit said..
The 2,200mah battery hasn't got enough oomph to run the wifi, hdmi and GPSdongle, I only get files in the bytes when I try this. A 3/4 charged 4,000mah batter gives me files in the kb and a fully charged 4,000mah battery stays up as long as I've left it on so far.

It sounds like either your batteries are almost dead, or you have an intermittent short in your system. The Pi should use less than 200 mA, so the 2200 mAh battery should last for hours. I do get several hours from a 1200 mA 3.7 V battery, with a dongle connected and everything turned on. No cable in the HDMI connector, but that should not make a drastic difference.

Try putting a USB tester between the Pi and the battery (like this one: https://www.amazon.com/gp/product/B00J3JSEG6/).

OK , that sounds plausible, I assumed it was a current limitation causing a voltage drop, rather than a lack of charge in the batteries.
Your link looks well worth it, but I get this.

Select to expand quote

Price:$9.99 In Stock. This item does not ship to Australia. Please check other sellers who may ship internationally.

I'll do some googling, and also see if your blue tooth device is available here


OK, both on order, it's very easy to spend money at $10, $20 a time!

Select to expand quote

decrepit said..
OK , that sounds plausible, I assumed it was a current limitation causing a voltage drop, rather than a lack of charge in the batteries.
Your link looks well worth it, but I get this.

Price:$9.99 In Stock. This item does not ship to Australia. Please check other sellers who may ship internationally.

I'll do some googling, and also see if your blue tooth device is available here


OK, both on order, it's very easy to spend money at $10, $20 a time!

Current limitation? Please be aware if you aren't already that lithium-ion cells can put out a ridiculous amount of energy in a short time, but you probably already know that. The voltage regulator part of the pack could be a problem as you suggested, but you would think that these things would do about 1amp and 500mA at a minimum. 500mA seems to be the minimum USB charging current these days, so you would think it could supply that easily.

Unfortunately in my case I decided to hook up two Li-ion chargers in parallel, just to test it, despite reading that it is a really bad idea. Result, one dead battery pack and a lot of glowing wires and a glowing circuit board for a relatively short time. I won''t be doing that again!

It was a very cheap no brand battery, so I thought I just got what I paid for. But when the tester arrives I may get an idea of what's going on

I just timed the battery life on my bluetooth setup. I got 7 1/2 hours with a 500 mAh LiPo battery (indoors, but that should not matter unless it's cold outside).

I also just discovered the Openlog serial recorder. That seems like a rather cheap ($9 in the US, $5 from China) and easy way to add logging. Could be used either as a backup for phone logging, or for a simple, small logger without a display. Hookup should be just as easy as the bluetooth hookup, configuration even easier (a file on the SD card instead of AT commands over a serial-to-USB link).

AliExpress also currently has a $5.80 special for a ublox 8 GPS (TopGNSS store). I tried to order a few, but got cold feet when they asked for a copy of the drivers license and more to "verify" the account. The description and the ucenter screen shots look pretty decent, though. But sending all the necessary information for identity theft to China .. maybe I rather pay a few dollars more. Thought I'd mention it in case someone has an account setup already.

How funny is that. The openlog serial data logger is pretty much an Arduino, albeit probably running non Arduino code and optimised just for the task its doing. Hardware wise, its just an Atmega328, which is pretty much the heart of most Arduino boards

Mike , I just received a couple of USB testers . Not using at the moment so if you want to check things while you wait for yours I will drop them over.
Peter, Banggood have the openlog serial data recorder for 8.50 US . I have found them good and professional. Ordered about 30 items so far.

Thanks David, I've put the GPS aside for the moment, while I finish my board. I've paid a bit more for postage, so mine should be here reasonably soon.

Select to expand quote

FormulaNova said..
How funny is that. The openlog serial data logger is pretty much an Arduino, albeit probably running non Arduino code and optimised just for the task its doing. Hardware wise, its just an Atmega328, which is pretty much the heart of most Arduino boards

Made for drones, to serve as a "black box" recorder. You actually can program it like an Arduino Uno, the software is open source. But you don't have to - it works out of the box. Not quite plug & play, but "solder and play".

I actually have an Arduino data logger board lying around (Adafruit Featherlogger). Adafruit has great instructions, and I have used the Arduino IDE a little bit in the past, but it looks so tedious that I was never tempted to try. In contrast, the Openlog has a couple of nice and simple ways to configure it: a configuration file you can edit on you PC/Mac, and a command line. A couple of potential issues are that it drops into command mode when it received 3 ASCII 26 characters in a row, and the limited buffer and SD card writing speed issues. But the character sequence can be configured to be longer and use different characters, and the 512 character buffer should be good enough to keep up with logging 5-10 Hz NAV-PVT messages, which are 88 bytes each. It may be necessary to drop the NAV-SVINFO messages, since they can be > 250 bytes if the GPS tracks 20 satellites. But that's information that's not used for track analysis, anyway, so dropping it is perfectly safe.

Select to expand quote

yoyo said..
Peter, Banggood have the openlog serial data recorder for 8.50 US . I have found them good and professional. Ordered about 30 items so far.

Thanks, good to know that you've have positive experiences with Banggood. I ordered one from ebay for $9.25 that is shipped from within the US, so I should get it next week. I could have gotten it through ebay from China for a few bucks less with a longer shipping time.

I got the Openlog logger today, and I am impressed. The thing is barely bigger than a micro SD card, and I had it logging GPS data within 5 minutes. With a fast micro SD card, it has no problems logging 10 Hz data, even when adding the (not really needed) NAV-SVINFO messages, which increase the data amount about 3-fold.

The only issue is that the files it creates have a ".TXT" extension, which mean GPSResults gives an error message when trying to open them. Simply changing the extension to .ubx fixes that. However, GPSResults also seems to have a problem with satellite numbers if the files contain SVINFO messages.GPSResults always shows fewer satellites than I see in u-center or my software. It appears that GPSResults does not count the GLONASS satellites. So if you want to see the correct number of satellites in GPSResults, omit the NAV-SVINFO message. The NAV-PVT message also contains the number of satellites used and is sufficient.

A little note of caution: if you're buying a cheap GPS module that's sold as "NEO-M8N GPS", you're likely to get a piece of **.

I received a second one of these modules today - this one from ebay (https://www.ebay.com/itm/132361760191). I bought it because it has a 25x4 mm antenna, which is bigger than the 18x2 and 25x2 antennas on the Beitian GPS modules I have (which don't claim to be ublox 8, and which work very well). The description specifically states:
"With data backup battery." and "EEPROM save the configuration parameter data when power-down."
That seems to be a blatant lie.

The first test was to change the baud rate, save the configuration, and see if the module remembers the changes. It does not. It does save the settings to BBR (battery backed RAM), but as soon as I unplug the module, it reverts back to 9600 baud. That makes the module completely unusable for bluetooth. Other settings are also "forgotten". There is something that looks like a battery on the board, but it's easy a dead battery or a fake. Unlike the BEITIAN modules, this module does not seem to have Flash memory at all (UBX-LOG-INFO show 0 kb storage).

The firmware on this module is 2.0.1, which means only 2 satellite systems can be used at the same time. Perhaps the firmware could be updated, but there's no point in doing that because the unit absolutely sucks otherwise .. read on.

The worst part of this module is that it is extremely slow to get a fix. Other GPS modules generally get a fix within a minute or so, even indoors. The BEITIAN 880 goes up to tracking 12-14 satellites within 10-15 minutes. This module takes much longer to get a fix; 25 minutes after plugging it in the last time, it currently tracks 3 (yes, three) satellites. Usually, GPS modules get faster at acquiring satellites if you plug them in and then unplug them several times, since they can save some of the tracking information. This module seems to get slower; the first times I plugged it in, it got up to 6 satellites within 15 minutes. It does not save anything to permanent memory; unplugging after 30 minutes starts the process over again.

I have had pretty similar issues with another "NEO-M8" GPS module I had bought 2 years ago from a different seller. At least this time, I'll be able to get my money back from ebay. Some of the problems Mike had with a his module sounded very similar. There are dozens of sellers on ebay, Aliexpress, etc. who sell GPS modules that look exactly the same, or very similar. Beware.
--
I also discovered why the seller and this item have such a positive rating on ebay (which I almost never use). Buyers have to wait 7 days before they can leave neutral or negative feedback, apparently because the seller is a "power seller". Almost amusing. Certainly distorting.

Does the 'Ublox' module have a QR code on it? I read somewhere that the fakes have a QR code (3 squares in the corners), and the originals have a code, but its a different type.

Decrepit's version had a capacitor, but installed incorrectly, so it would never have worked. Even the fakes should store the config changes from ram as its pretty simple, and the capacitor should be enough to keep it overnight.

If you went for a "minimal" GPS and bluetooth setup, I think it be almost worth the while to have an Arduino pro mini hooked up as well, just to send the config to the GPS on startup. Then again, in theory you shouldn't need to if the modules stored their configs.

Peter, the capacitor in mine, took a while to charge up, but 25min would have done it.
Just check it's not the same as mine was, with the contacts shorted out.

I should get my USB checker soon, my board is finished, so I'll be back to playing with GPSs

Relying on just BBR will cause problems every time the battery goes down. With a capacitor, which seems to be the common choice for the cheap fakes, that will happen after a few hours or days. With a battery, at some unpredictable time. So for a bluetooth setup, you would indeed need to add some controller to configure the GPS, or it would just stop working at some time. It's not trivial, since the controller would need auto-bauding. Also, it would be connected to two devices, the bluetooth module and the GPS. This gets complicated.

A proper NEO-M8 setup should have Flash memory for firmware upgrades and to store data. This fake does not have Flash. The BEITIAN 880 (which does not claim to be a NEO-M8, but uses a u-blox 8 chip) has both BBR and Flash. It can hold settings for at least several days. I verified that you can save the configuration separately to Flash and BBR. I hope that the firmware is smart enough to pull configuration data from Flash if the battery/capacitor is dead at startup, but have not verified that.

The real issue is that the "NEO-M8N" modules are a counterfeit product - someone is lying about what they are selling you. Often, they don't have a clue what they are doing. For example, they hook up an EEPROM to the u-blox chip. That worked with ublox 6 chips, but the newer 7 and 8 modules don't support EEPROM anymore (according to some posts on the ublox forum). If they are a bit smarter and add Flash, they often use a version that's too small for firmware updates, all to save a few cents.

So you're dealing with a seller that violates intellectual property law, lies about it, and puts together a product that is by design faulty. It's a safe bet that they will use cheap, low-quality components for everything else, too. That definitely appears to be the case here. The satellite acquisition is a lot slower than with other GPS modules (ublox 7 and ublox 8), even when cold starting the modules. Without Flash and working BBR, every start is a cold start - we're talking about at least 10-15 minutes before the satellite numbers become stable. Theoretically, this "NEO-M8N" module should have better reception than other modules, since the antenna is larger; in reality, reception is worse by all counts. This includes S/N ratios, which are lower than for the BEITIAN 220 with a much smaller 18x18x2 mm antenna.