Flex, how about printing a 2nd female housing that slides over your current one, sandwiching the clear plate in the middle (leave a window in it for your display. This would be a bit more bulky, but give you a lot more contact area to seal with

thanks Stretchy, will be a good idea to help eliminate a seam leak as each print could have the seam at different spots..will try that if next tests still leaking. For those not familiar with 3D printing, a seam is created every time the printer jumps to a new layer. You can fiddle with the position of them but they have to be somewhere. Annealing may also help.

Dropped 3 more prototypes in the pool yesterday morning and due to a bit of an adventure yesterday, left them a full 24hrs. All still leaked but by about the same amount of water as previous test left for less than 1/2 the time so an improvement but not perfect. One had beefed up bevels in the corners, one had random seams and the other is spray painted inside with bolt on silicon. I twisted off one bolt which probably didn't help the test but don't think that was the cause of leak. Now printing the shell approach as suggested by Stretchy. Given up putting any kind of mount on until can get something reasonably water tight.

looks good flex, be nice to have something you're confident of.

Could you make the cases round wth threads and as you screw them together they seal at the top with an o-ring or rubber insert in the lid, like a bottle of Tom sauce or wine bottle. Depending on how the window could be placed & display positioned.

Select to expand quote

rp6conrad said..

jn1 said..
BTW, if bandwidth is an issue, why not compress the stream with gz library ?. The ESP32 is probably equivalent to a Intel Pentium in the late 90s, so no problem with horse power.

@10Hz, bandwidth is just enough. Compressing the data could work, but I believe the issue is more with open file, write 100 bytes (1 navpvt sentence) and closing file. Maybe if you only write every s, compressing the 1000 bytes would make a difference.

Your application is accessing a filesystem operations directly ? (ie: stdio.h, fcntl.h etc ?), and you are performing a close()/fclose() after each record you write ?

I use the filesystem of arduino : #include "SD_card.h"
With every ubx message, the file is opened and closed :
if((logUBX==true)&(frame_count>10)){
//write ubx string to sd, last 2 bytes with checksum are missing !!
ubxfile = SD.open(filenameUBX, FILE_APPEND);
ubxfile.write(0xB5);ubxfile.write(0x62);//add ubx navPVT header !
ubxfile.write((const uint8_t *)&ubxMessage.navPvt, sizeof(ubxMessage.navPvt));
ubxfile.write(checksumA);ubxfile.write(checksumB);//add checksum
ubxfile.close();
//Serial.print ("Write_timeCSV= ");Serial.println(millis()-delta);
}
With a class10 SD card, this works fine@10Hz. If I want to write 2 different files with every message, this is not possible @10Hz (lost messages). Probably if the messages are buffered, and write in a block, it could be possible. I did not try this yet. Mean problem now is sealing the unit, and wireless charging. The T5 board is rather sensitive to the power supply, and with wireless charging, I blown a fuse with some trials.
Greetings, Jan.

Is this the Espressif Systems SD(esp32) library version 1.0.5 ?

This is the library :
esp32/1.0.4/
SD.h
// Copyright 2015-2016 Espressif Systems (Shanghai) PTE LTD
Should the 1.0.5 version be better ?
I still use the Arduino 1.8.1 version.

Select to expand quote

rp6conrad said..
I use the filesystem of arduino : #include "SD_card.h"
With every ubx message, the file is opened and closed :
if((logUBX==true)&(frame_count>10)){
//write ubx string to sd, last 2 bytes with checksum are missing !!
ubxfile = SD.open(filenameUBX, FILE_APPEND);
ubxfile.write(0xB5);ubxfile.write(0x62);//add ubx navPVT header !
ubxfile.write((const uint8_t *)&ubxMessage.navPvt, sizeof(ubxMessage.navPvt));
ubxfile.write(checksumA);ubxfile.write(checksumB);//add checksum
ubxfile.close();
//Serial.print ("Write_timeCSV= ");Serial.println(millis()-delta);
}
With a class10 SD card, this works fine@10Hz. If I want to write 2 different files with every message, this is not possible @10Hz (lost messages). Probably if the messages are buffered, and write in a block, it could be possible. I did not try this yet. Mean problem now is sealing the unit, and wireless charging. The T5 board is rather sensitive to the power supply, and with wireless charging, I blown a fuse with some trials.
Greetings, Jan.

That technique isn't optimal.

The Flash chips in an SD card has a maximum number of write-cycles. Opening and closing a file causes the internal logic (of the chip) to perform write leveling -> this has the effect of doubling or tripling the number of writes per user-write. In practice you should open the file once, then only over close it when finished.

Similarly, each write() does a read-before-write so that the byte offsets are maintained, which doubles the number of flash-writes per user-write. You should buffer the user-write to match the block-size of the card, then write out the whole block -> only a single hardware-write will be issued. ( It still causes other internal/hardware writes as block-pointers are updated, but it is far less than per-byte-writes. )

Also, SD cards can push 10MB-40MB per second -> two 10Hz streams should be trivial. Each of those open(), close() and read-before-write will cause a massive slow down of the I/O rate ( vs block-sized-writes ) -> each of those calls requires block-flushing (including those block-size-writes), so if you limit the writing to only using block-writes, you will be minimising the flushing.

Matt: Conrad is using the SD code via a FATfs file system layer. As you said, SD cards are block devices, but file systems do clever tricks to make a block device look like a character device, with what you said in mind. Unfortunately, the fatfs library is a complied object, and I can't see the code on my system, but I'm guessing that it is clever, depending how much effort Espressif have put into it. There is only one way to find out:

Conrad: the SD library you are using lives here (well, on my system):

~/.arduino15/packages/esp32/hardware/esp32/1.0.4/libraries/SD

I recommend you backup this directory and then add some debug serial printf() statements into this library. Then write a small test program to write a byte of data with your open() write(1 byte) close() convention, and then see what it does. Then write another program that writes two bytes: open() write(1 byte) write(1 byte) close() etc.

I recommend putting some printfs in sdReadBytes(), sdWriteBytes(), sdReadSector(), sdReadSectors(), sWriteSector(), and sdWriteSectors(). These are located in sd_diskio.cpp in the directory (folder) above.

What you want to know is:

- if I write a byte, is the file system reading 512 bytes and then writing 512 bytes (a block write) ?

- What happens if I write 2 bytes in a row ?. Does file system do 1 read block and 2 writes blocks ?. Or does it do nothing ? (implying the file system is caching).

Once you gauge what the file system and SD library are doing, you'll be able to write some elegant code that gets your application singing

Matt is right. You should have ooddles of bandwidth.

All good stuff above but remember he's on SPI, not SDIO. It should be well enough to log at 10Hz but it's not class 10 either.

The SD library (SD.h) looks like it sets a hard coded SPI clock frequency of 4MHz. It definitely should cope.

Bit of an update, the 3D printed sleeve housing was the least waterproof of all so far. Checked it after 5 mins at 2m water depth and had the same amount of water as all the previous attempts at 12 or 24hr submersion. The Polycarb face had been glued (dichloromethane) to the inner, then the outer glued to the polycarbonate and the mating edges 100% covered with silicon and all left to cure 24hrs+. Oddly, left it on the bench and all the water drained out within an hr (as per photo) which implies the base is leaking. The base was 3mm thick with 100% infill. Only real change was I changed filament colour (still PETG) so I believe this filament colour (same supplier) needs to be over extruded to produce water tight relative to the previous colour. However, I've had the same print fully submerged at 0m depth and it doesn't leak at all so might be being a bit harsh testing at 2m. Might try 50cm water depth test, then 1m etc till loose seal. Whilst this was happening the small off the shelf boxes arrived..next post

the off the shelf boxes arrived at the same time as the bits for more units so tested 4 units today. Was not much wind but ok for a test. Had 3D printed some nice boom mounts, put one on top of the helmet and the other strapped the the impact vest on my shoulder. For whatever reason I only ordered 3 little boxes but 5 sets of electronics (one of which I blew up immediately) so the one on helmet is still the big box. Considering the boom ones were in the water a bit and I was walking around with the body mounted units whilst the boom ones were stationary they compare ok. However had quite a few GPS dropouts on the boom mounted ones, a few on the head and none on the shoulder which implies hardware/something issue... early days

Nice work Jim, I have a feeling the boom mounted ones will catch on.

I've uploaded the 3D printed boom mount to thingiverse if anyone wants. (fits my 26mm Severne boom which has 31mm hand grip but probably will suit most booms)....of course it only suits the specific box which is mentioned above (and in the link below) www.thingiverse.com/thing:4927778 Adjust the idea as you see fit.

Wireless charging in the box is also working now. As the charge controller on the board is only connected to the micro-usb, there is no direct solderpoint for the Qi-receiver module. But you can solder the +5V from the Qi module direct the Vcc of the charge controller. The other possibility is the use a Qi-receiver with a usb-micro connector, but the standard connection is rather short. The module that I use : nl.aliexpress.com/item/4001062923015.html?spm=a2g0o.search0304.0.0.59392c7aQuKUGR&algo_pvid=40b7b857-3bce-4c73-a28d-d8ece8dcf43d&algo_exp_id=40b7b857-3bce-4c73-a28d-d8ece8dcf43d-17

You right Jan, the USB ones have short cables and they are also very weak. I managed to break mine at the USB connection today by doing a small adjustment. Of course I've been playing quite a bit with various orientations so its probably outside 'normal use'. Can still use it but now have add extensions and solder it in. Was nice being plug and play though.

Could you maybe test the 5V/1A output claim? Thank you

probably can test Julien since the wires are now busted. However is it really important? Also, really depends on SOC of battery. For $14 including charge pad or $2.5 for Jan's one without charge pad I'm not gonna worry about actual output. It seems to do the job. Probably for a 1% SOC battery it might get close to what they claim?? The thing fully charges the battery in a few hrs and normally like most people I would just leave it charging overnight. Certainly the battery gets pretty warm when charging so wouldn't want much more. Since a 2000mAhr battery lasts at least 24hrs of active logging (haven't actually done a real test of battery longevity but it is far far longer than what my body is capable of) there is no need for rapid charging as the body will take far longer to recharge.

Just pure curiosity.

The charge controller on the T5 board is limited to 500 mA, but my Qi module does no more then 300 mA (in the electrical box, distance ca 5mm of the sender). Maybe the limiting factor is my Qi sender, this is also a low power one (5W). But for now, even with 300 mA the lipo can be charged within 7 hours. The bad efficiency (<70%) is also a limiting factor, as the coil will heating up and the charging will stop due to overtemperature.

what software do you need to read the data on a Mac or does it not work for a Mac ?

GPSAR-Pro and GPS-Speedreader both run on Mac in Java and both read ubx files (and oao Motion files).

There is a Mac Version of GPS-Results that runs on Intel Macs that reads both as well, but wont work in the newest 64 bit only Macs.

The Windows Version of GPS-Results has more features than the Mac version and runs perfectly well on Intel Macs in Emulation Software. I use the free version of VMware with Windows7 on both my older, 2014 13" MacBook Pro, and my new 2020 16" MacBook Pro and it's great.
Parallels and Boot Camp apparently work as well, but I understand that the new M1 Macs cant run Windows in Emulation yet, and certainly not Boot Camp.

Select to expand quote

AUS 2459 said..
what software do you need to read the data on a Mac or does it not work for a Mac ?

if you are referring to the Wifi FTP xfer to move the files from SD card to elsewhere without having to open the box (which is super handy) it only works with PC (at present). I have Mac and no amount of trying with various FTP programs works. It works perfectly with WinSCP on the cheapest PC you can buy. Have not tried with PC emulation. If you have no PC, then you need to plan to plug in the SD card to get the data. For the actual data analysis refer to above posts.

Probably damm obvious too those that know this stuff but you really need to format the SD card properly before adding the config.txt file and going for a run. You should also buy the fastest SD card you can afford (but also the smallest size which seems to be 16G for me). A 4hr session is around 13MB which is roughly 5000 sessions. For my first unit I bought a good quality SD card, and formatted the card and all was fine. I used Disk Utility on Mac (I believe the PC alternative is www.sdcardformatter.com/). Then I got lazy on the next 3 units and didn't format the SD cards and got huge data drop outs (not GPS). Just tested the same units after formatting the SD cards and had massive improvement.

Select to expand quote

rp6conrad said..
Today, I had a low wind summersession on my homespot. The two left loggers (BN220)were on the boom, the two on the right(BN280) were on my helmet. The avg from the 4 devices were very close as you can see. All values recorded @5 Hz, in km/h.
The max 2s had a little more deviation, but on the helmet again very close !
Crashed into a submersed tree, had some sail damage. On the e-paper, you can see some ghosting. This is @power off, the screen will be refreshed every hour. After 10 refreshes, it looks better. Normal screen has almost no gosting.

Just looking back here Jan at your earlier posts...you were flying...impressive speeds. are you sure it was 'low wind' conditions? Or is the display in Km/hr?

Select to expand quote

Flex2 said..

rp6conrad said..
Today, I had a low wind summersession on my homespot. The two left loggers (BN220)were on the boom, the two on the right(BN280) were on my helmet. The avg from the 4 devices were very close as you can see. All values recorded @5 Hz, in km/h.
The max 2s had a little more deviation, but on the helmet again very close !
Crashed into a submersed tree, had some sail damage. On the e-paper, you can see some ghosting. This is @power off, the screen will be refreshed every hour. After 10 refreshes, it looks better. Normal screen has almost no gosting.

Just looking back here Jan at your earlier posts...you were flying...impressive speeds. are you sure it was 'low wind' conditions? Or is the display in Km/hr?

Here in Europe, we mostly use km/hr for speedsurfing ! So, my speeds were moderate ! I am not that good, my pr max_2s ist only 65 km/hr (Herkingen, last month !). I only have "Old School boards" my favourite is the F2 Sputnik 270, 82l and 52cm width (1995) !

Select to expand quote

mathew said..

rp6conrad said..
I use the filesystem of arduino : #include "SD_card.h"
With every ubx message, the file is opened and closed :
if((logUBX==true)&(frame_count>10)){
//write ubx string to sd, last 2 bytes with checksum are missing !!
ubxfile = SD.open(filenameUBX, FILE_APPEND);
ubxfile.write(0xB5);ubxfile.write(0x62);//add ubx navPVT header !
ubxfile.write((const uint8_t *)&ubxMessage.navPvt, sizeof(ubxMessage.navPvt));
ubxfile.write(checksumA);ubxfile.write(checksumB);//add checksum
ubxfile.close();
//Serial.print ("Write_timeCSV= ");Serial.println(millis()-delta);
}
With a class10 SD card, this works fine@10Hz. If I want to write 2 different files with every message, this is not possible @10Hz (lost messages). Probably if the messages are buffered, and write in a block, it could be possible. I did not try this yet. Mean problem now is sealing the unit, and wireless charging. The T5 board is rather sensitive to the power supply, and with wireless charging, I blown a fuse with some trials.
Greetings, Jan.

That technique isn't optimal.

The Flash chips in an SD card has a maximum number of write-cycles. Opening and closing a file causes the internal logic (of the chip) to perform write leveling -> this has the effect of doubling or tripling the number of writes per user-write. In practice you should open the file once, then only over close it when finished.

Similarly, each write() does a read-before-write so that the byte offsets are maintained, which doubles the number of flash-writes per user-write. You should buffer the user-write to match the block-size of the card, then write out the whole block -> only a single hardware-write will be issued. ( It still causes other internal/hardware writes as block-pointers are updated, but it is far less than per-byte-writes. )

Also, SD cards can push 10MB-40MB per second -> two 10Hz streams should be trivial. Each of those open(), close() and read-before-write will cause a massive slow down of the I/O rate ( vs block-sized-writes ) -> each of those calls requires block-flushing (including those block-size-writes), so if you limit the writing to only using block-writes, you will be minimising the flushing.

I adapted the logging to the SD : no more closing of the files, ony when shutting down. Every minute, I do now a flush, to prevent data loss when a unplanned shutdown occurs (empty battery). I will do more testing, but the first test look good ! Thanks for the advice.