The 18 second difference is indeed from the leap seconds. According to the u-blox specifications, "GPS time" does not include leap seconds, while "UTC time" includes the leap seconds. It appears that GPSResults adds the leap seconds to the UTC time (which is wrong!). Here is the same file, one with GPSResults and once with my parser:
My code parses the date and time fields in the NAV-PVT records directly, using the description from the u-blox documentation. The times I get always match the GW-60 times.
Some chips and GPS devices report times without adding the leap seconds (IIRC, the Canmore falls into that category). Perhaps GPSResults tries to correct for that with u-blox data, too. There are also other possible explanations, but whatever the cause, GPSResults reports the time for .ubx files incorrectly.
This requires a bug fix in GPSResults. You can try to contact Manfred Fuchs about this, or have Andrew contact him, since he seems to be friends with Manfred. Good luck.
Select to expand quotedecrepit said..
I've just been using NAV PVT, that seems to give everything needed.
I think that is true to get a result and use the minimum filters.
However, if you inclide the NAV-SAT info, you also get this: satellite constellation, satellites used and satellite signal strength. These can look really pretty, 
, but can also be sometimes useful in some sorts of data validation analysis.
Ublox 10 Hz file:
You get some of this data from the SBN file from the GT-31 (But without the satellite signal strength):
I heard from Core Electronics that the SD Logger is discontinued, the nice people are substituting the more expensive sparkfun logomatic V2. for no extra cost.
This has 2 sequential loggers, support for high speed cards, battery maintenance on board, and download of data via USB. So with any luck, it will do 10hz data with NAV SAT included.
It's got more features that I won't use, unless I figure out a use for the compass data in the BN280
Select to expand quotesailquik said..
However, if you inclide the NAV-SAT info, you also get this: satellite constellation, satellites used and satellite signal strength. These can look really pretty, , but can also be sometimes useful in some sorts of data validation analysis.
Pretty pictures, yes. But if you include NAV-SAT, GPSResults gives the wrong number of satellites tracked - it only counts the GPS satellites, and ignores GLONASS and Galileo. Here's an example:
This is from a prototype test. The left file has only NAV-PVT, the right file also has NAV-SAT. Both units were recording at 5 Hz from 3 GNSS systems. The unit at the right actually tracked 24 satellites here, as the screen shot from my parser shows:
This could possibly mean that data points are filtered out wrongly by GPSResults, e.g. if 10 satellites are tracked, but only 4 are from the GPS system. Until this bug in GPSResults is fixed, logging NAV-SAT info is not advisable.
The screen shot above is from the Mac version, which does not have the really pretty graphics. However, the Windows version seems to have the same problem; the low satellite number in the data you show is one indication in this direction.
I did a few stationary tests with u-blox 8 prototypes at 10 Hz today. The units used three GNSS systems (GPS, GLONASS, and Galileo), and usually tracked about 20-25 satellites.
Result:
The Openlog can definitely log just the necessary data (NAV-PVT) at 10 Hz without loosing points, using the ublox 8-based Beitian BN280 and BN880 chips. But when adding detailed satellite info (NAV-SVINFO) once per second, I routinely get missing data points even at 5 Hz. It's probably the larger data size of the NAV-SVINFO records (~ 300 bytes vs. ~100 bytes) that can cause buffer overflow. Since the detailed satellite info in not used in any of the filters or analysis functions in any software, but can cause lost points, it should be omitted.
I also looked at randomness of noise in stationary tests by examining the correlation of neighboring data points. I used velN and velE rather than the non-directional horizontal doppler speed. I did not find any significant amount of correlation between neighboring points, or between data from two units placed close to each other. So at least in this test, the errors are random. In windsurfing, there will be some actual causes for non-random spikes that show correlation, for example head or arm movements at frequencies below 5 Hz. When reception is poor, errors can also be non-random, with similar large errors observed over several seconds.
Taken together, this supports recording at 10 Hz (the highest possible rate for using 3 GNSS systems with u-blox 8 chips). I plan to do some testing with two prototypes and two GW-52s when we get wind again (probably Tuesday). It will be interesting to see what the overlap between the prototypes at 10 Hz will be, since the error range estimates should be tighter (around 70% of the 5 Hz ranges). Even at 5 Hz, the u-blox 8 Openlog units already have 2-3-fold lower +/- estimates than the Locosys units.
Select to expand quoteboardsurfr said.. <<<<<< The Openlog can definitely log just the necessary data (NAV-PVT) at 10 Hz without loosing points,
Interesting you achieved that Peter, what card were you using? I think the card is the key to getting the open logger working at higher rates.Select to expand quote But when adding detailed satellite info (NAV-SVINFO) once per second,
So how the hell do you do that? I've tried a few times, and I can only adjust all the rates together not individual sentences.
Good work with the analysis looks like 10hz is a worth while goal.
Select to expand quotedecrepit said..
Interesting you achieved that Peter, what card were you using? I think the card is the key to getting the open logger working at higher rates.
Both cards I am currently using are a SanDisk Ultra HC1 / A1 16 GB. The "A1" indicates some optimization for using apps, probably meaning better random access (as compared to generally more sequential access for movies and pictures). I'm not sure how much the exact card matters, though - I still get lost points when I add in NAV-SVINFO. In one test, I recorded from the same chip onto Openlog and via Bluetooth. Only the Openlog data had ten dropped points out of 33,000 (almost 2 hours). A couple of dropped points per hour seem to be typical for 5 Hz NAV-PVT / 1 Hz NAV-SVINFO recordings. Without NAV-SVINFO, I have not seen a single dropped point in about 6 hours of recording at 5 and 10 Hz (split over a couple of prototypes and several separate sessions).Select to expand quotedecrepit said..
So how the hell do you do that? I've tried a few times, and I can only adjust all the rates together not individual sentences.
In the configuration window in u-center, select the message (NAV-SAT or NAV-SVINFO), check the port (usually UART1), and then enter the number 5 (or 10, or any number up to 255) in the window next to it. This means the message will be sent only every 5th (or 10th, ...) period.
Recording the detailed sat info just once per second will reduce the number of missing points, and doing it just every few seconds may reduce it even more.
Having detailed satellite info can make sense when you want to check which satellites were tracked, but remember you'll have to use u-center, since GPSResults still does not show GLONASS or Galileo satellites. In general, you'll be better off disabling NAV-SAT and NAV-SVINFO. Without those, GPSResults reads the number of satellites from the NAV-PVT records, and shows it correctly. But if GPSResults finds NAV-SAT or NAV-SVINFO, it will show only the number of satellites from the US satellites.
Thanks Peter,
Here's some results I got from a couple of SD cards.
Ubuntu 14 "disks" formated to fat 32
balcony test at 10hz NAV-PVT sentence only
ex dot's class 4 about 8 misses per minute
ex sony sandisk ultra HC1 about 25 to 30 misses per minute.
I think I had better results from a class 10, but it still had misses, of course I can't find the file now.
At the moment I won't try NAV-SAT, but if the new logger proves faster, or a better SD card gives me the opportunity, I'd like to be able to reduce it to 1hz.
I ran another 8 1/2 h of tests logging at 10 Hz today. There was one instance where three points in a row were missing, so the difference between the complete sentences was 400 ms instead of 100 ms. I think I'll have a look at different Openlog firmware versions. Going from the regular version to Openlog_Light increases the buffer from 512 bytes to 850 bytes, which should have been enough in this instance. Going to Openlog_Minimal increases the buffer to 1024 bytes.
Downloading the Arduino IDE and the Openlog code from learn.sparkfun.com/tutorials/openlog-hookup-guide was easy enough. So was moving the libraries to the logical place (~/Documents/Arduino/Libraries). To get the Openlog sketches to compile, I had to edit SerialPort.h in the Libraries folder, as described in the last comment at github.com/sparkfun/OpenLog/issues/188
That's as far as I got for now, installing the new firmware versions will have to wait a day or two.
Aha, that explains the difference between my results and yours Peter. Aus Post say my new logger and serial to usb converter will arrive today. If that doesn't give me 10hz out of the box, I'm going to have to learn how to use Arduino stuff.
Select to expand quotedecrepit said..
Aha, that explains the difference between my results and yours Peter.
Not sure what you mean. So far, I have been running the standard Openlog software.
Unfortunately, there are many places where things can go wrong, so nailing down what exactly is the problem can be hard. It can be the GPS chip (e.g. when running firmware from Flash), or the memory card, or the Openlog software. With the Drotek NEO-8M, I had a few dropouts at 5 Hz when connected to the phone over USB. I can't be sure if it's the chip, though - it could also be the phone or my logging software.
Select to expand quoteboardsurfr said..
Not sure what you mean. So far, I have been running the standard Openlog software.
Unfortunately, there are many places where things can go wrong, so nailing down what exactly is the problem can be hard. It can be the GPS chip (e.g. when running firmware from Flash), or the memory card, or the Openlog software. With the Drotek NEO-8M, I had a few dropouts at 5 Hz when connected to the phone over USB. I can't be sure if it's the chip, though - it could also be the phone or my logging software.
OK, I miss read you, I guess I need to do more experimenting.
I have couple questions on the Beitian modules:
In what way is the BN880 different from the BN280? They seem to have the same specs, but I guess I'm missing something.
How does the BN800 perform compared to the BN880? The BN-800 seems to only have a passive antenna, but is slimmer because of it and has mounting holes that seem useful to me.
Select to expand quotesrtgumbee said..
I have couple questions on the Beitian modules:
In what way is the BN880 different from the BN280? They seem to have the same specs, but I guess I'm missing something.
How does the BN800 perform compared to the BN880? The BN-800 seems to only have a passive antenna, but is slimmer because of it and has mounting holes that seem useful to me.
The BN 800 and BN 880 have a compass, the BN 280 does not.
I'd assume that the active antenna improves accuracy, but have not tested the 800. When I looked at the different Beitians in the past, many units were still ublox 7 based. That seems to have changed (at least in the Amazon US listings), every one I looked at is now u-blox 8.
For the mounting, I have used Velcro dots. Works well, and it's easy to take the modules off when I want to hook them up to u-center for logging changes etc.
I think the 880 has a bigger antenna as well. I initially ordered a 280 because I don't need the compass, but it was unavailable so the seller substituted an 880
Select to expand quotedecrepit said..
I think the 880 has a bigger antenna as well. I initially ordered a 280 because I don't need the compass, but it was unavailable so the seller substituted an 880
The 880 and 280 that I have both have the same L0010 antenna (25 mm). The 180 and 220 have the smaller antenna (18 mm).
On a different topic - dropped points. I windsurfed for 1 1/2 hours yesterday with two prototypes (BN280 and 880 at 10 Hz). Both dropped a single point in the middle of a run twice. That did not affect the 2 and 10 sec numbers, which are good and in range as usual. However, since I was going slow on a longboard (just 10-12 knots of wind), I had only a few long runs, so dropped points complicated the comparison of 250 m to nautical mile (since GPSResults does not consider a mile if a single point is missing). Maybe I should have stayed home and changed the Openlog firmware, but long boarding on the ME2XR is more fun 
.
Funny thing - a few minutes after my last post, I open a package with a new BN280 in it. This one has a thicker antenna (4 mm instead of 2). Overall, it is thinner than the old one, since the middle "layer" is missing. Here's an image (the new one is on the right):
Thanks for the clarification on the Beitian modules.
It seems some sellers just copy and paste specifications and its hard to know what you are actually ordering. Beitian do have a website (www.sz-beitian.com/productsMukuaiEng?product_mukuai=GPS%E6%A8%A1%E5%9D%97), but unfortunately I can't find data sheets to specific modules on there.
Interesting photos of the BN-280 - seems one with an active antenna + short coax cable? and one with a passive antenna.
Did they both come from the same seller?
Still going around in circles on what GPS module to buy OR make my own using genuine parts.
Just found this info portal.u-blox.com/s/
See last post from Clive (Ublox Senior Principal Expert)
This company looks good reyax.com/products/?term_id=9 and I will look to get a module from their ebay store: www.ebay.com/sch/m.html?_ssn=reyax&_from=R40&_trksid=m570.l1313&_nkw=gps&_sacat=0 once I can work out which module will suit best
When it comes to genuine or not, keep in mind that Beitian states they use u-blox 8 chips. They are quite different from a lot of resellers of the fake products in that they don't just sell on e-bay under always changing names. If they were not using genuine u-blox chips, it would be quite easy for u-blox to stop them, or at least list them as a known supplier of fake modules. To I assume that the u-blox GPS chip in Beitian modules is genuine.
I have done one windsurfing comparison between the Drotek NEO-M8N, and two Beitians. Speeds and +/- estimates were very similar; the biggest difference was that the GPS on the helmet (a Beitian) had better accuracy than the two units I wore in armbands. That's most likely due to the better view of the satellites, not differences between the units.
It will be interesting to see how the new BN880 compares to the old one. The new antenna indeed looks like a passive one, but is a bit fatter than the old one. I bought both from the same seller (Geekstory) on Amazon. The pictures still show the active antenna. If the new module does not live up to the old one, I'll return it.
I just noticed that the same seller actually has two posts for BN280 - the old one with 25x2 (active) antenna for $16.95 (https://www.amazon.com/gp/product/B07BR3B1MQ/), and a new one with the 25x4 (passive) antenna for $13.74 (www.amazon.com/Beitian-U-G7020-KT-Antenna-Pixhawk-Geekstory/dp/B07DHK6FW3/). I ordered the first one but received the second one. So even if it works well, I should at least get $3.21 back.
A quick stationary test shows no significant differences between the old and the new BN280. Here's a plot of sAcc and satellites in u-center (old on top, new on bottom):
Data were recorded at 1 Hz, so the speed accuracy graph shows about 5 minutes of data. The number of sats tracked was around 20 for both units, with a mix of GPS (G), GLONASS (R), and Galileo (E). Next step will be to update the firmware on an Openlog, and to solder together a new prototype.
I ran into problems when trying to change the Openlog firmware. I can get the code to compile, and communicate with the Openlog through the serial monitor terminal, but when I try to upload the firmware onto the Openlog, I get error messages (avrdude: stk500_getsync() attempt 10 of 10: not in sync: resp=0x00 and avrdude: stk500_recv(): programmer is not responding).
I have spent a couple of hours trying to get around this, but to no avail. The instructions on sparkfun are quite simple, but don't work for me. I found some posts with explanation for the error messages is that there is no bootloader. More likely is that I don't have the right FTDI USB adapter- apparently it needs a DTR pin to reset the chip just before uploading a new Sketch. There might be a way around this by using the command line instead of the Arduino IDE, and a manual switch. Not sure it's really worth the effort. 5 Hz logging works just fine, and accuracy of the ublox prototypes is at least as good as for Locosys units. Even 10 Hz works quite well, except that GPSResults does not tolerate a single missing data point. But missing every second point by recording at 5 Hz works.. go figure.
I'm having trouble with the logomatic as well, the spark fun literature is far from clear.
It has 2 serial connections, the block diagram labels "0" as programing connection, so I connected to "1" no logging occured.
Then I noticed the config description talks about uart "0" so I've swapped to that, now both the buffer leds are flashing, so I'll see what's happened in a minute or so.
I've gone for broke first up, 10hz and NAV-Sat ever 2s, (I like Andrew's pretty colours)
Well so far so good.
4590 points read: checksum: 0, corrupted: 0, missing: 2
081344.098 200ms
081344.200 102ms
I'll have a look at these points and get back.
That's odd I can't find them, that time doesn't exist. Once it's looked in it starts at 081357
Select to expand quotedecrepit said..
I'm having trouble with the logomatic as well, the spark fun literature is far from clear.
It has 2 serial connections, the block diagram labels "0" as programing connection, so I connected to "1" no logging occured.
Then I noticed the config description talks about uart "0" so I've swapped to that, now both the buffer leds are flashing, so I'll see what's happened in a minute or so.
I've gone for broke first up, 10hz and NAV-Sat ever 2s, (I like Andrew's pretty colours)
Well so far so good.
4590 points read: checksum: 0, corrupted: 0, missing: 2
081344.098 200ms
081344.200 102ms
I'll have a look at these points and get back.
That's odd I can't find them, that time doesn't exist. Once it's looked in it starts at 081357
This can happen at the very beginning of a track, when the GPS does not have reception yet. The time is not synchronized yet, and the information about the leap seconds is not available. There are usually a couple of larger jumps at the beginning of the track. You may not see them because the control flags indicate "no GPS fix", so GPSResults may not show them. You could look at the file in u-center to confirm (check the GNSS fixType and valid flags).
Yes, that's what it looks like, the time is between when it first turns on with no fix, and when it does get a fix and the time becomes correct.
Looks like version 3 with a BN880 and logomatic V2 is a goer!
No missing points, so I think that confirms the initial fix was the problem before.
Here's the results from about a 90min balcony test at 10hz NAV-PVT and a 2s interval NAV-SAT
54147 points read: checksum: 0, corrupted: 0, missing: 0
I'm still using the old 2200maH power bank with it, so there's no room to fit it in an enclosure. A new 1000maH lipo should arrive early next week so I'll be able to take it for a sail, and compare with version 2 and the GW52.
Select to expand quotedecrepit said..
A new 1000maH lipo should arrive early next week so I'll be able to take it for a sail, and compare with version 2 and the GW52.
When you test it, try to have the two prototypes right next to each other. I did some tests with one unit on an arm and the other on the head, and the ranges did not overlap for longer runs. Differences were only 0.04 knots or less for 500 m, though. Turns out that this corresponds to just about a millimeter difference per data point, which could very easily be from differences in actual movements (perpendicular to the direction of motion, so it adds a bit to speed over ground).
