How to do NTP sync ?

[C1Ron]

I'm using version 01.01.01 - build 0000f4.
I've enabled NTP with servers: pool.ntp.org and time.nist.gov, and I'm connected to wifi, but the main clock quickly falls behind.
To resync the datetime, I've tried to restart device, disconnect/reconnect wifi, disable/re-enable NTP...
Nothing works.

I'm guessing the NTP won't resync unless the timedelta is above a certain threshold ?
How can I resync the clock ?

[gigapod]

HI @C1Ron -

We're basically using the default setup provided by the ESP32 API and following what the ESP32 Arduino implementation does.

Enabling/disabling the NTP option/service stops and starts it internally.

How far does it drift and how long to reach the drift point. And are you just comparing to other NTP synched devices?

Also, any other devices connected to the DataLogger? We'll try to reproduce this and find a solution.

-Kirk

[C1Ron]

Hi @gigapod

Last time I used the datalogger, it synchronized the clock when powering up in the morning, but then drifted by more than 10 sec behind in less than 5 hours later.

I'm comparing the clock to several other sources, e.g. to the time at https://time.is, and also to ntp requests from "europe.pool.ntp.org" from a Windows PC.

There are no devices connected.

Thanks :)

EDIT: note that the clock drifts less when not interfering the logging process. Today I was able to update the clock from zero unix time (1st jan 1970) and let the logger run for 30 minutes with ~ 1 sec drift. However, when going into the system settings trying to disable/enable the NTP and the WiFi, then it suddenly lagged by ~5 sec...

[gigapod]

Hi @C1Ron -

Hmm, I've had several of these devices just running, and the time is staying synched. Are you getting time output from log output or just when you bring up the settings menu (which shows time)?

When getting the data to log the system first grabs the timestamp, then loops over the connected devices to grab the latest data values for output. But this shouldn't cause any issues on the systems time value.

We'll keep trying to reproduce on our end.

-Kirk

[C1Ron]

Hi @gigapod

For checking time sync I'm watching the log output in the serial console using timestamps.

For reproducing a time lag, please start and stop the logging a few times, and make some changes in the settings. You'll quickly find that the timestamp lags by several seconds.

It would be really nice if you added a command for resyncing the clock. Perhaps the device won't resync unless the time-delta is above some threshold value. I purchased the IMU to measure angular displacement of a hanging load and need the timestamp to match NTP time as closely as possible, with no more error than a second.

Thanks
Ronny

[gigapod]

Hi @C1Ron,

Thanks for the additional updates on reproducing the error. We haven't seen it yet and have let several devices run over extending periods.

I wouldn't expect the clock to drift by so much time.

We are using the standard ESP functionality to update the clock via NTP. Additionally, our framework will synch the system and dependent clocks by a user set value - this is in the Clock settings of DataLogger. Key settings:

Reference Clock - This would be the NTP clock/system
Update Interval - Every "N" minutes, the system will grab epoch ticks from the reference clock, and set that value on the system clocks
Dependent Interval - If other clocks are connected to the system - say an RTC board - this is the time period between when the dependent clocks are update using the epoch value of the system.

While we continue to look at this issue - I'll see if we can also add a user driven clock synch command. The next release includes a command system for the serial console that skips the menu system to enter fast commands - a Time Sync command could be added here.

[C1Ron]

Thanks, it would be nice to get a user-driven synch command in the next release.

Today I'm not able to do NTP sync, at all. Even though the timestamp on the datalogger serial output says 31-12-1970 18:26:10.

PS I know that the wifi is connected because I just OTA updated the firmware and I've disabled/enabled the NTP, which produces [I] NTP startup [5 secs] ..... enabled.

The Update Interval is set to 1, so it should be updated after one minute, but unfortunately it doesn't.

[gigapod]

Hi @C1Ron ,

This is strange - Our latest take on this issue is that the clock on your board is defective - normally, the NTP sync happens every 60 minutes (default internally on the ESP32). We might need to send you another board for testing - but we want to check one more item.

For the current issue you are seeing - if you disable NTP and then re-enable it, does the clock synch up correctly ?

[C1Ron]

Hi Kirk. Yes, I've tried disabling and reenabling the NTP several times without success. The board was ordered via our departments purchaser, so I need to get order details from him if you need them to send me a new board. Thanks :) Ronny From: Kirk D. Benell ***@***.***> Sent: Wednesday, January 24, 2024 5:59 PM To: sparkfun/SparkFun_DataLogger ***@***.***> Cc: Ronny Landsverk ***@***.***>; Mention ***@***.***> Subject: Re: [sparkfun/SparkFun_DataLogger] How to do NTP sync ? (Issue #15) Hi @C1Ron<https://github.com/C1Ron> , This is strange - Our latest take on this issue is that the clock on your board is defective - normally, the NTP sync happens every 60 minutes (default internally on the ESP32). We might need to send you another board for testing - but we want to check one more item. For the current issue you are seeing - if you disable NTP and then re-enable it, does the clock synch up correctly ? - Reply to this email directly, view it on GitHub<#15 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ADT4EDW64AAGZDJF22EDLZLYQE4V5AVCNFSM6AAAAABAQJDG5OVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSMBYGU2DSOJSGM>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>>

[gigapod]

Did you ever get a replacement board to test?

If not - send me an email kirk.benell at sparkfun.com and I'll get a couple sent to you for testing....

[C1Ron]

Thanks Kirk - I never saw your final comment until today as I've been using a BNO055 on an R-PI and left the Sparkfun datalogger for a while. I sent an email to [email protected] just now - referencing this issue :)

[gigapod]

Hi @C1Ron -

I'll see another board gets set to you for testing/validation the other one had issues.

If the problem persists - which would be unexpected - we'll have to really did deeper on this ...

-Kirk

[C1Ron]

Hi Kirk.
I finally tested the new board that was sent to me, and it works perfectly. No issues with lagging timestamps anymore 👍

[gigapod]

Hi @C1Ron,

Great news! Thank you for getting back to us on this.

-Kirk

[C1Ron]

Hi again Kirk. I'm afraid I closed the issue a bit prematurely.

I'm once again struggling with the NTP sync on the datalogger.
I've uploaded an image where the time settings are appended.
The datalogger is online at all times and the lower image shows a side-by-side comparison of datalogger timestamp to the timestamp from querying the pool.ntp.org server using Python and ntplib.
I'm not able to get the datalogger to resync to the ntp server, and the update interval of 1 minute is not in effect.
I've also tried many other things, such as enabling/disabling the NTP client setting, and rebooting the device.

[gigapod]

Hi @C1Ron ,

This is very confusing -NTP is working - at least initially - since the time stamp value is close to actual time.

Looking at what we do, we are just making the same calls that the ESP32 Arduino core does as show in this function:
https://github.com/espressif/arduino-esp32/blob/66c9c0b1a6a36b85d27cdac0fb52098368de1a09/cores/esp32/esp32-hal-time.c#L47

We just don't make the timezone call (which is done somewhere else in our logic).

And I still can reproduce this.

Now on the ESP32 - the NTP update actually is handled by the IDF/NTP system. Our clock update system is generic so it can handle a variety of source clocks (RTC, GNSS/GPS).. Looking at this issue, I noted this before.

What if you disabled clock updates (set to 0, and not 1) - maybe someone our update system is fighting the internal NTP update?

Also, have you tried just running a generic ESP32 NTP example - either on another ESP32 board, or on the datalogger? There are many out there (example: https://www.upesy.com/blogs/tutorials/get-date-time-on-esp32-with-ntp-server-arduino-code?srsltid=AfmBOoq9FY5QTghWz9lK5gBjiHta0dsDj39fZ1W36K_gNZdv1a-x30WS)

If you do it on the datalogger, you then need to reflash the firmware using this:
https://github.com/sparkfun/SparkFun_DataLogger/releases/tag/v01.00.01

Then upgrading to latest firmware version.

-Kirk