[Dprglist] PID-tuned Clock in Python?
Chris N
netterchris at gmail.com
Wed Feb 10 18:13:21 PST 2021
Murray, my thoughts:
I don’t think you have a “clock accuracy” issue. I’m pretty sure the hardware clocks, as in crystal + PLL etc., in things like the Pi, are plenty accurate for our needs.
What I mean by that is that if Python or some Linux system call tells us that 50.123ms have elapsed, then I’m pretty confident that 50.123ms +/- epsilon have elapsed. Sure, the crystal might run slightly fast or slightly slow, but either way epsilon is going to be quite small. ( I have never actually measured clock/crystal accuracy on the pi, but now I’m curious and I have thoughts on how to quantify this. I might try this and let you know what I find. Basically run the Pi a day or so w/ out NTP time sync )
We are dealing with a different issue. The fact that we are waiting for, e.g., 50.0000ms to elapse, but then find that actually 50.123ms have elapsed, is not because the crystal is off by a few KHz due to manufacturing tolerances, or is affected by temperature.
The vast majority of any timing inaccuracy that we experience in things like loop timers is due to the relatively large scheduling time jitter and general execution time jitter that comes Linux, and then Python adds to that as well.
What do I mean by jitter in this context? Say you are doing a time.sleep(0.050) (50ms) in Python. When I measure the actual sleep time on a Pi 3 with no load, what I find is that it always sleeps for at least ~70us more than what I asked for and occasionally it sleeps for ~170us more that what you asked for. So the jitter here is 100us.
Frankly 100us of jitter is not that bad. In your original e-mail you sort of asked if a 100us error is good enough. Yes.
100us jitter or error is OK, but throw in certain types of background workloads and the jitter is now suddenly in the >10 millisecond territory. Running the code with elevated priority will help only to some degree but won’t filter out all spikes.
Now regarding the PID idea:
Say we have a loop that we want to run at a fixed rate. Say 20Hz / 50.0ms.
Using PID to improve the timing accuracy of such loops is certainly an interesting idea, but I believe PID makes things worse in this case.
See output from your clock_test.py below. I ran this on my Pi 3. I deliberately used a tough background load to amplify the effect (stress –vm 4 –vm-bytes 128M) , but even with normal loads the negative effect of the PID can be observed, just the errors would be much smaller.
In line 6 you can see the 27ms error. What’s happening in lines 7,8,9,10,11,12, however, is the effect of the PID I believe, and not the effect of the stress workload.
_
dt: 50.00ms; delta 49.27500ms; error: -0.72500ms;
dt: 50.00ms; delta 49.61500ms; error: -0.38500ms;
dt: 50.00ms; delta 49.88400ms; error: -0.11600ms;
dt: 50.00ms; delta 49.89300ms; error: -0.10700ms;
dt: 50.00ms; delta 49.88400ms; error: -0.11600ms;
dt: 50.00ms; delta 77.00700ms; error: 27.00700ms;
dt: 50.00ms; delta 44.25200ms; error: -5.74800ms;
dt: 50.00ms; delta 31.79900ms; error: -18.20100ms;
dt: 50.00ms; delta 38.10600ms; error: -11.89400ms;
dt: 50.00ms; delta 43.93100ms; error: -6.06900ms;
dt: 50.00ms; delta 46.21600ms; error: -3.78400ms;
dt: 50.00ms; delta 48.14100ms; error: -1.85900ms;
dt: 50.00ms; delta 48.60000ms; error: -1.40000ms;
dt: 50.00ms; delta 49.50300ms; error: -0.49700ms;
dt: 50.00ms; delta 49.58000ms; error: -0.42000ms;
dt: 50.00ms; delta 49.71800ms; error: -0.28200ms;
The reason I think PID is a bad idea here is because the nature of the disturbance is simply too random and its very intermittent. The best you can do really is to use basic loop timing logic to ensure that the next iteration starts at the right time, despite the fact that this iteration took an unusual amount of time or sleep() took an unusual amount of time. With PID you end up over-compensating and you are effectively hurting the timing of subsequent iterations.
There are straight forward ways to deal with the fact that time.sleep(x) doesn’t sleep for exactly x amount of time, and the fact that the amount of work which needs to be done every iteration is not 100% constant. A python version of such a fixed-rate loop is here: https://github.com/nettercm/timing I typically use similar loop timing logic in other languages and sometimes even on a microcontroller.
In pseudo python it looks as follows. Interval is 0.050 in your case. Offset is the fixed error or time.sleep(x) (e.g. around 70us on Pi 3 with no load)
T_next = now()
While true:
T_next = t_next + interval # this line is the key. This way, timing errors won’t accumulate
Sleep_time = now() – t_next
Sleep_time = sleep_time - offset
Sleep(sleep_time )
Error = now() – t_next
From: Murray Altheim via DPRGlist
Sent: Wednesday, February 10, 2021 6:03 AM
To: dprglist at lists.dprg.org
Subject: Re: [Dprglist] PID-tuned Clock in Python?
On 9/02/21 10:06 pm, Murray Altheim via DPRGlist wrote:
> Executive Summary: This has to do with Publish-Subscribe message buses,
> PID controllers, the accuracy of system clocks, Python clocks, threads
> and timing loops, and yes, I know that last subject tickles the fancy
> of those using microcontrollers where these kinds of problems never happen. :-)
Okay, I've had a bit of a play with this. I haven't been entirely
scientific (hey, when the wind is southerly I know a hawk from a
handsaw), but my PID-tuned Clock seems to work at least better than
my non-PID-tuned Clock.
Running the clock_test.py Python script on a Raspberry Pi 3 B+ (on a
robot with an assortment of sensors), I've got the PID control tuned so
that the error on a 50.0ms loop is generally less than 0.01ms, typically
around 0.001-0.004ms. It intermittently bumps up to around 0.02ms error,
and if while the test is running on another console window I start an
"sudo apt update" I can force a couple of burps of ~0.5 error, but
generally it recovers within a clock loop back to under 0.01ms error.
Without the PID control the error is generally more in the 0.1-0.3ms
(or much worse under load) per loop range.
Not bad, methinks. Within the requirements I (think I) have for a timing
loop, it's probably not worth digging much deeper. How accurate is a
millisecond clock on an Arduino, an ESP32, a Tiny or some other
microcontroller/microcomputer? Dunno.
I'd thought about getting a DS3231 Real Time Clock as they're quite
accurate, I2C bus compatible, and not particularly expensive:
https://www.adafruit.com/product/3013
but surprisingly the DS3231 doesn't support a millisecond function. That
kind of thing is really not an RTC's function, which is more suited as
an accurate system clock and is extremely accurate at the seconds output.
Though the lack of milliseconds or microseconds support makes me wonder
what kind of RTC might be available that does support ms or µs.
I searched around and apparently there aren't any hobbyist RTCs that do
provide milliseconds output. The DS3231 does provide five sub-second
outputs: 32 KHz, 1 KHz, 1.024 KHz, 4.096 KHz and 8.192 KHz*. I don't
know how accurate those outputs are, as I don't know if they are drift-
compensated or not.
Now, the idea does occur to me that if the seconds output or one of the
high frequency outputs is really accurate, what if I used my PID class to
divide the output, then tune the PID to the error so that the Clock's
milliseconds are synced to the DS3231. Maybe that's a crazy idea.
I thought about using a GPS unit, as its 1 PPS clock claims a +/-20-30ns
jitter, the commonly-available MTK3339 down to 10ns. I'm not sure how this
all works as I think they typically have just a 32kHz crystal oscillator,
so maybe they need satellite sync to be seconds-accurate (?). But even
if they have some internal-miracle timer I'd still need to divide that
1 PPS signal myself to get milliseconds. Ugh.
Does anyone know of a millisecond- or even microsecond-supporting RTC?
I'm trying to think of the math formula that states how fast a crystal
oscillator would have to be to provide a certain level of milli- or
micro-second accuracy, but I can't find it right now. Maybe that's the
issue with "cheap" RTCs.
Or maybe I should just declare victory with my +/-0.01ms accuracy and
move on to a more productive sub-project? I'm leaning toward the latter...
Cheers,
Murray
----
The clock_test.py script and the rest of my 'ros' project can be found at:
https://github.com/ifurusato/ros
https://github.com/ifurusato/ros/blob/master/clock_test.py
https://github.com/ifurusato/ros/blob/master/lib/clock.py
https://github.com/ifurusato/ros/blob/master/lib/pid.py
I've commented out the Potentiometer import in the checked-in file so the
test script can theoretically run on any computer (wouldn't have to be a
robot) with the proper imports/installs. E.g., I ran it on my MacBook on
the train to work this morning.
* https://stackoverflow.com/questions/44644383/how-to-get-millisecond-resolution-from-ds3231-rtc
...........................................................................
Murray Altheim <murray18 at altheim dot com> = = ===
http://www.altheim.com/murray/ === ===
= = ===
In the evening
The rice leaves in the garden
Rustle in the autumn wind
That blows through my reed hut.
-- Minamoto no Tsunenobu
_______________________________________________
DPRGlist mailing list
DPRGlist at lists.dprg.org
http://lists.dprg.org/listinfo.cgi/dprglist-dprg.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dprg.org/pipermail/dprglist-dprg.org/attachments/20210210/adc4c31e/attachment-0001.html>
More information about the DPRGlist
mailing list