[Dprglist] PID-tuned Clock in Python?

[Murray Altheim]

On 11/02/21 3:13 pm, Chris N wrote:> I don’t think you have a “clock accuracy” issue.  I’m pretty sure
> the hardware clocks, as in crystal + PLL etc., in things like the
> Pi, are plenty accurate for our needs.

Hi Chris,

First, thanks very much, I appreciate your thoughtful reply.

Yes, I understand that it's not the accuracy of the Raspberry Pi's
system clock that is in question, but the accuracy of time loops
when implemented in a Python application using a single or multiple
threads, executing within a time-sharing operating system like Linux.

This is clearly where a microcontroller (like an Arduino, STM32,
ESP32, Pico RP2040, etc.) has its advantages. As I noted, my
MacBook was actually worse than my Raspberry Pi, which in this
light is perhaps not so surprising, i.e., the MacBook's processor
is a lot busier.

My 3.5GHz multi-core i7 workstation is clearly giving at least a
whole core to the Python application and not struggling, so its
performance is very close to expected norms. But that's not the
case on the Pi, where I'll actually be using this timing loop.

[...]
> Now regarding the PID idea:
> 
> Say we have a loop that we want to run at a fixed rate. Say 20Hz / 50.0ms.
> 
> Using PID to improve the timing accuracy of such loops is certainly
> an interesting idea, but I believe PID makes things worse in this case.

I'm not clear that this is the case yet. My observation was the the PID
loop was improving things somewhat significantly, but more on that below.

> See output from your clock_test.py below.   I ran this on my Pi 3.
> I deliberately used a tough background load to amplify the effect 
> (stress –vm 4 –vm-bytes 128M) , but even with normal loads the 
> negative effect of the PID can be observed, just the errors would
> be much smaller.

Agreed, understood.

> In line 6 you can see the 27ms error. 
> [...]
> The reason I think PID is a bad idea here is because the nature of
> the disturbance is simply too random and its very intermittent. 
> The best you can do really is to use basic loop timing logic to 
> ensure that the next iteration starts at the right time, despite
> the fact that this iteration took an unusual amount of time or 
> sleep() took an unusual amount of time. With PID you end up  > over-compensating and you are effectively hurting the timing of
> subsequent iterations.

I certainly agree based on both logic and my observations that the
disturbances are intermittent, and from the perspective of the
Python program (i.e., it not knowing what else is running), would
appear entirely randomly.

But I'm not clear the PID loop is over-compensating in those cases.
When a disturbance is within one 20Hz clock cycle, there is no real
fix for that in software. I entirely agree with you in those cases.
But when the CPU is really busy with some longer-term process and
that activity extends say over several seconds, the PID (I believe)
would tend to compensate better than with no PID.

> There are straight forward ways to deal with the fact that  
> time.sleep(x) doesn’t sleep for exactly x amount of time, and the
> fact that the amount of work which needs to be done every iteration
> is not 100% constant.  A python version of such a fixed-rate loop > is here: https://github.com/nettercm/timing   I typically use
> similar loop timing logic in other languages and sometimes even 
> on a microcontroller.
> 
> In pseudo python it looks as follows. [...]
If I'm reading your code correctly, this is kinda what I was doing
before I adopted the PID into the mix. I had a fixed trim value on
the Rate, and when the 50ms loop was consistently a bit slow or a
bit fast I'd just modify the trim value. It was trying to auto-correct
that trim value that led me to using a PID loop. Now, a fixed trim
would obviously not take into account those nasty intermittent surges
or lags, but is certainly simpler to implement and a lot less
mysterious in use than a PID loop.

So if the PID turns out to be causing actual problems as you suggest,
I can set the flag to disable it, and maybe re-implement the fixed
trim. I've been setting the PID's kp constant from an RGB LED
potentiometer like:

   https://shop.pimoroni.com/products/rgb-potentiometer-breakout

so if I end up disabling the PID I could manually set the trim from
the potentiometer, then just fix it in my YAML configuration. This
would be a "best guess" on a fixed, constant trim value.

I think we both agree that there's nothing anyone can do to get a
truly consistent loop using Python on Linux, but if I'm willing to
accept that those intermittent <50ms disturbances are also unfixable,
then the PID does still at least (IMO) manage the longer lags and
surges of the clock. This *seems* to be what I'm seeing in the
console outputs.

The whole subject of PID is all a bit dream-catcher, Voynich Manuscript,
ouija board, patchouli incense kind of thing... verra-mysterious.

Cheers,

Murray


...........................................................................
Murray Altheim <murray18 at altheim dot com>                       = =  ===
http://www.altheim.com/murray/                                     ===  ===
                                                                    = =  ===
     In the evening
     The rice leaves in the garden
     Rustle in the autumn wind
     That blows through my reed hut.
            -- Minamoto no Tsunenobu