[Dprglist] Running a Python thread at elevated priority

[Chris N]

Murray,

You can find a Pi3 compatible real-time kernel at
https://github.com/nettercm/timing

Just drill into the rt_kernel/pi3 directory for files and instructions and
important notes specific to the Pi 3.  I plan to also populate the pi4
direction with corresponding files for the Pi 4.

The "nice" API or  command line utility is not the right way to adjust
priorities.  Nice doesn't actually deal with real priorities.  It simply
tells the "completely fair scheduler" to give certain processes more or
less preference.  You must use the chrt utility or in python the
os.sched_setscheduler() API.   Both require special privileges so the
easiest way to try this out, before you aim for elegance inside your python
scripts, is via the command line examples below.

If you launch your threaded python application in this fashion, all python
threads will be at the same (elevated) priority.  There is a way to adjust
them individually, but there is no advantage to be gained, because as you
know there are inherent limitations to the python "threading" module
(keyword GIL).   So it's not possible for python "threading" to meet all
nuances of "prerequisite #1" from my other e-mail.  Don't worry about it
for now.  Just make sure none of your python threads are CPU hogs.


The following bash command line transcript illustrates how to display and
change affinity and priority for the instance of bash that you are in. By
default, child processes such as the instance of python that you run when
you start your script, inherit all this.   Just be sure to only launch your
time critical processes from this bash instance.


pi at raspberrypi:~ $ echo $$
6174

pi at raspberrypi:~ $ ps -ef | grep 6174
pi        5713  6174  0 11:45 pts/3    00:00:00 ps -ef
pi        5714  6174  0 11:45 pts/3    00:00:00 grep --color=auto 6174
pi        6174  1581  0 Feb19 pts/3    00:00:01 bash

pi at raspberrypi:~ $ taskset -c -p $$
pid 6174's current affinity list: 0-2

pi at raspberrypi:~ $ sudo taskset -c -p 3 $$
pid 6174's current affinity list: 0-2
pid 6174's new affinity list: 3

pi at raspberrypi:~ $ chrt -p $$
pid 6174's current scheduling policy: SCHED_OTHER
pid 6174's current scheduling priority: 0

pi at raspberrypi:~ $ sudo chrt -v -p -f 60 $$
pid 6174's current scheduling policy: SCHED_OTHER
pid 6174's current scheduling priority: 0
pid 6174's new scheduling policy: SCHED_FIFO
pid 6174's new scheduling priority: 60

pi at raspberrypi:~ $ python3 some_time_sensitive_python_script.py



On Sat, Feb 20, 2021 at 5:00 PM Murray Altheim <murray18 at altheim.com> wrote:

> Hi Chris,
>
> I just wanted to tease out one thing that would help me and potentially
> anyone else using Python on a robot.
>
> On 21/02/21 6:43 am, Chris N wrote:
> [...]
> > If you are not doing ALL of the following, then you are simply wasting
> your time:
> >
> > 1. Run your application – and all its threads - at elevated priority. Not
> >    only that, be sure to run your real-time threads at higher priority
> than
> >    your non-real-time threads.    The same would be true when one is
> using a
> >    RTOS. The OS can’t magically know which activity is most important.
> You
> >    have to tell it.
>
> Can you do this from within your Python code, or is it done when you call
> the Python interpreter on the command line? I'm currently starting my
> ros.py
> (main application) up from the command line (it uses argparse internally),
> or from a rosd.py systemctl daemon triggered by listening to a GPIO-wired
> toggle switch on the robot, so I can turn my ROS on and off with a switch.
>
> So I can either do something inside ros.py or inside rosd.py, since the
> latter creates the command line that calls the former. If its possible
> to do so inside ros.py that'd be easier and more consistent as I normally
> just type "ros.py -s" to start things up.
>
> I'm currently doing this using psutil* to set thread priority from within
> the ros.py file** (line 93):
>
>      import psutil
>      ...
>      proc = psutil.Process(os.getpid())
>      proc.nice(10)
>
> Is that correct? Or perhaps, is that what you're doing? Or something else?
>
> I'm currently hoping to whittle my application down to three threads:
>
>    1. main ros.py application
>    2. system Clock
>    3. gamepad (when used)
>    4. ballistic behaviours (run individually then quit)
>
> You're suggesting running all my application threads at an elevated
> priority. Do you suggest running them all at the same elevated priority,
> or perhaps prioritising, say, my Clock thread even higher?
>
> Cheers,
>
> Murray
>
> * https://psutil.readthedocs.io/en/latest/
> ** https://github.com/ifurusato/ros/blob/master/ros.py
> ...........................................................................
> Murray Altheim <murray18 at altheim dot com>                       = =  ===
> http://www.altheim.com/murray/                                     ===
> ===
>                                                                     = =
> ===
>      In the evening
>      The rice leaves in the garden
>      Rustle in the autumn wind
>      That blows through my reed hut.
>             -- Minamoto no Tsunenobu
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dprg.org/pipermail/dprglist-dprg.org/attachments/20210221/8101b6cd/attachment.html>