[Dprglist] Visualizing data [was: PID]

[David Anderson]

Hey Murray,

Sounds like great progress.

I also log robot data on board in real-time and then dump it at the end 
of the run.  The command on the PC end that fetches the data also makes 
a gnuplot "*.plot" file and invokes gnuplot with that as an argument.  
So I type in "get_log" and a graph pops up.

The graph that Bob Cook sent is exactly what we were looking for.  
(thanks Bob!).   A couple of comments. Note that his PID is responding 
to a large step function, no slewing at all of the input.  This is the 
test that you want to run.   Disable any input slewing.  It just makes 
the PID tuning harder.

It's useful to reflect that the PID controller must be able to react 
quickly to step changes in the environment, even if the control input 
itself is correlated and smooth.  So running on a rough surface, or when 
a wheel drops into a depression for example, the feed back velocity 
coming from the motor encoders may change a lot, and change rapidly.   
That means the proportional error will also change a lot and change 
rapidly, even if the control input is smooth.  The PID controller has to 
be able to react quickly to counter those changes.

So acceleration and deceleration ramps that present a smoothly changing 
input to the controller can't be implemented by making the PID 
controller itself sluggish overall. Or rather, they can, but it's 
probably not what you want  :)

Every robot builder should have that Flynn/Jones book on their shelf.  A 
classic.  If you haven't already seen it there is another, later book by 
Joseph Jones, "Robot Programming: A Practical Guide to Behavior Based 
Robotics" which is also excellent, a bit more technical and focused on 
subsumption.

onward!
dpa



On 10/3/20 2:53 PM, Murray Altheim via DPRGlist wrote:
> On 4/10/20 5:33 am, David Anderson via DPRGlist wrote:
> [...]
>> It's actually pretty trivial to use gnuplot if you are comfortable
>> with Linux.  Here's what I did to plot your log data. First, data
>> from your html file were cut and pasted to a flat ASCII file named
>> "murray_log-20200929.txt."
>
> Hi David,
>
> I've been a daily Linux user since the mid-90s, and UNIX before that,
> so the command line is a happy place for me.
>
>> Then, because Linux uses white space as a column separator by 
>> default, a few places had to be modified which was done like this:
>>
>> $ cat murray_log-20200929.txt | grep error | sed 's/=/= /g'
>>
>> which passes through the lines which have the word "error" in them 
>> and extends the "error=" and "D=" labels by an extra space so they 
>> are seen by the parser as separate fields.
>
> Ahh yes.
>
>> That output was piped to a gawk command that printed out only the 
>> data values of interest and not the accompanying ASCII labels:
>>
>> $ cat murray_log-20200929.txt | grep error | sed 's/=/= /g' | gawk 
>> '{print $1,$2,$7,$9,$11,$13,$15,$17}'
>>
>> Then a separate file was created for the left and right motors by 
>> grep'ing on the terms "pid:port" and "pid:stbd" so the whole thing 
>> for the port motor looks like this:
>>
>> $ cat murray_log-20200929.txt | grep error | sed 's/=/= /g' | gawk 
>> '{print $1,$2,$7,$9,$11,$13,$15,$17}' | grep "pid:port" > port.dat
>>
>> The process of graphing these data with gnuplot is then just a matter 
>> of plotting the fields of interest with gnuplot's "plot" command:
>>
>> gnuplot> plot "port.dat" using ($1):($3) title "Error" with lines, \
>>          "port.dat" using ($1):($4) title "P " with lines, \
>>          "port.dat" using ($1):($7) title "Setpt" with lines, \
>>          "port.dat" using ($1):($8) title "Output" with lines
>>
>> Each of these lines uses the first field, the line number, as the 
>> horizontal X value, and the given field as the Y value.  If the log 
>> file is created with a time stamp as the first entry and white space 
>> (or commas) as the field separators, then none of the data 
>> preprocessing steps would be necessary,  and the graphs could be 
>> generated by gnuplot directly from the log files with just a single 
>> command, as above.
>
> Well, I may be a daily Linux user and familiar with all those tools, but
> you've still made quick work of it, very nice job. Seeing it all laid out
> like that is very helpful.
>
> I'm trying to think whether to go with the Grafana idea or GnuPlot at 
> this
> point, as Grafana is very good at plotting time series data.
>
> https://grafana.com/docs/grafana/latest/getting-started/timeseries/
>
> I had a little experience with this kind of thing when working at 
> MetService,
> the NZ weather service. Shame I didn't jump into more projects that would
> have amplified those skills. There were some serious experts there who 
> could've
> lent a hand.
>
> I've cleaned up my log now as per Chris' recommendation, so I'm 
> generating
> clean CSV with millisecond timestamps and all the data for both PIDs 
> on one
> line. I started on this journey just printing it in color to sysout so I
> could "manually" view what was happening, but obviously that's nowhere 
> near
> as useful as actually graphing it, especially when we're trying to 
> compare
> multiple values, and looking at this motion profile idea on my "one meter
> behaviour".
>
> It's a Sunday here and for a change not raining (though we're expecting
> gales but my back yard is sheltered a bit) so I may have to do some 
> firewood
> chopping and get out the power tools and start to put together my cypress
> wood garden boxes. Otherwise Spring will be over before we ever plant any
> vegetables... so hopefully later I can do a bit more robotic works.
>
> Oh, an almost pristine copy of the 1st edition Jones/Flynn "Mobile 
> Robots:
> Inspiration to Implementation" book arrived in the mail yesterday, thanks
> for the recommendation. It turns out I had seen this book before, I think
> I'd checked out a copy from the Wellington Central Library (now condemned
> and to be bowled due to severe damage from the last earthquake). As it's
> from 1993 the tech is old school but the ideas still very useful. I don't
> think when I'd looked at it before the concept of a PID controller would
> have had any currency, funny it's central to getting my robot running 
> now.
>
> Maybe I've learned a bit.
>
> Cheers,
>
> Murray
>
> ........................................................................... 
>
> Murray Altheim <murray18 at altheim dot com>                       = 
> =  ===
> http://www.altheim.com/murray/ ===  ===
> = =  ===
>     In the evening
>     The rice leaves in the garden
>     Rustle in the autumn wind
>     That blows through my reed hut.
>            -- Minamoto no Tsunenobu
>
> _______________________________________________
> DPRGlist mailing list
> DPRGlist at lists.dprg.org
> http://lists.dprg.org/listinfo.cgi/dprglist-dprg.org