[Dprglist] Visualizing data [was: PID]

[Bob Cook]

HI Murray,

Grafana does pretty slick graphs without much work at all. Here is an example image of a (not very good) PID implementation showing the target speed, actual speed, and error. As you can see this is going from speed=0 to speed=20 then speed=0 again after 30 seconds.



Happy to share the detail of my Docker setup if that would help. I’m using Docker Compose to create three containers: ElasticSearch, Kibana, and Grafana. The three containers share a private network, and the data for each resides on the host rather than inside the containers themselves - makes upgrading the containers easy without losing the data. Sort of a beautiful thing made simple by Docker.

- Bob


> On Oct 1, 2020, at 1:24 AM, Murray Altheim via DPRGlist <dprglist at lists.dprg.org> wrote:
> 
> On 1/10/20 9:25 am, Bob Cook via DPRGlist wrote:
>> Hi Murray,
>> You may want to look at Grafana as a tool to visualize time series
>> data. It is open source and easy to set up if you are familiar with
>> Docker containers. It pulls data from a variety of sources, I’m using it with an Elasticsearch instance, also as a Docker container.
>> I set them up on a Linux host with Docker, but apparently you can
>> use Windows or macOS as your Docker host pretty easily too.
> 
> Hi Bob,
> 
> I've as part of various jobs designed and implemented both Solr and
> ElasticSearch services but I've never tried standing up a Docker
> instance nor have I used Grafana, though I've looked over the website
> a few times. I'm a Linux user so yeah, it doesn't sound like it'd be
> too difficult to attempt what you've done, maybe easier than trying
> to do it the hard way, which is kinda what I'd done on my version 1
> PID controller: write the data from the PID to a file on the robot,
> then either on a HDMI-connected monitor to the Raspberry Pi or on my
> workstation computer doing the GnuPlot visualisation. That took a
> fair bit of effort so if Grafana is substantially easier and more
> configurable I might give it a try. Good learning experience both for
> my robotics as well as job skills too.
> 
>> I’m using Python to push data into the ES instance. There is a Python
>> lib for easy access to ES. My robot records a binary stream of stats
>> that I convert and upload offline, after I’ve collected data from a test run.
> 
> Wow, thanks for that clue -- Python to ElasticSearch would be a pretty
> huge simplification if the ElasticSearch to Grafana connection were
> similarly easy. But again, a good learning experience in either case.
> 
> If I were doing a microservice architecture between powerful services
> on a fast network I might want to send all the log messages directly
> to ElasticSearch, but considering this is running on a Raspberry Pi
> over WiFi and I don't need the data to be live-streamed, I'll likely
> think about how to load a completed log (as CSV, as Chris suggested)
> after the fact.
> 
> Thanks much,
> 
> Murray
> 
> ...........................................................................
> Murray Altheim <murray18 at altheim dot com>                       = =  ===
> http://www.altheim.com/murray/                                     ===  ===
>                                                                   = =  ===
>    In the evening
>    The rice leaves in the garden
>    Rustle in the autumn wind
>    That blows through my reed hut.
>           -- Minamoto no Tsunenobu
> 
> _______________________________________________
> DPRGlist mailing list
> DPRGlist at lists.dprg.org
> http://lists.dprg.org/listinfo.cgi/dprglist-dprg.org

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dprg.org/pipermail/dprglist-dprg.org/attachments/20201002/0abb374f/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: PastedGraphic-1.png
Type: image/png
Size: 96959 bytes
Desc: not available
URL: <http://lists.dprg.org/pipermail/dprglist-dprg.org/attachments/20201002/0abb374f/attachment-0001.png>