[Dprglist] Visualizing data [was: PID]

[Murray Altheim]

On 1/10/20 9:25 am, Bob Cook via DPRGlist wrote:
> Hi Murray,
> 
> You may want to look at Grafana as a tool to visualize time series
> data. It is open source and easy to set up if you are familiar with
> Docker containers. It pulls data from a variety of sources, I’m 
> using it with an Elasticsearch instance, also as a Docker container.
> I set them up on a Linux host with Docker, but apparently you can
> use Windows or macOS as your Docker host pretty easily too.

Hi Bob,

I've as part of various jobs designed and implemented both Solr and
ElasticSearch services but I've never tried standing up a Docker
instance nor have I used Grafana, though I've looked over the website
a few times. I'm a Linux user so yeah, it doesn't sound like it'd be
too difficult to attempt what you've done, maybe easier than trying
to do it the hard way, which is kinda what I'd done on my version 1
PID controller: write the data from the PID to a file on the robot,
then either on a HDMI-connected monitor to the Raspberry Pi or on my
workstation computer doing the GnuPlot visualisation. That took a
fair bit of effort so if Grafana is substantially easier and more
configurable I might give it a try. Good learning experience both for
my robotics as well as job skills too.

> I’m using Python to push data into the ES instance. There is a Python
> lib for easy access to ES. My robot records a binary stream of stats
> that I convert and upload offline, after I’ve collected data from 
> a test run.

Wow, thanks for that clue -- Python to ElasticSearch would be a pretty
huge simplification if the ElasticSearch to Grafana connection were
similarly easy. But again, a good learning experience in either case.

If I were doing a microservice architecture between powerful services
on a fast network I might want to send all the log messages directly
to ElasticSearch, but considering this is running on a Raspberry Pi
over WiFi and I don't need the data to be live-streamed, I'll likely
think about how to load a completed log (as CSV, as Chris suggested)
after the fact.

Thanks much,

Murray

...........................................................................
Murray Altheim <murray18 at altheim dot com>                       = =  ===
http://www.altheim.com/murray/                                     ===  ===
                                                                    = =  ===
     In the evening
     The rice leaves in the garden
     Rustle in the autumn wind
     That blows through my reed hut.
            -- Minamoto no Tsunenobu