[Dprglist] Optical flow sensors - down the rabbit hole

[John Swindle]

Karim and Murray,
This may have already been in your research, but I didn't see it in your postings:
Using lenses to allow optical mouse encoders to operate at greater distances:
"Toward Refocused Optical Mouse Sensors forOutdoor Optical Flow Odometry"
www.researchgate.net/publication/234016792_Toward_Refocused_Optical_Mouse_Sensors_for_Outdoor_Optical_Flow_Odometry
Although I don't care about video, I took a course on stroboscopy, took pictures of bullets cutting cards in a dark room (kinda scary), developed and printed film, and wrote an automated test for EGA. So, I kinda understand. I just don't care.
It seems the Agilent or TI DSP functions that are used in their products could be applied to a heavily pixelated camera. I think the functions are openly available. I read one of those programs a decade or so ago. Don't know where it is now.
As the article above says, the image is different than just looking at a flat surface. But whatever tricks they show in the article ought to work for a camera.
Back to audio now.
John Swindle


-----Original Message-----
From: Karim Virani via DPRGlist <dprglist at lists.dprg.org>
To: dprglist <dprglist at dprg.org>
Sent: Wed, Jul 1, 2020 8:35 pm
Subject: [Dprglist] Optical flow sensors - down the rabbit hole

Hey Murray,
I kinda went down the rabbit hole with the optical tracking sensor you shared in RBNO last night. It looked really promising and might still be - I'm not sure.
So you've been experimenting with the Pimoroni optical flow sensor for odometry:https://shop.pimoroni.com/products/pmw3901-optical-flow-sensor-breakout 
which in turn is based on the PMW3901 PixArt sensor:https://www.pixart.com/products-detail/44/PMW3901MB-TXQT  
 
Issues with this sensor are partly related to its intended usage - this is optimized for drones and their typically greater distance from the ground. Also this usage is not really odometry. It's more optimized for detecting velocity rather than distance . Technically those should be convertible - but that's in a perfect world independent of the kinds of error that might arise.

You mentioned the possibility of switching the sensor a similar one from the same manufacturer, the PAA5100JE-Q:https://www.pixart.com/products-detail/74/PAA5100JE-Q 
This one looks optimized for indoor floor driving robots like vacuum bots. It's tuned to the kinds of surfaces we (DPRG) often see. Note that it has some sister products that might work better on highly glossy floors (like in the Perot), but they are a different form factor and have a larger minimum distance.
You noted that this sensor might be an easy drop in replacement for the existing Pimoroni carrier board and so asked Pimoroni to look into that. I did a +1 vote on that and if anyone else is interested, add your vote here:https://forums.pimoroni.com/t/request-new-floor-tracking-sensor-based-on-pmw3901-breakout/14360  

Not wanting to wait on a decision from Pimoroni, I looked for a carrier board that could accommodate the LGA package and found this one:
http://www.proto-advantage.com/store/product_info.php?products_id=3100108  

But then I took a closer look at the datasheet for the PAA5:https://www.pixart.com/_getfs.php?tb=product&id=74&fs=ck2_fs_en

and I noticed it needed a ground plane in the middle which that breakout doesn't have. It also requires a minimal amount of external components to work according to the reference schematic and that also is a reason the Pimaroni option looks good if they choose to support it. 
However it might not be a perfect fit for my team. At 45 inches per second maximum tracking speed, it would handle most of the club's robots and most FTC robots. But it wouldn't cover the speediest of FTC robots. Patrick's team would likely be pushing that speed limit. And FRC robots would definitely be too fast.
Deepening the rabbit hole, I found another sensor by PixArt - the PMT9101DM-T2QU which is more closely related to optical mouse history and I've actually used a variant of this one.https://www.pixart.com/products-detail/69/PMT9101DM-T2QU 
This part solves one of my concerns - it has a much greater detection speed.  This is a motion tracker that's been optimized for industrial applications like tracking paper passing through high speed copiers. So it can handle much greater velocities and accelerations.

And then I found a paper on using that sensor to detect/control smart hydraulic cylinders which did some performance analysis:https://www.politesi.polimi.it/bitstream/10589/114682/3/2015_12_CamposAlache.pdf

And this brings up some other related concerns. The PAA5 specs have this line:
Repeated Accuracy Error at Nominal Height: 3% to 10% (3% on carpet & 6% on laminated wood)

Now what does that mean? It's really hard to say, but my initial reaction is not great. That seems like a lot more error than we get with free-wheeling odometry.
PixArt uses that Repeated Accuracy Error for most of their related products, although they aren't necessarily specified in the same way. For example the high speed PMT9101DM-T2QU claims:Repeated Accuracy Error at Nominal Height ±0.25mm over 25mm
Which is 1%, so better. The improvement might be related to the much more constrained range to surface detected. But 1% is still problematic. 
Going back to the hydraulic motion paper linked above, it studies this sensor specifically. I can also recommend that paper because it has a pretty good general treatment of optical flow and optical mouse sensors.
It turns out that according to this paper, there is a problem with measurement drift in these sensors. And it seems to be susceptible to changes in direction. The paper goes on to explore a correction algorithm that achieves good results, but that is dependent on placing a correction marker in the surface being tracked and that can't be done when we are tracking motion across an arbitrary and unknown surface.
One other concern for my team is that our hardware doesn't have SPI, so we'd probably need to incorporate an I2C to SPI bridge.
I'm still interested to test these out. But I now have limited expectations about the buildup of error in absolute distances travelled.

_______________________________________________
DPRGlist mailing list
DPRGlist at lists.dprg.org
http://lists.dprg.org/listinfo.cgi/dprglist-dprg.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.dprg.org/pipermail/dprglist-dprg.org/attachments/20200715/ef7375bc/attachment.html>