@rickwierenga and I had some conversations about whether people might be interested in a direct integration of cameras into PLR?
Reasoning: There are many ways of connecting cameras to Python. So many actually that it can be hard to just find one simple one that captures images directly in your script.
This is incredibly useful for image processing and image-based feedback in your automation, e.g. ultra low-cost 2D barcode readers, deck verification, labware transfer verification, etc.
OpenCV is one of the most common options but can be a dependency nightmare and is definitely not beginner friendly.
I’ve built a simple imageio-based implementation for fetching images that my company is happy to share with the PLR open-source community.
But we were wondering who would be interested in this?
love the idea of an easy to use python camera lib, but simultaneously wondering whether this should be in the scope of PLR. in any case, thank you for be willing to open source this! happy to discuss
interesting! we would definitely use it if it was plr, but we could also split camera work to pynopticon without much issue
direct camera integration might open the can of worms to error correction & state resolution, which could be out of scope for plr in general (I hear this is hard to solve, even for specific methods, so I expect a fully general solution that is helpful will be challenging)
I’m also leaning towards saying this is outside the scope of PLR itself. But: super useful to have this as an adjacent open source project.
I think Pynopticon should be camera agnostic, and could maybe use Camillo’s new camera library? I’m not sure Pynopticon really is the best place for camera drivers.
In general, it would be best if we could find an existing project that does most/all of what we want.
Interesting! I need to look more into pynopticon (love the name btw).
One potential reason for integrating a camera directly into PLR is to handle sample management completely in PLR.
Liquid handlers usually have no barcode scanner or a 1D barcode scanner, and sometimes a “2D barcode” scanner. Some robots can, of course, be bought with a “2D barcode” scanner, aka a camera with a barcode segmentation and encoding algorithm. But buying them retroactively is usually not possible. A simple camera and some image processing can change that.
Then the question becomes whether sample management beyond what an in-built barcode reader can provide should be in the scope of PLR.
I wouldn’t call wrapping some imageio functions in some logic a library
imageio is indeed camera agnostic though (at least for multiple USB cameras), and I agree it might be better to build out pynopticon and make it work fast to enable real-time image processing for 2D barcode reading and autonomous decision making.
I agree that using cameras with PLR is very useful, but that’s not a conclusive argument for building them into the library. (pandas is also useful with PLR)
pynopticon is for retroactively recording errors, and not useful/designed for this use case.
Fair point; I was trying to emphasise with the 2D barcode example that actually some liquid handlers already have a camera integrated and PLR will need to expose it once a user joins PLR with a 2D barcode scanner integrated into their liquid handler.
But I guess you are right and people who haven’t bought a liquid handler with a 2D barcode reader can still integrate cameras themselves and write their own cheap, efficient barcode reader.
The LabEye software by Robiotec can work with any camera and provide vision based detection for any liquid-handler or Lab-Automation system.
See here:
API is super easy to integrate on Python and on any automation software with no need for driver.
If anyone would like to see a demo please email me at shaik@robiotec.com
Thank you for the info @Shaik.
I’ll email you early next week for a demo.
I have seen LabEye in a different post on this forum about camera on deck integration but couldn’t find a pricing model on the website, and have not found the time to chase someone from robiotec for more information.
In regards to Rack2D, this looks very interesting.
But is the camera system fixed in place, how does it identify far away objects, and what price range is it in?
I have quite a bit of computer vision experience and don’t mind building my own solutions but am always looking for ways to get a job done faster.
