Writing a new backend / Agnosticity

[naikymen]

About serializing the “state”.

@rickwierenga

Having them in the same file would make source control and state management harder.

I agree, the issue is that we have drawn the line for what is “static” or “mutable” in different places.

In my case, the only static things are the shapes of labware (tips, racks, plates, etc), the shape of the deck (and any slots in it), and the parameters of tools, which you could get from datasheets. Then, all of the mutable stuff exists in the instance of a deck: locations, volumes, etc.

Given this, I’d ask how to get a file for the “whole state”, that can be linked back (hopefully easily) to the resources in the “regular” serialization. I have only seen hints of this in the code. Are there docs for it i can read?

@rickwierenga

Since tips are moved around all the time and discarded, it did not make sense to me to make them Resources. We have TipSpot instead, as children of TipRack, because they are actually fixed on the deck.

Hmm, this is part of the life of well-plates too, and can be said of many other kinds of plastic labware.

A Tube is considered to be a resource, even though it does the same things that a Tip does: holds liquid, can be moved around, can be placed in a “spot”, can be discarded, will fit differently in different tip racks, and maybe more stuff. Perhaps the Tip should be a Container even.

As a side note, I’d also argue that nothing in the “deck” is fixed, it is actually the part that is meant to change. Consider, for example, moving resources around: all it takes is a gripper, or a person, to move a plate as part of a protocol.

The only things that are fixed are wells in a plate. Or are they…?

@rickwierenga

For TubeRack, probably, because tubes are sometimes moved around. Do you want to PR this?

For Plate, I think it’s fine to stick with Well. (There is no moving around wells)

Yes I will PR.

And yes, granted, it sounds strange to allow Wells to move around, but i see it as an exception in a more general case.

What will happen when the robot gains a CO2 laser tool, and cuts out wells from the plate (!?).

@rickwierenga

Storing the information would only work for regular labware, introducing two cases of 1) regular and 2) irregular labware. Simpler to treat everything as irregular imo.

Ok, I understand. What I would like to have in PLR is a property that saves the abstract parameters inside ItemizedResource, and makes the information available by serialization. You already have some of these “abstract” parameters as properties (num_items_x and num_items_y) but most are missing.

What we have implemented is regular labware, and then added a “CUSTOM” type, which has spots for containers in arbitrary locations. I could use this for PLR resources instead of my usual types, which would make things simpler at the cost of making PLR decks a second-class citizen in the UI (for now).

[rickwierenga]

I like to look at physical reality, and model that as correctly as possible in the level of detail we require. I would say our users are developers, we try to design for them (imo ‘automation engineers’ are software developers). If you can’t use the PLR serialization standard, I’d love to learn why and then make sure that you can!

(I firmly believe standards follow implementation, so let’s refine the implementation)

[rickwierenga]

@naikymen

In my case, the only static things are the shapes of labware (tips, racks, plates, etc), the shape of the deck (and any slots in it), and the parameters of tools, which you could get from datasheets. Then, all of the mutable stuff exists in the instance of a deck: locations, volumes, etc.

We are pretty similar. The shapes/parameterization you’re describing are encoded as functions in PLR that will return a class, the general structure of labware is stored in classes (like TipRack). This is all easy to serialize into JSON. For a particular plate, the locations of wells are always fixed in that plate (hence relative location) and they don’t have an absolute location (‘absolute’ locations, wrt the deck, are computed on demand)

For decks, which are (roughly) constant at the start of an experiment, those are essentially higher level resources, where carriers/plates/tip racks are children. They must have a pre-defined location for an experiment to start. (but not to write or define an experiment.)

The state is what changes during an experiment run.

But I agree that this is somewhat ambiguous because plates often change location during a run. Serializing a deck without locations doesn’t really make sense to me (what labware you require is essentially encoded in the protocol file). But if there’s a better way to implement this, let’s do that.

@naikymen

Given this, I’d ask how to get a file for the “whole state”, that can be linked back (hopefully easily) to the resources in the “regular” serialization. I have only seen hints of this in the code. Are there docs for it i can read?

we do actually have a little bit of documentation on serializing state: Resources Introduction — PyLabRobot documentation.

You can get the whole state with like deck.serialize_all_state() or workcell.serialize_all_state(). Or plate.serialize_all_state().

@naikymen

A Tube is considered to be a resource, even though it does the same things that a Tip does: holds liquid, can be moved around, can be placed in a “spot”, can be discarded, will fit differently in different tip racks, and maybe more stuff. Perhaps the Tip should be a Container even.

I get your point. Maybe we should try making Tip a Resource, it would be more consistent. The location of a Tip should probably be None when it is mounted on a pipetting channel (for many robot’s it’s hard to continuously keep track of the exact location in the background, involves a lot of querying, if at all possible).

@naikymen

And yes, granted, it sounds strange to allow Wells to move around, but i see it as an exception in a more general case.

Since it’s an exception, this behavior can’t really be abstracted in ItemizedResource (subclasses should extend behavior but not limit it).

@naikymen

What will happen when the robot gains a CO2 laser tool, and cuts out wells from the plate (!?).

Unlikely to happen during a protocol, but if you have wells laying around nothing stops you from assing a Well object to any parent.

@naikymen

Ok, I understand. What I would like to have in PLR is a property that saves the abstract parameters inside ItemizedResource, and makes the information available by serialization. You already have some of these “abstract” parameters as properties (num_items_x and num_items_y) but most are missing.

I don’t like saving because we would have two sources of truth. They can be dynamically computed properties. For serialization, that should not contain duplicate information. What if these values conflict? We can easily derive them from the general case.

@naikymen

What we have implemented is regular labware, and then added a “CUSTOM” type, which has spots for containers in arbitrary locations. I could use this for PLR resources instead of my usual types, which would make things simpler at the cost of making PLR decks a second-class citizen in the UI (for now).

why not make everything ‘CUSTOM’ by default?

[naikymen]

@rickwierenga

If you can’t use the PLR serialization standard, I’d love to learn why and then make sure that you can!

Except for the details mentioned above, our data “standards” are mostly equivalent: they contain the same information in a different shape. I think I could use it, the same way that PLR could serialize to/from my schemas instead (with some work).

As far as I know, only PLR is using the PLR serialization format. Is this accurate? To me this means that, for now, the gain of migrating my programs to it would be equal to the gain of writing bridges between them (which is easier, and mostly done done by now).

Besides that, there are only a few things that make it less appealing than what we have right now.

• Differences discussed above, for example: in the parametrization of labware (saving rack parameters, the equivalence between racks and between containers, …), unclear support for multi-tool machines, what counts as state and what does not, etc.
• The definition of a PLR protocol is a python script, but we use JSON. This means two things: (1) PLR can make “decisions” based on new data, and, because of that, (2) there is no serializable representation of such a protocol.

The bits about tools and protocols are the most concerning from my perspective. I actually like the difference in protocol representation, because I can have the best of both worlds: static protocols for routine tasks, and dynamic PLR protocols for a self-driving lab.

All this considered, I feel that building bridges is what’s important. If stuff is open and interoperable, all will be well.

[naikymen]

@rickwierenga

we do actually have a little bit of documentation on serializing state: Resources Introduction — PyLabRobot documentation.

You can get the whole state with like deck.serialize_all_state() or workcell.serialize_all_state(). Or plate.serialize_all_state().

Thanks a lot!

@rickwierenga

get your point. Maybe we should try making Tip a Resource, it would be more consistent.

I almost forgot something.

I have separate definitions for racks and for tips, which are linked relationally (in DB terms). This is because there is a many to many relationship between them.

Is this contemplated in PLR? Or are tips in PLR only meant for a particular rack? (and vice-versa).

@rickwierenga

The location of a Tip should probably be None when it is mounted on a pipetting channel (for many robot’s it’s hard to continuously keep track of the exact location in the background, involves a lot of querying, if at all possible).

Sounds good.

@rickwierenga

They can be dynamically computed properties.

Yes please

@rickwierenga

why not make everything ‘CUSTOM’ by default?

Definitely a possibility. I was considering this for the resource types that I have no definition for.

The issue of using it for racks and plates is that the behavior of column-row selection becomes undefined.