Evo Advanced Worklist commands don't work with some characters

Tecan
1

Hi Friends,

TL:DR - The ascii character 160 (non-blocking space) works fine in well selection strings in .esc files but not in .gwl files.

I’m writing some python code that will generate worklists for a particular protocol we are wanting to automate. The plate layouts change from run to run and I thought it would be easiest to set it up as a worklist. And in order to be efficient with my tips I am using the advanced aspirate and dispense commands.

These commands take a well selection string that is composed of ascii characters with each representing 7 wells on the plate. These are assembled as a bitfield and then the arbitrary value 48 is added to make sure that they are all printable characters.

However this puts some of the possible values above 127 which are also not regular ascii characters. In my particular issue I am trying to aspirate from the second column of a 4 row by 6 column plate. When I do this selection string I have a binary 01110000 which is 112. I then add 48 and this gives 160. That as an ascii character is a non-blocking space character. So the string would look like “0604 100” where that space is actually NBSP (ascii 160).

I opened the file up in binary mode to look and see what value it had in that position and this is what I see.

00000410 31 2C 22 30 36 30 34 A0 31 30 30 22 2C 30 2C 30 1,“0604 100”,0,0

That A0h is the character in question. Whenever that character is there I get this error when I try to execute the worklist “Cannot evaluate expression 0604 100”.

I decided to try to see what happens if I let Tecan software write the same step so I created a protocol that included the same step. I then opened the .esc file and to my surprise found the same character working just fine in the well selection string there.

(Showing 2 lines because the string wraps. The offending char is first char in second line)
00000840 30 2C 30 2C 31 36 2C 30 2C 31 2C 22 30 36 30 34 0,0,16,0,1,“0604
00000850 A0 31 30 30 22 2C 30 2C 30 29 3B 0D 0A 41 73 70 100”,0,0);…Asp

So it is still using A0h and here it is working just fine and EVOware has no problem parsing the string.

I’ve checked the encoding. Both files are using ansi encoding and the same \r\n windows style line endings.

I’m sort of up against a brick wall here. 160 is one of the possible values for a character in that well selection string, but for some reason EVOware won’t parse that for some reason.

Does anyone have any ideas where to go next?

2

I also get an error that says:
“labware “” not defined” and I’m not sure where that is coming from. Here is a copy of the worklist I have:

W;
B;GetDiTi2(255,"DiTi 200ul Filter LiHa",1,0,0,70);
B;Aspirate(255,"IDeA NODET","100.00","198.25","194.51","197.71","195.90","197.79","198.03","196.38",1,0,1,"0108¯1",0,0);
B;Dispense(255,"IDeA NODET","100.00","198.25","194.51","197.71","195.90","197.79","198.03","196.38",22,0,1,"0C08¯1000000000000",0,0);
B;Aspirate(255,"IDeA NODET","198.12","198.36","197.89","196.45","191.80","197.19","198.81","198.50",1,0,1,"0108¯1",0,0);
B;Dispense(255,"IDeA NODET","198.12","198.36","197.89","196.45","191.80","197.19","198.81","198.50",22,0,1,"0C080®300000000000",0,0);
B;Aspirate(255,"IDeA NODET","198.41","182.14","197.92","183.33","193.42","198.50","191.23","197.13",1,0,1,"0108¯1",0,0);
B;Dispense(255,"IDeA NODET","198.41","182.14","197.92","183.33","193.42","198.50","191.23","197.13",22,0,1,"0C0800¬70000000000",0,0);
B;DropDiTi(255,1,6,0,70,0);
B;GetDiTi2(255,"DiTi 200ul Filter LiHa",1,0,0,70);
B;Aspirate(15,"IDeA NODET","100.00","1.75","5.49","2.29",16,0,1,"0604?000",0,0);
B;Aspirate(240,"IDeA NODET","4.10","2.21","1.97","3.62",16,0,1,"0604 100",0,0);
B;Dispense(255,"IDeA NODET","100.00","1.75","5.49","2.29","4.10","2.21","1.97","3.62",22,0,1,"0C08¯1000000000000",0,0);
B;DropDiTi(255,1,6,0,70,0);
B;GetDiTi2(255,"DiTi 50ul Filter LiHa",1,0,0,70);
B;Aspirate(15,"IDeA NODET","1.88","1.64","2.11","3.55",16,0,1,"06040N00",0,0);
B;Aspirate(240,"IDeA NODET","8.20","2.81","1.19","1.50",16,0,1,"0604030",0,0);
B;Dispense(255,"IDeA NODET","1.88","1.64","2.11","3.55","8.20","2.81","1.19","1.50",22,0,1,"0C080®300000000000",0,0);
B;DropDiTi(255,1,6,0,70,0);
B;GetDiTi2(255,"DiTi 50ul Filter LiHa",1,0,0,70);
B;Aspirate(15,"IDeA NODET","1.59","17.86","2.08","16.67",16,0,1,"060400l0",0,0);
B;Aspirate(240,"IDeA NODET","6.58","1.50","8.77","2.87",16,0,1,"060400p7",0,0);
B;Dispense(255,"IDeA NODET","1.59","17.86","2.08","16.67","6.58","1.50","8.77","2.87",22,0,1,"0C0800¬70000000000",0,0);
B;DropDiTi(255,1,6,0,70,0);

I don’t know how to attach the .esc file or I would share that as well. If someone has an idea I’m all ears.

3

This was an interesting problem when I tackled it. You run into characters that you can’t type, so you’re unable to create a dictionary for all possible options.

What I had done was created functions to go at it programmatically. Looks like you figured out the math portion for adding wells and already figured out you needed the correct encoding.

It’s hard to see without your code but I believe the last portion is that you need to write everything in bytes. If you go at it with strings, it throws off the encoding when you write to the file. The file also needs to be written into in byte mode.

It took me so long to figure out, but when it worked it felt like magic.

Here’s an example of my python code for creating an advanced worklist command for a gwl:

"""Module example to demonstrate advanced worklist commands."""

from math import ceil

TABLE_STRING = "0C08"  # Dimensions code for 96 well labware
TO_BIT = {1: 1, 2: 2, 3: 4, 4: 8, 5: 16, 6: 32, 7: 64, 8: 128}
FROM_BIT = {1: 1, 2: 2, 4: 3, 8: 4, 16: 5, 32: 6, 64: 7, 128: 8}


def get_int_from_char(character) -> int:
    """Returns ordinal value - 48 from character."""
    if isinstance(character, int):
        return character - 48
    else:
        return ord(character) - 48


def get_char_from_int(number: int) -> bytes:
    """Returns character from integer + 48."""
    return bytes((number + 48,))


def get_well_from_string(string: str) -> int:
    """Returns well from the 14 character string encoding."""
    well = 0
    for char in string:
        value = FROM_BIT[get_int_from_char(char)]
        if value:
            well += value
            break

        # Every left-sided 0 place holder holds value of 7
        well += 7

    return well


def get_string_from_well(well: int) -> bytes:
    """Returns 14 character string encoding from well."""
    string = bytes("", "cp1252")
    for _ in range(14):
        if well == 0:
            string += bytes("0", "cp1252")
        elif well > 7:
            string += bytes("0", "cp1252")
            well -= 7
        else:
            # Value will be 1 - 7
            # Apply bit encoding and return character
            string += get_char_from_int(TO_BIT[well])
            well = 0

    return string


def add_wells(wells: "list[int]", dimensions: str = "") -> bytes:
    """Returns the string sum from list of wells."""
    strings = [get_string_from_well(well) for well in wells]
    if not dimensions:
        dimensions = TABLE_STRING
    summation = bytes(dimensions, "cp1252")
    values = [[] for _ in range(14)]
    for string in strings:
        for i, char in enumerate(string):
            number = get_int_from_char(char)
            values[i].append(number)

    for number in values:
        summation += get_char_from_int(sum(number))

    hex1 = dimensions[0] + dimensions[1]
    hex2 = dimensions[2] + dimensions[3]
    table_dimensions = (int(hex1, 16)) * (int(hex2, 16))
    char_nums = ceil(table_dimensions / 7)
    summation = summation[: char_nums + 4]

    return summation


def make_advanced_pipette_command(
    action: str,
    tip_mask: int,
    liquid_class: str,
    volume: list,
    grid: int,
    site: int,
    well: bytes,
    spacing: int = 1,
) -> bytes:
    """Advanced worklist command for dispense.

    Parameters
        action: selected action of aspirate or dispense
        tip_mask: selected tips, bit-coded (1 - 128)
        liquid_class: liquid class name
        volume: 12 expressions which specify the volume for each tip
        grid: carrier grid position
        site: labware location starting at 0
        well: bit-coded well selection
        spacing: tip spacing

    Returns:
        command: advanced worklist command
    """
    volumes = ""
    for vol in volume:
        if vol:
            volumes += f'"{vol}",'
        else:
            volumes += '"0",'

    command = bytes(f"B;{action.title()}(", "cp1252")
    command += bytes(f'{tip_mask},"{liquid_class}",{volumes}', "cp1252")
    command += bytes(f'{grid},{site},{spacing},"', "cp1252")
    command += well + bytes('",0,0);', "cp1252")

    return command


if __name__ == "__main__":

    action = "dispense"
    tip_mask = 128  # Bit sum for all tips: 1 = tip 1, 2 = tip 2, 4 = tip 3, etc...
    liquid_class = "Water Free Dispense"
    # Include volumes for tips that don't exist. Must be 12 volumes.
    volume = [100, 200, 300, 400, 500, 600, 800, 900, 0, 0, 0, 0]
    grid = 12
    site = 2
    dispense_wells = [1, 2, 3, 4, 5, 6, 7, 8]

    well = add_wells(dispense_wells)

    commands = []
    command = make_advanced_pipette_command(
        action,
        tip_mask,
        liquid_class,
        volume,
        grid,
        site,
        well,
    )

    print(command)

    commands.append(command)

    # Open the file in binary mode and write the command
    with open("worklist.gwl", "wb") as file:
            for line in commands:
                # Write each command as bytes to the file
                file.write(line + bytes("\r\n", "cp1252"))
3 Likes
4

Thanks for the reply. I’m doing things the python 3 way with the encoding set on the file object and everything gets converted to byte when they get written. I’ll try encoding everything on the fly but I don’t think that will make any difference. I’ve got the hex dump and I think it is writing the correct bytes.

I noticed that you’re using cp1252 encoding, which is one that I haven’t tried. Perhaps that will be the key. When I open up the .esc file in notepad it tells me that it is ANSI encoded, but I think that can be wrong sometimes. I’ll try with cp1252 tomorrow and then I’ll try your code just as is and see how the output differs.

This is the code that I’m using:

from PlateModel import *
from SampleModel import *
import csv


class WorkList():

    def __init__(self):
        self.records = []

    def selectTip(self, tip_index):
        self.records.append(f"S;{tip_index}")

    ####  These are the simpler worklist commands.  They all contain *Record in the name.
    ####  These are easy to build with for simple things, but they aren't tip efficient. 

    def addTransfer(self, asp, disp):
        self.addAspirateRecord(**asp)
        self.addDispenseRecord(**disp)
        self.addWashRecord()
        
    def addWashRecord(self):
        self.records.append("W;")

    def addBreakRecord(self):
        self.records.append("B;")

    def addAspirateRecord(self,
            rack_label="",
            rack_id="",
            rack_type="",
            position="",
            tube_id="",
            volume="",
            liquid_class="",
            # tip_type="",
            tip_mask="",
            forced_rack_type="",
            min_detected_volume=""):

        self.records.append(f"A;{rack_label};{rack_id};{rack_type};{position};{tube_id};{volume};{liquid_class};;{tip_mask};{forced_rack_type};{min_detected_volume}")

    def addDispenseRecord(self,
            rack_label="",
            rack_id="",
            rack_type="",
            position="",
            tube_id="",
            volume="",
            liquid_class="",
            # tip_type="",
            tip_mask="",
            forced_rack_type="",
            min_detected_volume=""):

        self.records.append(f"D;{rack_label};{rack_id};{rack_type};{position};{tube_id};{volume};{liquid_class};;{tip_mask};{forced_rack_type};{min_detected_volume}")

    def addReagentDistributionRecord(self,
            src_rack_label="",
            src_rack_id="",
            src_rack_type="",
            src_start_pos="",
            src_end_pos="",
            dest_rack_label="",
            dest_rack_id="",
            dest_rack_type="",
            dest_start_pos="",
            dest_end_pos="",
            volume="",
            liquid_class="",
            diti_reuses="",
            multi_disp="1",
            direction="0",
            exclude=[]):

        self.records.append(f"R;{src_rack_label};{src_rack_id};{src_rack_type};{src_start_pos};{src_end_pos};{dest_rack_label};{dest_rack_id};{dest_rack_type};{dest_start_pos};{dest_end_pos};{volume};{liquid_class};{diti_reuses};{multi_disp};{direction};{';'.join(str(e) for e in exclude)}")


    ###  These are the advanced worklist commands.  I will implement them as needed. 

    def addGetDiTi(self,
            tipMask="",
            type="",
            options="0",
            arm="0"):

        self.records.append(f"B;GetDiTi({tipMask},{type},{options},{arm});")

    def addGetDiti2(self,
            tipMask="",
            LabwareName="",
            options="1",
            arm="0",
            AirgapVolume="0",
            AirgapSpeed="70"):

        self.records.append(f'B;GetDiTi2({tipMask},"{LabwareName}",{options},{arm},{AirgapVolume},{AirgapSpeed});')

    def addDropDiti(self,
            tipMask="",
            grid=1,
            site=7,
            AirgapVolume="0",
            AirgapSpeed="70",            
            arm="0"):

        self.records.append(f"B;DropDiTi({tipMask},{grid},{site-1},{AirgapVolume},{AirgapSpeed},{arm});")

    def addAspirate(self,
            tipMask="255",
            liquidClass="",
            volumes=['0']*12,
            grid=0,
            site=0,
            spacing=1,
            wellSelection="",
            # noOfLoopOptions=0,
            # loopName="",
            # action="",
            # difference="",
            arm=0,
            ):
        
        vol_str = ','.join(f'"{v:.2f}"' for v in volumes)
        self.records.append(f'B;Aspirate({tipMask},"{liquidClass}",{vol_str},{grid},{site-1},{spacing},"{wellSelection}",0,{arm});')

    def addDispense(self,
            tipMask="255",
            liquidClass="",
            volumes=['0']*12,
            grid=0,
            site=0,
            spacing=1,
            wellSelection="",
            # noOfLoopOptions=0,
            # loopName="",
            # action="",
            # difference="",
            arm=0,
            ):
        
        vol_str = ','.join(f'"{v:.2f}"' for v in volumes)
        self.records.append(f'B;Dispense({tipMask},"{liquidClass}",{vol_str},{grid},{site-1},{spacing},"{wellSelection}",0,{arm});')
        

    

    def saveToFile(self, filename):
        if filename:
            with open(filename, 'w', encoding='ansi') as file:
                for rec in self.records:
                    file.write(rec)
                    file.write('\n')
        return


    ###  Take list of Positions and convert to Tecan Well Selection String format
    @classmethod
    def positionsToWellString(cls, positions):

        ###  Make sure same plate in all positions
        plate = positions[0].plate
        if any(pos.plate != plate for pos in positions):
            raise Exception("Multiple plates in well string")

        ###  Create list of integers
        count = -(-plate.number_of_wells // 7)
        ###  Everything but the first 4 characters that tell how many wells (add them at end)
        places = [0] * count

        ###  For each position in list, set appropriate bit
        for p in positions:
            i = p.index
            which_char = i // 7
            which_bit = i % 7

            places[which_char] |= (1 << which_bit)

        ###  Add 48 to each and convert to ascii
        places = [chr(x + 48) for x in places]
        
        return f'{plate.columns:02X}{plate.rows:02X}{"".join(places)}'

        


if __name__ == "__main__":

    dplate = Plate("KingFisherPlate",8,12)
    splates = [Plate(n, 4, 6) for n in ["Samples1", "Samples2", "Samples3"]]

    lq = "IDeA NODET"
    sgrid = 16
    dgrid = 22
    ssites = [1,2,3]
    dsite = 1

    ripa_grid = 1
    ripa_site = 1

    wlist = WorkList()
    wlist.addWashRecord()

    ######  To build for JEKB samples

    ###  Get Projects from sample lists so we have the sample names
    je_file = "Z:\\Active_projects\\EdmondsonJ_20250231_01_DIA\\EdmondsonJ_20250231_01_DIA_SampleList.xlsx"
    kb_file = "Z:\\Active_projects\\BronsonK_20250231_02_DIA\\BronsonK_20250231_02_DIA_SampleList.xlsx"

    je_project = Project.createFromSampleList(je_file)
    kb_project = Project.createFromSampleList(kb_file)
    
    ###  Get the concentrations from csv file by matching up sample names and keep in dict with sample names as keys
    conc_file = "Z:\\David\\JEKB\\JEKB_Conc_Only_all_72.csv"
    with open(conc_file, 'r') as file:
        reader = csv.reader(file)
        


    ###  Create a dict of volumes based on the concentrations
        amounts = [(10.0 / float(line[1])) for line in list(reader)]
        
        sample_positions = [pos for plate in splates for pos in plate.positions]

####   RIPA
        wlist.addGetDiti2(tipMask=255, LabwareName="DiTi 200ul Filter LiHa")
    ###  See comments in section for sample
        for i, col in enumerate(dplate.positions_by_column[:9]):         
        ### Make a list of the volumes (RIPA or sample) that need to be moved
            vols = [(200 - a) if a<100 else 100 for a in amounts[(8*i) : (8*(i+1))]]

        ###  Get sample from reservoir in 1,1 and dispense to plate column.     
        ### Make a aspirate and dispense commands from the two lists
            ### aspirate all 8 from RIPA trough
            wlist.addAspirate(tipMask=255,liquidClass=lq,volumes=vols,grid=ripa_grid,site=ripa_site,wellSelection="0108¯1")
            ### dispense
            wlist.addDispense(tipMask=255,liquidClass=lq,volumes=vols,grid=dgrid,site=dsite,wellSelection=WorkList.positionsToWellString(col))
        wlist.addDropDiti(255)

####   Sample
    ###  Get positions column by column (this is faster than always splitting by 8)
        for i, col in enumerate(dplate.positions_by_column[:9]):         
        ### Make a list of the volumes (RIPA or sample) that need to be moved
            
            vols = amounts[(8*i) : (8*(i+1))]
            vols = [v if v<100 else 100 for v in vols]
        ###  Get sample positions
            splate_idx = i//3
            spos = sample_positions[(8*i):(8*(i+1))]        
        ### Make a aspirate and dispense commands from the two lists
            if all(v<50 for v in vols):
                wlist.addGetDiti2(tipMask=255, LabwareName="DiTi 50ul Filter LiHa")
            else:
                wlist.addGetDiti2(tipMask=255, LabwareName="DiTi 200ul Filter LiHa")
            ### aspirate first 4
            wlist.addAspirate(tipMask='15',liquidClass=lq,volumes=vols[:4],grid=sgrid,site=ssites[splate_idx],wellSelection=WorkList.positionsToWellString(spos[:4]))
            ### aspirate second 4
            wlist.addAspirate(tipMask='240',liquidClass=lq,volumes=vols[4:],grid=sgrid,site=ssites[splate_idx],wellSelection=WorkList.positionsToWellString(spos[4:]))
            ### dispense
            wlist.addDispense(tipMask=255,liquidClass=lq,volumes=vols,grid=dgrid,site=dsite,wellSelection=WorkList.positionsToWellString(col))
            wlist.addDropDiti(255)

        wlist.saveToFile("C:\\IDeA_Scripts\\TestData\\JEKB_wlist_test_1.txt")

And the Plate and Position objects come from:

from collections import OrderedDict
from SampleModel import *


row_letters = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ'

class Position(object):

    def __init__(self, plate, idx):

        self.plate = plate
        self.index = idx
        return    
    
    def __eq__(self, other):
        if isinstance(other, Position):
            return (self.plate, self.index) == (other.plate, other.index)
        return False
    
    def __hash__(self):
        return hash((self.plate, self.index))
    
    @property
    def row(self):
        return row_letters.index(self.plate.position_string_list[self.index][:1])
    
    @property
    def column(self):
        return int(self.plate.position_string_list[self.index][1:]) - 1
    
    @property
    def label(self):
        return self.plate.position_string_list[self.index]
    
    @classmethod
    def from_string(cls, plate, instr):
        idx = plate.position_string_list.index(instr)
        return Position(plate, idx)
    
    @classmethod
    def from_rowcol(cls, plate, row, col):
        idx = plate.position_string_list.index(f'{row_letters[row]}{col + 1}')
        return Position(plate, idx)
    

class Plate(OrderedDict):

    def __init__(self, name, rows, columns):
        super().__init__()
        self.name = name
        self.rows = rows
        self.columns = columns
        self.position_string_list = [f'{c}{i+1}' for i in range(self.columns) for c in row_letters[:self.rows]]
        self.data = {pos : None for pos in self.position_string_list}
        return
    
    def __hash__(self):
        return hash(self.name)

    def __eq__(self, other):
        if isinstance(other, Plate):
            return other.name == self.name
        return False
    
    def __setitem__(self, key, value):
        if isinstance(key, Position):
            key = key.label
        if key not in self.data.keys():
            raise KeyError
        if value is not None and not isinstance(value, Sample):
            raise TypeError("Only samples or `None` can be assigned to plate wells")
        self.data[key] = value

    
    def __getitem__(self, key):
        if isinstance(key, Position):
            return self.data[key.label]
        else:
            return self.data[key]
        

    @property
    def samples(self):
        return [sample for sample in self.data.values() if sample is not None]
        
    @property
    def projects(self):
        return (list(dict.fromkeys([s.project for s in self.samples])))

    @property
    def number_of_wells(self):
        return (self.rows * self.columns)
    
    @property
    def positions(self):
        return [Position.from_string(self, pos) for pos in self.position_string_list]
    
    @property
    def positions_by_column(self):
        return [[self.positions[r + (8*c)] for r in range(self.rows)] for c in range(self.columns)]
    
    @property
    def positions_by_row(self):
        return [[self.positions[r + (8*c)] for c in range(self.columns)] for r in range(self.rows)]
    
    @property
    def used_wells(self):
        return [Position.from_string(self, key) for key in self.data.keys() if self.data[key] is not None]

    @property
    def free_wells(self):
        return [Position.from_string(self, key) for key in self.data.keys() if self.data[key] is None]
5

If changing the file encoding to cp1252 fails, you can always check by copying and pasting from the esc file. Make sure to use notepad. I was doing it with VS Code so it was modifying the encoding every time.

If that fails then you know it’s something from the encoding in the file itself or something wrong in the command, but it won’t be which characters were written to the file. That should narrow the scope of where you look. If I remember correctly the character I ran into an issue with was the DELETE key, but copying and pasting the entire string worked, which is what clued me in that it had to be how my python script was writing it.

Initially I too used the char function. I can’t remember why I switched it out for writing directly in bytes. Let me know if that works for you.

1 Like
6

Using cp1252 actually failed on a different value. It couldn’t encode 144 and gave an error writing to the file.

I had a problem with 128 which is delete, that’s when I started running the ansi encoding. That fixed that particular one but left me with the nbsp problem with 160.

Thanks for mentioning that VSCode changed the encoding. I’ll have to be more careful. I haven’t been able to copy paste but I’m guessing this is why.

What’s funny is that when I examine the .esc file and the .gwl file with the same command I see that byte for byte they are identical. So I’m having a hard time wrapping my head around what could be going wrong.

As soon as I get all the lab fires put out and get some time I’m going to put together some output with your code and the particular well designations that cause me issues and see what happens.

7

@avargas
I think you may be on to something here. I set up your code to produce something that worked for my layout and such and included the offending tube set and your code produced something that didn’t throw any error.

I’m still confused because when I look at the files in raw hex they look identical, but the one made with your code works and the one made with my code doesn’t.

Now I’m going to try to make my code work like yours and see what I get.

Thanks for taking the time to post that for me. I think you’re getting me going in the right direction.

1 Like
8

Turns out I’m chasing the wrong problem. The issue wasn’t the character, the issue was the number of volumes I had listed. I didn’t have all 12 entries in there so it was trying to parse my position string as a volume.

I feel stupid now, but at least I have it working.

Thanks for the help.

9

You’re welcome. Glad you figured out the root cause.