We have a need to incubate a plate at an Inheco ODTC with the door open on our Cellario system. The current ODTC drivers don’t allow for running a protocol with the door open so we use scripts to warm the ODTC to the correct temp and move the plate on and off the ODTC. Currently the script essentially pauses the whole system to keep the plate on the ODTC, but I’d like the script to only pause that plate at the ODTC. However, I can’t figure out how to do that. The current script were using the pause is:
System.Threading.Thread.Sleep();
I was thinking for my next step I would try:
await Task.Delay(TimeSpan.FromMinutes(10));
But I’m not sure if the system will just hang up on executing the script for 10mins and will essentially result in the same outcome.
Gonna make a few assumptions. Easiest way to wait at a resource is to add a “Pause Before”/“Pause After” or to “Incubate” by making the ODTC an eligible storage resource. Think I’ve seen from one of your previous posts that you have to do the moves via scripts, so these won’t work.
From what I know, script steps won’t execute concurrently in Cellario (for good reason because imagine having to mitigate race conditions across all your scripts and orders). Both of your wait methods will have the same effect where the script step won’t finish for 10 minutes, and you need to return control back to the scheduler earlier to run other plates/orders.
Here’s what might work. Since you can’t just end the script and move on without waiting the necessary 10 minutes, you’ll have to loop the script step in the protocol. In the script, seed the initial wait start time in PerPlateData (you can use a different data source but this might be the easiest). You’ll retrieve this each time the script runs to compare to the current time. If 10 minutes have not elapsed, find the “Loop End” step for the plate and increment the NumberOfLoops by 1 (makes the protocol re-run the script), but also add in a shorter wait time using Thread.Sleep() or Task.Delay(). That wait time depends on how often you want to finish the script step and return control back to the scheduler. If you can afford to wait like 30 seconds before releasing other plates, you won’t be as inundated with this script’s traces in the diagnostic log/order events, but feel free to be as impatient as you’d like.
Thanks for that Dennis, I had considered doing a loop on the script with a shorter pause but was hoping there’d be a more “elegant” solution. Seems like the loop will mean that Cellario may progress one or two steps between each shorter pause. Right now it even pauses at Open Door and Close Door commands. So it’ll be a few loops just to move a single plate around, but better than nothing! I’ll report back when I’m able to test it.
Unfortunately looping the script didn’t help. Cellario doesn’t attempt to complete any other tasks between each loop of the script. I’ll keep at it to see if we can come up with other solutions.
Need to understand your protocol/order setup and what you see in Run Time Viewer more. I quickly tested this in v4.0.1.31 (oldest version we have) and for these scenarios (a different plate in the same thread, a plate in a different thread, or a plate in a different order), the scheduler is able to switch off and execute other tasks in-between each loop of a script. Do you by chance have a flow gate of 1 set over the steps where you want to see concurrency and maybe all your plates are for the same thread?
I was looking at your other post again, and wondering if you ever gave opening the door in an after move script a try. Don’t have an ODTC to confirm, but here’s what I imagine. The door will close once after the arm places the plate, but should immediately re-open, so if that’s acceptable, you won’t have to do the moves with scripts and you can incubate by setting “Pause Before” in the “Execute Method” operation. This should also eliminate the need to re-invent the wheel with inventory management and resource pooling.
I’ll try to get more details. The test we did had no flow gates that should hold up other plates from processing. But, I want to do a more thorough test on a protocol written just to test this functionality.
The plates we’re putting in the ODTCs are unsealed, so closing the door on them, even just for a second, is something that I’d like to avoid. I think that’s a recipe for carryover contamination.
Turns out there was some miscommunication between myself and the person who was doing the scripting and the loop was added to the script instead of the Cellario protocol.
So once the loops were added in the correct spot Cellario will move plates around between loops of the script.
The next problem (there’s always a next problem) is that we run this with two plates in parallel, we have two Bravos and use 2 of our 4 ODTCs. We do 10 loops of a script that pauses for 1 minute. The problem is that now our pauses alternate and our plates are getting incubated for roughly 20mins instead of the desired 10mins. The easy answer would be to reduce our loops, but we do also run this with one plate occassionally. One plate (or the last plate of an odd number plate run) would then only incubate for 5mins.
What I’d like to do is track the time that the plate was placed on the ODTC and then exit the Cellario loop once 10mins is achieved for that plate. I suppose I would use an array variable for the time and the element number would be the plate number. Pretty sure I can get all of that, but not sure how to exit the Cellario loop using a script. Any thoughts?
Glad to hear you’re getting somewhere! To deal with your new problem, I’ll refer you back to my previous post.
It’s great that the scheduler is switching off, but as you can see, the script step in the loop can’t run concurrently with another script step. Since you care more about tracking incubation time rather than the amount of script executions, you don’t want to hard-code the number of loops. Instead, you should start with 1, and dynamically increment by 1 each time your script evaluates that 10 minutes have not passed.
After you make the change, with your setup, you might see cases where you break out of the loop closer to 11 minutes if the last script execution happened at 9:59 for example. If that’s not acceptable, just bring down your 1 minute wait time in the script to do the evaluations more frequently.
Thanks for the help Dennis. I have it working in a test script and plan to implement it in our actual script today. In the test script I have it loop with a 10 second delay, will probably change it to somewhere around 30 seconds to reduce blasting the debug window.
I ended up using api.Data.PerPlateData just before the loop to store the time that the plate initially gets to the ODTC. Then in the loop I just compare the data I stored there to DateTime.Now. So far so good! Learning how to store data in the PerPlateData (or PerStepData) will be incredibly useful going forward, so I’m glad my team revisited our sub-par solution that we had.
I’m a Product Manager at HighRes and we are getting close to releasing CellarioScheduler 4.3 and one of the new improvements is the ability to run scripts asynchronously from the scheduler.
In the Protocol Designer, scripts now have a parameter to change Blocking Mode. Non-blocking (async) will allow the script to execute without blocking the scheduler from proceeding. This means you can do some long-running tasks in a script without blocking the rest of the system from continuing to process other plates. The async script still blocks the current plate/thread, it just doesn’t block other plates.
Async scripts should typically have some sort of loop or periodic check of the system state to see if its pausing, and it should exit in that case. Otherwise, an async script will prevent a system from pausing.
Since async scripts are executing in another thread separate from the main scheduler thread, this can cause some variations in the gantt chart in simulations. This is a known issue but there’s not much we can do about it, since it gets into low-level operating system behavior of threads.
Our target release date is towards the end of the month, but that is subject to change as we go through our SQA cycle.
Seems like the loop will mean that Cellario may progress one or two steps between each shorter pause. Right now it even pauses at Open Door and Close Door commands. So it’ll be a few loops just to move a single plate around, but better than nothing! I’ll report back when I’m able to test it.