Out of Memory error with Venus 4 + Venus Scheduler

Hamilton (Venus) Venus Questions
1

I am having an issue with Venus Scheduler where running larger runs results in an Out of Memory error. The machine running it has 16GB of memory with plenty of ceiling, but the issue seems to be when the HxRun Application itself surpasses more than around 1100MB of memory usage as seen through task manager during the run. The memory usage seems directly related to how many plates my run has in it. If I do an 3 plate + 3 intermediate plates it runs fine and tops out on memory usage around 800MB. 4 Plates + 4 Intermediate plates crashes with an Out of Memory error as it exceeds the limit. I found I could get 4 Plates to run if I close all of the View Windows in Run Control (Scheduler, Trace, Deck Layout) and it just comes in under the limit. But 5 or more is out of the question. It’s not a PC issue I’ve encountered this on several PCs, in Simulation and Live runs. This is on Venus 4 on a Windows 10 machine.

Here’s what a 6 plate run looks like in scheduler. I watch Task Manager and every time a new thread kicks off the memory usage jumps by about 100MB.

image
image1488Ă—466 38.9 KB

Task manager the moment it crashes out

image
image1115Ă—163 8.28 KB

Error in the Trace

image
image936Ă—34 3.77 KB

Any ideas or has anyone seen this behavior before?

Thanks!

2

Yes, this behavior has been encountered before and there are some general and specific steps you can take to mitigate it.

Overview

During runtime execution of a VENUS Scheduler method, it is possible that an unspecified or out of memory error could occur, which results in an automatic abort. For a scheduled process, the memory usage increases as the amount of tasks are added and executed in the workflow. After a certain amount of tasks, the memory is exceeded and the error occurs. There are some general guidelines to avoid this problem and the use of a specific library that limits the inclusion of libraries that get added per task (which contributes to the increased memory usage).

To avoid this error, there are some general steps one can follow as well as specific libraries and a modified run control executable that can be implemented.

General Steps

  1. Avoid using a parallel process in a scheduler method. Parallel processes use a considerable amount of memory. It is recommended to use pre & post activities instead. Please note that if you had parallel processes in a method, it is not enough to delete them from the method as the process can still exist in the method’s hsl file, just not being used. Confirm if still present by viewing the hsl file and deleting them from the code.
  2. Avoid using Smart Steps in the scheduler methods. They ​use a lot of memory.
  3. Do not include the smart steps library in any of the scheduled methods as it also takes up memory. If it is in your toolbox, it has not been removed.
  4. Limit the inclusion of libraries as much as possible. Ensure that if a library is not being used by a method that it is removed.​

Common Includes Library

It is also recommended to utilize the Common Includes hsl library and add it to the scheduler workflow. When this hsl library is edited to specify your scheduler method libraries, it prevents the scheduler from unnecessarily loading the defined libraries again for each single process/task during runtime execution. See below for more details:

  1. Add the Includes hsl library → Open the workflow in the HSL editor and add the line after copying the file to your libraries folder.
  2. Open the “Common_Includes_2.hsl” in the HSL editor (see image below)
  3. Remove any includes that don’t get used
  4. IMPORTANT → Add ones you are using!

image
image796Ă—223 82.4 KB

LARGEADDRESSAWARE Run Control

For older VENUS setups, use a modified run control executable to enable the /LARGEADDRESSAWARE option. This option will allow the program to use the available RAM for software versions before VENUS 5 (VENUS 5 and higher includes this option by default). See below for instructions:

  1. At a minimum, Windows 7 64-bit is required​ and PC must be equipped with enough RAM >= 8GB
  2. Make backups of the HxRun.exe and HxStandardLanguage.dll files found in the HAMILTON\Bin folder
  3. Download the zip file and replace the existing files in the Bin folder with the included HxRun.exe and HxStandardLanguage.dll files

I hope this helps!

Eric

5 Likes
3

Thank you Eric!

I implemented the General recommendations and saw marginal improvements. Still crashed out.

Replacing the RunControl files worked like a charm though. It cruises around 1200MB on a 6 plate (12 active thread) run and completed without an issue.

Thanks again!

3 Likes