I am making a dynamic PDF run report library in HSL and was wondering if there was any way to obtain an overview image of the deck layout, or ideally at specific times in the run.
Ideally I want to create a library function which can be used to capture the deck layout at that point in time. I know that labware loading library takes the images from the labware definitions but ideally getting a snapshot of the entire deck layout dynamically is better if possible.
Also if not possible with Venus is there a way to render a .lay as an image with PyHamilton @Stefan ?
Not really, with PyHamilton you are pretty locked in to the layfile system Hamilton gives you. If there were a way to dynamically change what resources were shown in the layout within a run control session I imagine that could get you a lot of the way there though, something like what exists in AppsLib.
I might have to be a little creative and use the .x files for each labware and the metadata from .lay to build the deck manually in a new program. I can also try to bring in the device render from the blank layout to see if I can find a way to get the desired result.
you can use the COM object of HxInstrument3DView from venus software to show deck layout in your WinForm application, and then you can capture its image.
also, you can use COM object of deck layout to get all the information of deck, then you can draw the layout.
but all these things are done in dotNet. For pyhamilton, it is only a communication channel between python and venus, I do not think that you can get the detailed information of deck layout with pyhamilton, you have to do it outside pyhamilton.
Using IntelliSense I see that there are the following com object functions available for HxInstrument3DView, have you ever successfully viewed the deck layout in WinForm?
public void Initialize(int hWnd, object systemDeck, string instrumentName)
{
_InitializeEx(hWnd, systemDeck, instrumentName);
}
public void Initialize2(int hWnd, object instrumentDeck)
{
_InitializeEx(hWnd, instrumentDeck, null);
}
Basically I am just looking to obtain a screenshot of the deck layout but do so dynamically based on the state of the run, and take a screanshot/bitmap of the ‘3Dview’ or even the ‘Top View’ with all of the labware and components rendered, this can be then used in a report, or even automatically saved as an image for complex runs which may have different labware at different times.
Any advice you have would be greatly appreciated!
Yes, you can check the code which I use HxInstrument3DView to display deck layout in githup
also you can use the code from github to load detailed information from deck layout file, and then you can draw the deck. I simulated the run of STAR in my software
When I load a layout file with any components other than the waste, it flashes the other on deck components and then it removes them, especially if they are set to invisible on runtime. Is there a way to get the simple deck layout of the system from a .lay filepath? is this something where the ml_star instrument can be passed in as an object to the COM component/C# and then get the current state of the deck?
It looks like the SharpHamilton is more designed for control vs obtaining information from the hamilton dlls. Basically I am hoping to get a library deveoloped which basically is GetCurrentLayoutImage which can be called on to launch this outside C# code, or even better yet use the native dll controls in HSL to just get a snapshot of the deck at that timepoint as a viewable image.
if C# needs to be used, where would I get the hWnd especially if I want to have this snapshot be in the background (assuming that the initialize function is the target function to call and not just the HxSystemView3D function which has zero params to pass in)?
Yeah the layfile contains all the deck layout information (stating the obvious here) and it can be parsed by external software to determine spatial layout. They’re technically just text files with some weird characters. It’s very unintuitive though, and doesn’t conform to a generic format like JSON.
You can convert the layfiles to ASCII with an executable to make them slightly more readable and get a sense of the structure, but they always convert back when you open them in method editor.
If you check the code, you can find how to use HxInstrument3DView. It is only two steps to use it,
1 load and init the SystemDeck from .lay file
2 initialize the HxInstrument3DView with hwnd of control and SystemDeck
then the deck will be shown on the control with hwnd.
HxSystemDeck = new SystemDeck();
HxSystemDeck.InitSystemFromFile(cmdRunDeckLayoutFile);
HxInstrument3DView view = new HxInstrument3DView();
view.Initialize(hwnd, HxSystemDeck, "ML_STAR");
view.Mode = HxSys3DViewMode.RunVisualization;
view.ModifyEnable = false;
then you can capture the image of control.
Hi @zachary.milot did you manage to get your application working ? I’ll be very interested in seeing what the code can look like ?
C# code for COM object
using Hamilton.HxSys3DView;
using HxSysDeckLib;
using System.Runtime.InteropServices;
using System.Threading;
using System.Windows.Forms;
namespace Test
{
[Guid("04988F73-2593-4D69-AFF5-40B6B7EC8440")]
[ComVisible(true)]
public interface IDeckLayoutView
{
void ShowLayout(string file);
}
[Guid("3B6FC438-9185-49C6-91F8-B47273DAE0C5")]
[ClassInterface(ClassInterfaceType.None)]
[ProgId("Test.DeckLayoutView")]
[ComVisible(true)]
public class DeckLayoutView : IDeckLayoutView
{
static Form Show3DSystemViewForm(string file)
{
var form = new Form();
form.Width = 800;
form.Height = 600;
form.Text = "3D system view";
var HxSystemDeck = new SystemDeck();
HxSystemDeck.InitSystemFromFile(file);
HxInstrument3DView view = new HxInstrument3DView();
view.Initialize(form.Handle.ToInt32(), HxSystemDeck, "ML_STAR");
view.Mode = HxSys3DViewMode.RunVisualization;
view.ModifyEnable = false;
return form;
}
public void ShowLayout(string file)
{
Thread t = new Thread(() =>
{
Show3DSystemViewForm(file).ShowDialog();
});
t.SetApartmentState(ApartmentState.STA);
t.Start();
t.Join();
}
}
}
command to register the COM:
C:\Windows\Microsoft.NET\Framework\v4.0.30319\RegAsm.exe DeckLayoutView.dll /codebase
hsl to use the COM to display a deck file
method main()
{
object obj;
obj.CreateObject("Test.DeckLayoutView");
obj.ShowLayout("C:\\Program Files (x86)\\HAMILTON\\Methods\\Test\\Method1.lay");
obj.ReleaseObject();
}
run result:
I upload these codes to github, and also add the function of converting the layout to image.
Wow ! You’re awesome, it’s a lot more clear now
Thank you so much @Huajiang that makes total sense now!!
I have made code below which uses the winform to screenshot the visible area and then crop it to just the deck:
using Hamilton.HxSys3DView;
using HxSysDeckLib;
using System;
using System.Drawing;
using System.Drawing.Imaging;
using System.Runtime.InteropServices;
using System.Threading;
using System.Windows.Forms;
namespace Test
{
[Guid("9654cd04-832b-4bfd-93ce-70423c3c3713")]
[ComVisible(true)]
public interface IDeckLayoutView
{
string CaptureDeckImage(string file, string savePath);
}
[Guid("d163dfbb-de10-44aa-b1ad-b73fc24424e0")]
[ClassInterface(ClassInterfaceType.None)]
[ProgId("Test.DeckLayoutView")]
[ComVisible(true)]
public class DeckLayoutView : IDeckLayoutView
{
private static Form CreateHiddenDeckForm(string file, out HxInstrument3DView view)
{
var form = new Form
{
Width = 1920,
Height = 1080,
Text = "Hidden Deck Capture",
Visible = false
};
var HxSystemDeck = new SystemDeck();
HxSystemDeck.InitSystemFromFile(file);
view = new HxInstrument3DView();
view.Initialize(form.Handle.ToInt32(), HxSystemDeck, "ML_STAR");
view.Mode = HxSys3DViewMode.RunVisualization;
view.ModifyEnable = false;
return form;
}
public string CaptureDeckImage(string file, string savePath)
{
string finalPath = savePath;
Thread t = new Thread(() =>
{
Form form = CreateHiddenDeckForm(file, out HxInstrument3DView view);
form.Show();
form.Refresh();
Thread.Sleep(2000);
for (int i = 0; i < 5; i++)
{
SendKeys.SendWait("^{+}");
Thread.Sleep(500);
}
Thread.Sleep(2000);
Bitmap deckImage = CaptureClientArea(form.Handle);
Bitmap processedImage = ProcessImage(deckImage);
processedImage.Save(savePath, ImageFormat.Png);
Console.WriteLine($"Deck layout image saved at: {savePath}");
form.Close();
});
t.SetApartmentState(ApartmentState.STA);
t.Start();
t.Join();
return finalPath;
}
private static Bitmap CaptureClientArea(IntPtr hwnd)
{
RECT rect;
GetClientRect(hwnd, out rect);
int width = rect.Right - rect.Left;
int height = rect.Bottom - rect.Top;
Bitmap bitmap = new Bitmap(width, height);
using (Graphics g = Graphics.FromImage(bitmap))
{
IntPtr hdcBitmap = g.GetHdc();
IntPtr hdcWindow = GetDC(hwnd);
BitBlt(hdcBitmap, 0, 0, width, height, hdcWindow, 0, 0, SRCCOPY);
ReleaseDC(hwnd, hdcWindow);
g.ReleaseHdc(hdcBitmap);
}
// Crop 5% from the right
int cropWidth = (int)(width * 0.95);
Rectangle cropArea = new Rectangle(0, 0, cropWidth, height);
Bitmap croppedBitmap = bitmap.Clone(cropArea, bitmap.PixelFormat);
return croppedBitmap;
}
private static Bitmap ProcessImage(Bitmap image)
{
Color transparentColor = Color.FromArgb(0xF5, 0xEB, 0xD7);
image.MakeTransparent(transparentColor);
return CropToContent(image);
}
private static Bitmap CropToContent(Bitmap source)
{
int minX = source.Width, minY = source.Height, maxX = 0, maxY = 0;
for (int y = 0; y < source.Height; y++)
{
for (int x = 0; x < source.Width; x++)
{
if (source.GetPixel(x, y).A != 0)
{
if (x < minX) minX = x;
if (y < minY) minY = y;
if (x > maxX) maxX = x;
if (y > maxY) maxY = y;
}
}
}
int cropWidth = maxX - minX + 1;
int cropHeight = maxY - minY + 1;
Rectangle cropRect = new Rectangle(minX, minY, cropWidth, cropHeight);
return source.Clone(cropRect, source.PixelFormat);
}
#region Windows API
private const int SRCCOPY = 0x00CC0020;
[DllImport("user32.dll")]
private static extern bool GetClientRect(IntPtr hWnd, out RECT lpRect);
[DllImport("user32.dll")]
private static extern IntPtr GetDC(IntPtr hWnd);
[DllImport("user32.dll")]
private static extern int ReleaseDC(IntPtr hWnd, IntPtr hDC);
[DllImport("gdi32.dll")]
private static extern bool BitBlt(IntPtr hdcDest, int xDest, int yDest, int w, int h,
IntPtr hdcSrc, int xSrc, int ySrc, int rop);
[StructLayout(LayoutKind.Sequential)]
private struct RECT
{
public int Left;
public int Top;
public int Right;
public int Bottom;
}
#endregion
}
}
Maybe you have some ideas on how to make this process more efficient/ how to keep the winform hidden during the process?
If getting the .x directx file is possible maybe that would allow for the resultant image be a vector such as an svg vs just a bitmap which would be even more ideal.
Let me know your thoughts!
In my github code I also put some code to capture the deck image, which I think is better, you can check it. With get rect and get DPI to capture the right image.
I have the .x file and I can simulate the STAR running with 3D
Also actually you can get all the information from deck layout file, and draw it simply with several code
I’ve been trying to find the 3D simulator in your GitHub repositories, but I can’t seem to locate it. I might just be missing the correct name, as I was trying to match it based on the window title.
Additionally, is there a quick and straightforward way to extract the .x file render from the 3D system view without first rendering it in a WinForm? This would allow me to convert the .x file to a .obj and then use apose to turn it into U3D for direct embedding as a 3D object within the PDF report library I’m developing, rather than relying on capturing a flat bitmap screenshot.
Let me know if you have any thoughts on how to achieve this more efficiently!
I did not share the source code of 3D simulator.
To load .x file in U3D, you can try AssimpNet. I am sorry that I did now know how to render the 3D system view without showing WinForm.
Actual you can get all the details from deck layout file, and then you draw the layout or setup 3D models
Do you know how I would get the .x file out using c# and passing in the deck object like your ShowDeck2 function shows the deck of the object?
I never did thing like that, and I do now know how to get .x file out.
