For my 3D simulation project, we can control instruments including robot arms/device/liquid handler virtually. Recently I have an idea that if we can get status or data (e.g. values of each joint for robot arm) of hardwares through web service, then we can make synchronization between real and 3D simulation. So following test was done, standalone application acted as real devices, web simulation in browser communicated and got all the data of devices through web service, and then synchronization was achieved.
I think this may be prototype of digital twin, it is very simple to setup the system and then we can access it on any computers/phone with browser.
maybe later I can build a website for the simulation tool, where people can share the models of hardware and setup their system to monitor/simulate the system run.
With this simulation tool, you can drag instrument to the scene, setup your system, customize each instrument (arm type/gripper type/door handling), check plate transportation, record and replay these recorded actions.
Also you can add your models to scene, download glb file of scene, write/execute javascript to control all these instruments (event digital twin is possible, if you can setup a web service to get the details of your instruments)
Currently this tool support following types of 3D model
add instrument to scene
click on instrument in the list and move the instrument to position you want, or drag the instrument in the list to position that you want
move/rotate the instrument
click on instrument and drag to your position, or click on the transformation handler to move/rotate the instrument. Click the button of rotation to rotate instrument.
setup/perform action on selected instrument
click on instrument to select it, set each setting on the left property grid to setup it, or click on buttons to perform actions
transport plate
click on the instrument to select it, and click the “move plate” button in toolbar, the transportation panel will be showed, select robot arm and setup transporation parameter, and then click the “movePlate” button below.
mount arm on AGV
move the robot arm with mouse, move it on AGV or arm rail, and the robot arm will be mounted to AGV or rail.
move the gripper with mouse
click and drag the gripper of robot arm, then the robot arm will move with the moving of mouse. Many instruments support mouse dragging operation, including STAR (arm, MPH, iSWAP, channel) and robot rail.
recording action/movement and replaying
click on the recording button in toolbar, perform the action of instrument or moving plate, then these actions will be recorded to replay list in the right. You can edit these recorded actions, or replay it.
add your move 3D model to scene
click the last button in toolbar, setup your model in popup window. Select path of your model, set the right scale and up direction, then you can add it to scene by clicking on the second button below.
you can see this example, and test all these functions and see the recorded actions in replay list.
