<div dir="ltr"><div id="gmail-:1p6" class="gmail-Am gmail-aO9 gmail-Al editable gmail-LW-avf gmail-tS-tW gmail-tS-tY" aria-label="Message Body" role="textbox" aria-multiline="true" tabindex="1" style="direction:ltr;min-height:85px">Hello Mijat,<div><br></div><div>It is possible to run Gate simulation in parallel cores as the Gate documentation stated. There are several platforms you can take advantage of as described in the documentation, but we only made the way using HTcondor works. I am not sure what operating system you are using but we are running our cluster with more than 600 cores on Linux system. You need to configure a share folder among your cluster workstations then you could set up your HTcondor for running as many jobs as you want up to how many cores you have following the step in Gate documentation. </div><div><br></div><div>Hope this helps!</div><div><br></div><div>Best,</div><div>Xinjie</div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Mon, Feb 14, 2022 at 3:27 AM olie pickford scienti <<a href="mailto:oliepickford1@hotmail.co.uk">oliepickford1@hotmail.co.uk</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div>Hello Mijat, </div><div dir="auto"><br></div><div dir="auto">From my experience Gate runs on a single core by default, and I do not think you can run it in parallel on multiple cores from a single file though am happy to be corrected about this. What I do to use multiple cores on my laptop when running simulations (I also have 8 cores) is to copy the mac file but with different start and finish times and different random seeds. So for example if running for 1 second on 4 cores, have mac files which run 0-0.25 s, 0.25-0.5 s, 0.5-0.75 s and 0.75-1 s, each with a different random seed. Then I would open 4 terminals and run 4 instances of Gate, 1 for each mac script. You should then be able to combine the resulting root files, but I process my results outside of root so just export each file individually to that framework. </div><div dir="auto"><br></div><div dir="auto">In your case I think, so long as random seed is different and you're just looking for testing your reconstruction rather than making statistical assessments of the data, you should be fine to use (10^13)/n primaries in each of n simulations then combine them. If n becomes very large, or you do this a lot, you may wish to write your own script for splitting the jobs if you can't get gjs working (I don't use gjs so can't advise). Hopefully someone else on here can explain why gjs doesn't work for you though. </div><div dir="auto"><br></div><div dir="auto">I hope this helps. </div><div dir="auto"><br></div><div dir="auto">Olie<br><div class="gmail_extra" dir="auto"><br><div class="gmail_quote">On 13 Feb 2022 12:00, <a href="mailto:gate-users-request@lists.opengatecollaboration.org" target="_blank">gate-users-request@lists.opengatecollaboration.org</a> wrote:<br type="attribution"><blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><p dir="ltr">Send Gate-users mailing list submissions to<br>
<a href="mailto:gate-users@lists.opengatecollaboration.org" target="_blank">gate-users@lists.opengatecollaboration.org</a></p>
<p dir="ltr">To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0" target="_blank">https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:gate-users-request@lists.opengatecollaboration.org" target="_blank">gate-users-request@lists.opengatecollaboration.org</a></p>
<p dir="ltr">You can reach the person managing the list at<br>
<a href="mailto:gate-users-owner@lists.opengatecollaboration.org" target="_blank">gate-users-owner@lists.opengatecollaboration.org</a></p>
<p dir="ltr">When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Gate-users digest..."<br></p>
<p dir="ltr">Today's Topics:</p>
<p dir="ltr"> 1. Running Gate on a Cluster (Mijat Paunovic)<br></p>
<p dir="ltr">----------------------------------------------------------------------</p>
<p dir="ltr">Message: 1<br>
Date: Sat, 12 Feb 2022 21:22:17 +0100<br>
From: Mijat Paunovic <<a href="mailto:paunovicjata@gmail.com" target="_blank">paunovicjata@gmail.com</a>><br>
To: <a href="mailto:gate-users@lists.opengatecollaboration.org" target="_blank">gate-users@lists.opengatecollaboration.org</a><br>
Subject: [Gate-users] Running Gate on a Cluster<br>
Message-ID:<br>
<CAKS_kRr=<a href="mailto:q-NXBcH_E9Bzc%2BEryFpT_GOps16eqy1kvZavCLCh3g@mail.gmail.com" target="_blank">q-NXBcH_E9Bzc+EryFpT_GOps16eqy1kvZavCLCh3g@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"</p>
<p dir="ltr">Dear Gate Community,<br>
I have created a simple Cone Beam CT simulator in order to obtain<br>
projections which will be fed to a reconstruction algorithm. In order for<br>
the algorithm to work, I need to have a large number of detected photons in<br>
each pixel of the detector. In order to obtain this large number, I wanted<br>
to run the simulation for ~10^13 primaries. This is of course an impossible<br>
job for my computer which has only 8 logical cores (4 after assigning half<br>
to the virtual machine on which Gate is run) so I will get access to a<br>
supercomputer which will be able to do this.</p>
<p dir="ltr">In the meantime, I wonder how I could maximize the performance on my<br>
computer? When I run the Gate simulation and monitor the CPU performance, I<br>
can see that only 1/4 logical cores is being used for the simulation.</p>
<p dir="ltr">In Gate documentation I found the '6.1. How to use Gate on a Cluster'<br>
explaining how I should use the *gjs *(Gate job splitter) which will split<br>
my work on 4 jobs which I can independently run on each node of the<br>
cluster, after which I should use the *gjm *(Gate file merger) in order to<br>
combine the results.</p>
<p dir="ltr">Several questions arose from this:<br>
1. Is it possible to make Gate make the use of additional 3 cores that I<br>
have assigned to the virtual machine without doing the job splitting?<br>
2. If I perform the job splitting, can my 4 cores be considered a cluster<br>
of it's own?<br>
3. When running *gjs*, I am only able to get an output when am using<br>
*-numberofsplits* of 10 or more jobs. If I try to run it for 9 or less, I<br>
get 'floating point error'.<br>
4. Since I am only running a simulation of 10^13 independent photon<br>
histories (only interested in photon interactions within a phantom and the<br>
count of the photons on the detector) would it be the same if I just fed<br>
the same simulation to on *n *nodes, with the number of primaries being<br>
10^13/n?</p>
<p dir="ltr">For the first test for this simulator, I will be using it on a cylindrical<br>
phantom. Instead of making a large number of projections which will be fed<br>
to a reconstruction algorithm, I will be generating particles from only one<br>
angle and the result (2D matrix with the photon count) will be copied<br>
several hundred times and hopefully, upon reconstruction, we will get a<br>
circular cross-section.</p>
<p dir="ltr">All in all, I need a way to speed up the process of obtaining this 2D<br>
matrix if I am going to make the simulator more realistic.</p>
<p dir="ltr">Any suggestions on this matter would be extremely helpful. Thanks in<br>
advance!</p>
<p dir="ltr">Kind regards,<br>
Mijat Paunović<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fpipermail%2Fgate-users%2Fattachments%2F20220212%2Fe318e521%2Fattachment-0001.html&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mpq2bZwjMGSTedKw2QWhGIUeWkfXQ1%2B%2BwoExIP1xiw0%3D&reserved=0" target="_blank">https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fpipermail%2Fgate-users%2Fattachments%2F20220212%2Fe318e521%2Fattachment-0001.html&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mpq2bZwjMGSTedKw2QWhGIUeWkfXQ1%2B%2BwoExIP1xiw0%3D&reserved=0</a>></p>
<p dir="ltr">------------------------------</p>
<p dir="ltr">Subject: Digest Footer</p>
<p dir="ltr">_______________________________________________<br>
Gate-users mailing list<br>
<a href="mailto:Gate-users@lists.opengatecollaboration.org" target="_blank">Gate-users@lists.opengatecollaboration.org</a><br>
<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0" target="_blank">https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0</a></p>
<p dir="ltr">------------------------------</p>
<p dir="ltr">End of Gate-users Digest, Vol 189, Issue 13<br>
*******************************************<br>
</p>
</blockquote></div><br></div></div></div>_______________________________________________<br>
Gate-users mailing list<br>
<a href="mailto:Gate-users@lists.opengatecollaboration.org" target="_blank">Gate-users@lists.opengatecollaboration.org</a><br>
<a href="http://lists.opengatecollaboration.org/mailman/listinfo/gate-users" rel="noreferrer" target="_blank">http://lists.opengatecollaboration.org/mailman/listinfo/gate-users</a></blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div dir="ltr"><div><b><font color="#000000">....................................................</font></b><br></div><div><b><span style="color:rgb(153,0,0);font-family:"arial black",sans-serif;font-size:x-small">Xinjie Cao</span></b><br></div><div><b><font size="1" face="arial black, sans-serif">M.Eng. / Ph.D. Candidate</font></b></div><div><b><font size="1" face="arial black, sans-serif">Research Project Assistant</font></b></div><div><b><font size="1" face="arial black, sans-serif">Department of Electrical and Computer Engineering & Radiology </font></b></div><div><b><font size="1" face="arial black, sans-serif">Novel Medical Imaging Technologies Lab</font></b></div><div><b><font size="1" face="arial black, sans-serif">Health Science Center Level 8</font></b></div><div><b><font size="1" face="arial black, sans-serif">Stony Brook, NY 11794-8460 </font></b></div><div><b><font size="1" face="arial black, sans-serif">Tel: +1 (631)202-9445</font></b></div><div><a href="https://you.stonybrook.edu/goldan/people/" target="_blank">you.stonybrook.edu/goldan/people/</a><b><font size="1" face="arial black, sans-serif"><br></font></b></div><div><font size="1" face="arial black, sans-serif"><b>email: </b><a href="mailto:xinjie.cao@stonybrook.edu" target="_blank"><b>xinjie.cao@stonybroo</b>k.edu</a></font></div><div><font size="1" face="arial black, sans-serif"><br></font></div><div><font size="1" face="arial black, sans-serif"> </font><img src="https://docs.google.com/uc?export=download&id=12iMPAuaMtpKRdkcmBleVsP6iWiM4wlm_&revid=0B_TLWOu4jP1eWnFFRW9qQXlicTY3ZTVzWnR5QkRVY3RRT3JVPQ" width="200" height="28"> <img src="https://drive.google.com/a/stonybrook.edu/uc?id=17TB1Ha3enD8HC5e_Ez238vpeBF92aCgH&export=download" width="200" height="33"></div><div><div><b><font color="#000000">....................................................</font></b></div></div></div></div></div></div></div></div></div></div></div></div></div>