[Gate-users] Running Gate on a Cluster (Mijat Paunovic)

Tue Feb 15 16:48:00 CET 2022

Hello Mijat,

It is possible to run Gate simulation in parallel cores as the Gate
documentation stated. There are several platforms you can take advantage of
as described in the documentation, but we only made the way using HTcondor
works. I am not sure what operating system you are using but we are running
our cluster with more than 600 cores on Linux system. You need to configure
a share folder among your cluster workstations then you could set up your
HTcondor for running as many jobs as you want up to how many cores you have
following the step in Gate documentation.

Hope this helps!

Best,
Xinjie

On Mon, Feb 14, 2022 at 3:27 AM olie pickford scienti <
oliepickford1 at hotmail.co.uk> wrote:

> Hello Mijat,
>
> From my experience Gate runs on a single core by default, and I do not
> think you can run it in parallel on multiple cores from a single file
> though am happy to be corrected about this. What I do to use multiple cores
> on my laptop when running simulations (I also have 8 cores) is to copy the
> mac file but with different start and finish times and different random
> seeds. So for example if running for 1 second on 4 cores, have mac files
> which run 0-0.25 s, 0.25-0.5 s, 0.5-0.75 s and 0.75-1 s, each with a
> different random seed. Then I would open 4 terminals and run 4 instances of
> Gate, 1 for each mac script. You should then be able to combine the
> resulting root files, but I process my results outside of root so just
> export each file individually to that framework.
>
> In your case I think, so long as random seed is different and you're just
> looking for testing your reconstruction rather than making statistical
> assessments of the data, you should be fine to use (10^13)/n primaries in
> each of n simulations then combine them. If n becomes very large, or you do
> this a lot, you may wish to write your own script for splitting the jobs if
> you can't get gjs working (I don't use gjs so can't advise). Hopefully
> someone else on here can explain why gjs doesn't work for you though.
>
> I hope this helps.
>
> Olie
>
> On 13 Feb 2022 12:00, gate-users-request at lists.opengatecollaboration.org
> wrote:
>
> Send Gate-users mailing list submissions to
> gate-users at lists.opengatecollaboration.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0
> or, via email, send a message with subject or body 'help' to
> gate-users-request at lists.opengatecollaboration.org
>
> You can reach the person managing the list at
> gate-users-owner at lists.opengatecollaboration.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Gate-users digest..."
>
> Today's Topics:
>
>    1. Running Gate on a Cluster (Mijat Paunovic)
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Sat, 12 Feb 2022 21:22:17 +0100
> From: Mijat Paunovic <paunovicjata at gmail.com>
> To: gate-users at lists.opengatecollaboration.org
> Subject: [Gate-users] Running Gate on a Cluster
> Message-ID:
> <CAKS_kRr=q-NXBcH_E9Bzc+EryFpT_GOps16eqy1kvZavCLCh3g at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
>
> Dear Gate Community,
> I have created a simple Cone Beam CT simulator in order to obtain
> projections which will be fed to a reconstruction algorithm. In order for
> the algorithm to work, I need to have a large number of detected photons in
> each pixel of the detector. In order to obtain this large number, I wanted
> to run the simulation for ~10^13 primaries. This is of course an impossible
> job for my computer which has only 8 logical cores (4 after assigning half
> to the virtual machine on which Gate is run) so I will get access to a
> supercomputer which will be able to do this.
>
> In the meantime, I wonder how I could maximize the performance on my
> computer? When I run the Gate simulation and monitor the CPU performance, I
> can see that only 1/4 logical cores is being used for the simulation.
>
> In Gate documentation I found the '6.1. How to use Gate on a Cluster'
> explaining how I should use the *gjs *(Gate job splitter) which will split
> my work on 4 jobs which I can independently run on each node of the
> cluster, after which I should use the *gjm *(Gate file merger) in order to
> combine the results.
>
> Several questions arose from this:
> 1. Is it possible to make Gate make the use of additional 3 cores that I
> have assigned to the virtual machine without doing the job splitting?
> 2. If I perform the job splitting, can my 4 cores be considered a cluster
> of it's own?
> 3. When running *gjs*, I am only able to get an output when am using
> *-numberofsplits* of 10 or more jobs. If I try to run it for 9 or less, I
> get 'floating point error'.
> 4. Since I am only running a simulation of 10^13 independent photon
> histories (only interested in photon interactions within a phantom and the
> count of the photons on the detector) would it be the same if I just fed
> the same simulation to on *n *nodes, with the number of primaries being
> 10^13/n?
>
> For the first test for this simulator, I will be using it on a cylindrical
> phantom. Instead of making a large number of projections which will be fed
> to a reconstruction algorithm, I will be generating particles from only one
> angle and the result (2D matrix with the photon count) will be copied
> several hundred times and hopefully, upon reconstruction, we will get a
> circular cross-section.
>
> All in all, I need a way to speed up the process of obtaining this 2D
> matrix if I am going to make the simulator more realistic.
>
> Any suggestions on this matter would be extremely helpful. Thanks in
> advance!
>
> Kind regards,
> Mijat Paunović
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: <
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fpipermail%2Fgate-users%2Fattachments%2F20220212%2Fe318e521%2Fattachment-0001.html&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mpq2bZwjMGSTedKw2QWhGIUeWkfXQ1%2B%2BwoExIP1xiw0%3D&reserved=0
> >
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> Gate-users mailing list
> Gate-users at lists.opengatecollaboration.org
>
> https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0
>
> ------------------------------
>
> End of Gate-users Digest, Vol 189, Issue 13
> *******************************************
>
>
> _______________________________________________
> Gate-users mailing list
> Gate-users at lists.opengatecollaboration.org
> http://lists.opengatecollaboration.org/mailman/listinfo/gate-users

-- 
*....................................................*
*Xinjie Cao*
*M.Eng. / Ph.D. Candidate*
*Research Project Assistant*
*Department of Electrical and Computer Engineering & Radiology *
*Novel Medical Imaging Technologies Lab*
*Health Science Center Level 8*
*Stony Brook, NY 11794-8460 *
*Tel: +1 (631)202-9445*
you.stonybrook.edu/goldan/people/
*email: **xinjie.cao at stonybroo*k.edu <xinjie.cao at stonybrook.edu>

*....................................................*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opengatecollaboration.org/pipermail/gate-users/attachments/20220215/2767fdea/attachment.html>