<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body dir="auto">
Hi all,
<div><br>
</div>
<div>Just chiming in with my workflow a bit. I’ve had great success using the Task Spooler to submit and split my parametrized simulation over multiple cores. I use a python script to calculate the the parameters and set the aliases based on the user arguments.
Then I have a complimenting script to combine the output. Let me know if you’re interested in the code at all.<br>
<br>
Best,<br>
<div dir="ltr">Matthew</div>
<div dir="ltr"><br>
<blockquote type="cite">On Feb 14, 2022, at 10:18 AM, Josh Knowland <jknowland@lucernodynamics.com> wrote:<br>
<br>
</blockquote>
</div>
<blockquote type="cite">
<div dir="ltr">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--><!-- START CAUTION Box Code -->
<table align="center" border="0" cellpadding="0" cellspacing="0" style="padding:10px 0 10px 0" width="100%">
<tbody>
<tr>
<td style="line-height:0px;font-size:0px;mso-line-height-rule:exactly;">
<table align="center" border="0" cellpadding="0" cellspacing="0" style="background:#707372;background-color:#707372;width:100%;border-radius:5px;overflow:hidden;">
<tbody>
<tr>
<td style="border-top:solid 8px #fbe122;padding:4px 8px;text-align:left;vertical-align:top;">
<table role="presentation" border="0" cellpadding="0" cellspacing="0">
<tbody>
<tr>
<td align="left">
<div style="font-family:Arial,sans-serif;font-size:12px;line-height:16px;text-align:left;color:#ffffff;">
<span style="font-weight:bold;font-size:12px;">CAUTION:</span> The Sender of this email is not from within Dalhousie.</div>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
</td>
</tr>
</tbody>
</table>
<!-- END CAUTION Box Code -->
<div>
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Thanks for sending this out, Mark! I should do a formal write-up of the technique with pictures and add it to the wiki.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Josh<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Gate-users <gate-users-bounces@lists.opengatecollaboration.org>
<b>On Behalf Of </b>BAKER, Mark (THE CLATTERBRIDGE CANCER CENTRE NHS FOUNDATION TRUST)<br>
<b>Sent:</b> Monday, February 14, 2022 5:08 AM<br>
<b>To:</b> gate-users@lists.opengatecollaboration.org<br>
<b>Subject:</b> Re: [Gate-users] Running Gate on a Cluster (Mijat Paunovic)<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Mijat<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I cannot take credit for this solution (thanks to Josh Knowland who sent me the instructions on how to do this), but in essence you can run Gate
several times in parallel. The downside of this is that each version of Gate then needs full resources (so you might hit a RAM limit depending on your setup) but it does utilise as many cores as you like then. Make sure you point your outputs to different
folders, and then obviously they need merging afterwards.<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I’ve copied Josh’s reply to me from when I asked a similar question<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">This is something I’ve worked on a fair bit and here’s what I came up with. I run my simulations remotely on a 64-core virtual machine through Azure, but the process should
be the same.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Note that this method works well with the new unified Tree output in GATE 9, but also works with the ‘old’ output files and previous versions of GATE.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Please note! I do not use ROOT file outputs. I have started using the NUMPY file outputs for singles and hits so that I can do custom analysis. This process should work just
fine with ROOT files, though, and I talk about that a bit at the end of the email.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">The overall idea is to split your macro in time and then combine the results from each. Generate a macro that does what I want. Create multiple copies of the macro,
<b>each with a different Tree output filename</b>. All other simulation parameters can be the same (volume names, etc. do not need to change). Run all the macros concurrently using a shell script like below. Each process will be put on a different core and
htop will display CPU usage (F10 to exit htop).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">#!/bin/bash<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">(<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in;text-indent:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">(<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_00.mac > macro_00_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_01.mac > macro_01_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_02.mac > macro_02_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_03.mac > macro_03_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_04.mac > macro_04_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_05.mac > macro_05_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_06.mac > macro_06_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.0in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate macro_07.mac > macro_07_log.txt &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.75in;text-indent:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">wait<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in;text-indent:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">)<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">) &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.25in"><span style="font-size:10.0pt;font-family:"Courier New"">htop<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">An alternative approach is to use parameterized macros. For this, you’d create one macro with
<b>GATE parameters</b> as place holders using braces. Then, the bash script file would call that macro multiple times while changing the parameters. For instance, you could run a simulation and split it over time by parameterizing
<b>timestart</b> and <b>timestop</b> like this in the GATE macro: <o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:11.0pt;font-family:"Courier New"">/gate/application/setTimeSlice 0.125 s<br>
/gate/application/setTimeStart {timestart} s<br>
/gate/application/setTimeStop {timestop} s<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New""><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Then the Bash script would define the parameters for each process:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">#!/bin/bash<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">(<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">(<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.000][timestop,0.125] paramMacro.mac > output_00_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.125][timestop,0.250] paramMacro.mac > output_01_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.250][timestop,0.375] paramMacro.mac > output_02_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.375][timestop,0.500] paramMacro.mac > output_03_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.500][timestop,0.625] paramMacro.mac > output_04_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.625][timestop,0.750] paramMacro.mac > output_05_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.750][timestop,0.875] paramMacro.mac > output_06_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">Gate -a [timestart,0.875][timestop,1.000] paramMacro.mac > output_07_log.txt &
<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:1.5in"><span style="font-size:10.0pt;font-family:"Courier New"">wait<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in;text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">)<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">wait<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">) &<o:p></o:p></span></p>
<p class="MsoNormal" style="margin-left:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">htop<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">Either way, you will end up with multiple output files:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">output_00_log.txt<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">output_00.Run.dat<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">output_00.root<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">etc...</span><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">If you’re using text-based output files, there are several ways within Ubuntu to combine them.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">To combine root files, you can use the ‘hadd’ application which should be available to you:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> hadd - Merge ROOT histogram files into one file<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> hadd outputfile inputfiles ...<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New"">
<o:p></o:p></span></p>
<p class="MsoNormal" style="text-indent:.5in"><span style="font-size:10.0pt;font-family:"Courier New"">This program will add histograms from a list of ROOT files and write them to a target ROOT<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;font-family:"Courier New""> file. The target file is newly created and must not be identical to one of the source files.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hope this helps you<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">Mark Baker<o:p></o:p></span></b></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">Principal Clinical Scientist (Interim)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">Imaging Physics (Ionising)<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">The Clatterbridge Cancer Centre NHS Foundation Trust<o:p></o:p></span></p>
<p class="MsoNormal"><i><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">(he, him)<o:p></o:p></span></i></p>
<p class="MsoNormal"><b><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871"><o:p> </o:p></span></b></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">CCCW Tel: 0151 556 5030
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">CCCL Tel: 0151 318 8438<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">Email:
</span><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#4472C4"><a href="mailto:mark.baker23@nhs.net"><span style="color:#4472C4">mark.baker23@nhs.net</span></a></span><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#006871">
<o:p></o:p></span></p>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#4472C4"><a href="https://teams.microsoft.com/l/chat/0/0?users=mark.baker23@nhs.net"><span style="color:#4472C4">Microsoft Teams (click here)</span></a></span><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Gate-users <<a href="mailto:gate-users-bounces@lists.opengatecollaboration.org">gate-users-bounces@lists.opengatecollaboration.org</a>>
<b>On Behalf Of </b>olie pickford scienti<br>
<b>Sent:</b> 14 February 2022 08:27<br>
<b>To:</b> <a href="mailto:gate-users@lists.opengatecollaboration.org">gate-users@lists.opengatecollaboration.org</a><br>
<b>Subject:</b> Re: [Gate-users] Running Gate on a Cluster (Mijat Paunovic)<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<div>
<div>
<p class="MsoNormal"><span lang="EN-GB">Hello Mijat, <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB">From my experience Gate runs on a single core by default, and I do not think you can run it in parallel on multiple cores from a single file though am happy to be corrected about this. What I do to use multiple cores
on my laptop when running simulations (I also have 8 cores) is to copy the mac file but with different start and finish times and different random seeds. So for example if running for 1 second on 4 cores, have mac files which run 0-0.25 s, 0.25-0.5 s, 0.5-0.75
s and 0.75-1 s, each with a different random seed. Then I would open 4 terminals and run 4 instances of Gate, 1 for each mac script. You should then be able to combine the resulting root files, but I process my results outside of root so just export each file
individually to that framework. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB">In your case I think, so long as random seed is different and you're just looking for testing your reconstruction rather than making statistical assessments of the data, you should be fine to use (10^13)/n primaries in
each of n simulations then combine them. If n becomes very large, or you do this a lot, you may wish to write your own script for splitting the jobs if you can't get gjs working (I don't use gjs so can't advise). Hopefully someone else on here can explain
why gjs doesn't work for you though. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB">I hope this helps. <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span lang="EN-GB">Olie<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><span lang="EN-GB">On 13 Feb 2022 12:00, <a href="mailto:gate-users-request@lists.opengatecollaboration.org">
gate-users-request@lists.opengatecollaboration.org</a> wrote:<o:p></o:p></span></p>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt">
<p><span lang="EN-GB">Send Gate-users mailing list submissions to<br>
<a href="mailto:gate-users@lists.opengatecollaboration.org">gate-users@lists.opengatecollaboration.org</a><o:p></o:p></span></p>
<p><span lang="EN-GB">To subscribe or unsubscribe via the World Wide Web, visit<br>
<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0">https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0</a><br>
or, via email, send a message with subject or body 'help' to<br>
<a href="mailto:gate-users-request@lists.opengatecollaboration.org">gate-users-request@lists.opengatecollaboration.org</a><o:p></o:p></span></p>
<p><span lang="EN-GB">You can reach the person managing the list at<br>
<a href="mailto:gate-users-owner@lists.opengatecollaboration.org">gate-users-owner@lists.opengatecollaboration.org</a><o:p></o:p></span></p>
<p><span lang="EN-GB">When replying, please edit your Subject line so it is more specific<br>
than "Re: Contents of Gate-users digest..."<o:p></o:p></span></p>
<p><span lang="EN-GB">Today's Topics:<o:p></o:p></span></p>
<p><span lang="EN-GB"> 1. Running Gate on a Cluster (Mijat Paunovic)<o:p></o:p></span></p>
<p><span lang="EN-GB">----------------------------------------------------------------------<o:p></o:p></span></p>
<p><span lang="EN-GB">Message: 1<br>
Date: Sat, 12 Feb 2022 21:22:17 +0100<br>
From: Mijat Paunovic <<a href="mailto:paunovicjata@gmail.com">paunovicjata@gmail.com</a>><br>
To: <a href="mailto:gate-users@lists.opengatecollaboration.org">gate-users@lists.opengatecollaboration.org</a><br>
Subject: [Gate-users] Running Gate on a Cluster<br>
Message-ID:<br>
<<a href="mailto:CAKS_kRr=q-NXBcH_E9Bzc+EryFpT_GOps16eqy1kvZavCLCh3g@mail.gmail.com">CAKS_kRr=q-NXBcH_E9Bzc+EryFpT_GOps16eqy1kvZavCLCh3g@mail.gmail.com</a>><br>
Content-Type: text/plain; charset="utf-8"<o:p></o:p></span></p>
<p><span lang="EN-GB">Dear Gate Community,<br>
I have created a simple Cone Beam CT simulator in order to obtain<br>
projections which will be fed to a reconstruction algorithm. In order for<br>
the algorithm to work, I need to have a large number of detected photons in<br>
each pixel of the detector. In order to obtain this large number, I wanted<br>
to run the simulation for ~10^13 primaries. This is of course an impossible<br>
job for my computer which has only 8 logical cores (4 after assigning half<br>
to the virtual machine on which Gate is run) so I will get access to a<br>
supercomputer which will be able to do this.<o:p></o:p></span></p>
<p><span lang="EN-GB">In the meantime, I wonder how I could maximize the performance on my<br>
computer? When I run the Gate simulation and monitor the CPU performance, I<br>
can see that only 1/4 logical cores is being used for the simulation.<o:p></o:p></span></p>
<p><span lang="EN-GB">In Gate documentation I found the '6.1. How to use Gate on a Cluster'<br>
explaining how I should use the *gjs *(Gate job splitter) which will split<br>
my work on 4 jobs which I can independently run on each node of the<br>
cluster, after which I should use the *gjm *(Gate file merger) in order to<br>
combine the results.<o:p></o:p></span></p>
<p><span lang="EN-GB">Several questions arose from this:<br>
1. Is it possible to make Gate make the use of additional 3 cores that I<br>
have assigned to the virtual machine without doing the job splitting?<br>
2. If I perform the job splitting, can my 4 cores be considered a cluster<br>
of it's own?<br>
3. When running *gjs*, I am only able to get an output when am using<br>
*-numberofsplits* of 10 or more jobs. If I try to run it for 9 or less, I<br>
get 'floating point error'.<br>
4. Since I am only running a simulation of 10^13 independent photon<br>
histories (only interested in photon interactions within a phantom and the<br>
count of the photons on the detector) would it be the same if I just fed<br>
the same simulation to on *n *nodes, with the number of primaries being<br>
10^13/n?<o:p></o:p></span></p>
<p><span lang="EN-GB">For the first test for this simulator, I will be using it on a cylindrical<br>
phantom. Instead of making a large number of projections which will be fed<br>
to a reconstruction algorithm, I will be generating particles from only one<br>
angle and the result (2D matrix with the photon count) will be copied<br>
several hundred times and hopefully, upon reconstruction, we will get a<br>
circular cross-section.<o:p></o:p></span></p>
<p><span lang="EN-GB">All in all, I need a way to speed up the process of obtaining this 2D<br>
matrix if I am going to make the simulator more realistic.<o:p></o:p></span></p>
<p><span lang="EN-GB">Any suggestions on this matter would be extremely helpful. Thanks in<br>
advance!<o:p></o:p></span></p>
<p><span lang="EN-GB">Kind regards,<br>
Mijat Paunović<br>
-------------- next part --------------<br>
An HTML attachment was scrubbed...<br>
URL: <<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fpipermail%2Fgate-users%2Fattachments%2F20220212%2Fe318e521%2Fattachment-0001.html&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mpq2bZwjMGSTedKw2QWhGIUeWkfXQ1%2B%2BwoExIP1xiw0%3D&reserved=0">https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fpipermail%2Fgate-users%2Fattachments%2F20220212%2Fe318e521%2Fattachment-0001.html&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=mpq2bZwjMGSTedKw2QWhGIUeWkfXQ1%2B%2BwoExIP1xiw0%3D&reserved=0</a>><o:p></o:p></span></p>
<p><span lang="EN-GB">------------------------------<o:p></o:p></span></p>
<p><span lang="EN-GB">Subject: Digest Footer<o:p></o:p></span></p>
<p><span lang="EN-GB">_______________________________________________<br>
Gate-users mailing list<br>
<a href="mailto:Gate-users@lists.opengatecollaboration.org">Gate-users@lists.opengatecollaboration.org</a><br>
<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0">https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Flists.opengatecollaboration.org%2Fmailman%2Flistinfo%2Fgate-users&data=04%7C01%7C%7Cc6c4214ec02b4feee08e08d9eee56f6b%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637803491457006083%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p3da2LQQypsTp4beX1sPcTnu7HwV85pAFvkUYNpnbWI%3D&reserved=0</a><o:p></o:p></span></p>
<p><span lang="EN-GB">------------------------------<o:p></o:p></span></p>
<p><span lang="EN-GB">End of Gate-users Digest, Vol 189, Issue 13<br>
*******************************************<o:p></o:p></span></p>
</blockquote>
</div>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
</div>
</div>
</div>
<p><span lang="EN-GB"> <o:p></o:p></span></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" align="left" width="100%" style="width:100.0%;margin-left:.45pt;margin-right:.45pt">
<tbody>
<tr style="height:10.5pt;height:px">
<td style="background:#910A19;padding:5.25pt 1.5pt 5.25pt 1.5pt;height:10.5pt">
<p class="MsoNormal" style="mso-element:frame;mso-element-frame-hspace:2.7pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<o:p></o:p></p>
</td>
<td width="100%" style="width:100.0%;background:#FDF2F4;padding:0in 3.75pt 0in 6.25pt;height:10.5pt">
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-element:frame;mso-element-frame-hspace:2.7pt;mso-element-wrap:around;mso-element-anchor-vertical:paragraph;mso-element-anchor-horizontal:column;mso-height-rule:exactly">
<span style="font-size:9.0pt;font-family:"Segoe UI",sans-serif;color:#212121">This message originated from outside of NHSmail. Please do not click links or open attachments unless you recognise the sender and know the content is safe.
</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="MsoNormal"><span lang="EN-GB"><o:p> </o:p></span></p>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif"><br>
</span><span lang="EN-GB" style="font-size:10.0pt;font-family:"Courier New";color:black"><br>
************************************************************************************** ******************************<br>
<br>
This message may contain confidential information. If you are not the intended recipient please:<br>
i) inform the sender that you have received the message in error before deleting it; and
<br>
ii) do not disclose, copy or distribute information in this e-mail or take any action in relation to its content (to do so is strictly prohibited and may be unlawful).
<br>
Thank you for your co-operation.<br>
<br>
NHSmail is the secure email, collaboration and directory service available for all NHS staff in England. NHSmail is approved for exchanging patient data and other sensitive information with NHSmail and other accredited email services.<br>
<br>
For more information and to find out how you can switch visit <a href="https://support.nhs.net/article-categories/joining-nhsmail/">
Joining NHSmail – NHSmail Support</a></span><span lang="EN-GB" style="font-size:11.0pt;font-family:"Calibri",sans-serif"><o:p></o:p></span></p>
</div>
</div>
<span>_______________________________________________</span><br>
<span>Gate-users mailing list</span><br>
<span>Gate-users@lists.opengatecollaboration.org</span><br>
<span>http://lists.opengatecollaboration.org/mailman/listinfo/gate-users</span></div>
</blockquote>
</div>
</body>
</html>