[Gate-users] gate on cluster: speedup

[Philipp Matthies]

Dear GT,

what the job splitter does is split up your simulations into 40
sub-simulations NOT by splitting the total number of primary particles
but by splitting the TIME instead!

This lines
/gate/application/startDAQCluster 0 0.025 0 s
/gate/application/startDAQCluster 0.025 0.05 0 s
...
/gate/application/startDAQCluster 0.975 1 0 s

indicate this.

The total simulation time of 1s is split into 40 sub-simulations, each
with a duration of 0.025s. Run 1 simulated from time 0.0 to 0.025, run
2 from 0.025 to 0.05 and so on and so forth.

Best regards,
Philipp

---
Mit freundlichen Grüßen / Kind regards,
Philipp Matthies

Lead Engineer Nuclear Imaging
SurgicEye GmbH - www.surgiceye.com

2017-05-11 20:43 GMT+02:00 G Tom <gregthom992 at gmail.com>:
> Thanks Reza,
>
> I undertand that line you mentioned with gjs. That is what  I used to split
> my main.mac into N=40 where N is the number of cores. However, in each of
> the
>
> main1.mac
> main2.mac
> ....
> main40.mac
>
> I find the lines
>
> in main1.mac:
> ...
> /gate/application/setTotalNumberOfPrimaries 1e8
> /gate/random/setEngineSeed 2267427155918198868
> /gate/application/setTimeStart 0. s
> /gate/application/addSlice     1. s
> /gate/application/startDAQCluster 0 0.025 0 s
>
> in main2.mac:
> ...
> /gate/application/setTotalNumberOfPrimaries 1e8
> /gate/random/setEngineSeed 1260235483062616104
> /gate/application/setTimeStart 0. s
> /gate/application/addSlice     1. s
> /gate/application/startDAQCluster 0.025 0.05 0 s
>
> ...
>
> and in main40.mac
>
> /gate/application/setTotalNumberOfPrimaries 1e8
> /gate/random/setEngineSeed 1683838443918737612
> /gate/application/setTimeStart 0. s
> /gate/application/addSlice     1. s
> /gate/application/startDAQCluster 0.975 1 0 s
>
>
> My question is this: is each one simulating 1e8 particles ? What does the
> line /gate/application/startDAQCluster 0.975 1 0 s mean ?
>
> If possible, can someone provide example of gjs with the two methods of
> splitting : event based and time based splitting as mentioned in the talk
> of D. Sarrut:
> http://www.opengatecollaboration.org/sites/default/files/Talk6.pdf.
>
> Also I want a moving phantom that moves at a certain speed: how should I
> account for that in the splitting of my macros ?
>
> Thanks
>
> GT
>
> On Thu, May 11, 2017 at 1:47 PM, M.R Teimoori <reza.teimoori.s at gmail.com>
> wrote:
>>
>> Hi,
>>
>> As far as I know the following command splits the total acquisition time
>> to N subsets (N = number of physical CPU cores) and generates N macros with
>> various random seed generator numbers in a folder called ".Gate" as well as
>> a "submit" file right next to your actual macro file located at let's say
>> "/address/to/your/macro.mac" .
>>
>> Go to GATE's  "jobsplitter" address and run:
>> ./gjs -numberofsplits N -clusterplatform
>> CHOOSE_A_PLATFORM_BASED_ON_CLUSTER_ARCHITECTURE  /address/to/your/macro.mac
>>
>> From now on ignore your original macro file (macro.mac) or just delete it.
>> In order to run the generated macros, all you have to do is to execute the
>> "submit" file. As a side note, I am also not quite sure what "0.95 0.975 0
>> s" part does in your macro.
>>
>> Hope that helps!
>>
>> Cheers,
>> Reza
>>
>> On Wed, May 10, 2017 at 11:07 PM, G Tom <gregthom992 at gmail.com> wrote:
>>>
>>> Hi all,
>>> I just had a question about running gate on a cluster.
>>> I used gjs to split my simulation into 40 macros. I was simulating 1e8
>>> particles for a radiotherapy application.
>>> Now I thought the 40 jobs will each run 1e8/40 particles but  I am seeing
>>> instead that each is running 1e8 macros ?
>>>
>>>  I took a look at the simulations
>>> statistics output and indeed each job runs 1e8 particles:
>>>
>>> ==============
>>> # NumberOfRun    = 1
>>> # NumberOfEvents = 100000000
>>> # NumberOfTracks = 1754799115
>>> # NumberOfSteps  = 5139294665
>>> # NumberOfGeometricalSteps  = 330040574
>>> # NumberOfPhysicalSteps     = 4809254091
>>> # ElapsedTime           = 69568.3
>>> # ElapsedTimeWoInit     = 69559.5
>>> # StartDate             = Wed May 10 00:20:45 2017
>>> # EndDate               = Wed May 10 19:40:14 2017
>>> # StartSimulationTime        = 0
>>> # StopSimulationTime         = 1
>>> # CurrentSimulationTime      = 1.2
>>> # VirtualStartSimulationTime = 0.2
>>> # VirtualStopSimulationTime  = 0.225
>>> # ElapsedSimulationTime      = 1
>>> # PPS (Primary per sec)      = 1437.62
>>> # TPS (Track per sec)        = 25227.3
>>> # SPS (Step per sec)         = 73883.4
>>>
>>>
>>> examples of split macro files are :
>>> ..
>>> /gate/application/setTotalNumberOfPrimaries 1e8
>>> /gate/random/setEngineSeed 1683838443918737612
>>> /gate/application/setTimeStart 0. s
>>> /gate/application/addSlice     1. s
>>> /gate/application/startDAQCluster 0.975 1 0 s
>>>
>>> .....
>>> /gate/application/setTotalNumberOfPrimaries 1e8
>>> /gate/random/setEngineSeed 313149272728410404
>>> /gate/application/setTimeStart 0. s
>>> /gate/application/addSlice     1. s
>>> /gate/application/startDAQCluster 0.95 0.975 0 s
>>>
>>>
>>> I am not sure I fully understand how to obtain speedup by actually
>>> splitting the
>>> number of simulated particles using gjs,
>>>
>>> Thanks
>>>
>>> GT
>>>
>>>
>>>
>>> _______________________________________________
>>> Gate-users mailing list
>>> Gate-users at lists.opengatecollaboration.org
>>> http://lists.opengatecollaboration.org/mailman/listinfo/gate-users
>>
>>
>
>
> _______________________________________________
> Gate-users mailing list
> Gate-users at lists.opengatecollaboration.org
> http://lists.opengatecollaboration.org/mailman/listinfo/gate-users