[Gate-users] Adding G4MTRunManager Support to GATE

Alex Vergara Gil alexvergaragil at gmail.com
Tue Mar 31 14:57:35 CEST 2015


Dear All

I have managed to create a patch that makes gate using G4MTRunManager,
it compiles fine and run, but I am facing that it doesn't run in
several threads, I need somebody who guide me in the right direction.

Dear Marc

Thank a lot for your suggestions they help me a lot in creating this
patch. Can you or some G4 member take a look to this and see what is
happening here.

Regards
Alex

PS: Dear Marc, sorry for double mail you, I missed to check the
respond to all tick.

2015-03-26 9:31 GMT-04:00, Marc Verderi <verderi at in2p3.fr>:
> Dear All,
>
>      The interest of G4MTRunManager is that the geometry and the
> cross-section tables are shared among the threads. For big applications
> -and simulation of phantom irradiation is one example- this represents a
> large memory. For machines with many cores, spawning N jobs of such
> application may exhaust the memory, preventing to use all the available
> cores. By sharing geometry and cross-section tables, the G4MTRunManager
> saves a large fraction of memory, allowing to use much more cores. Some
> tests have been done by Geant4, on Xeon Phi, see for example
> https://twiki.cern.ch/twiki/bin/view/Geant4/MultiThreadingTaskForce#CPU_and_Memory_Performances
> and one single application of high energy physics type (simplified CMS
> simulation) could run smoothly with 240 threads, the maximum available
> (the machine has 60 user cores, up to 4 thread/core). Without MT, just
> spawning jobs, only ~30 jobs could have been run in parallel, leaving 30
> cores unoccupied, because of lack of memory !
>
>      Moving to multi-threading has some constraints. Each thread
> processes a bunch of events. Events are hence generated and processed in
> parallel, independently. This means that primary generator action, event
> action, stepping action have to have independent instances in each
> thread. This is the very purpose of the new class
> G4VUserActionInitialization : the method "Build()" is called for each
> thread, to instantiate in each of these the above actions. For the run
> action it is a bit more complicated : a run action may be for the entire
> application, or may be for each thread. For an "all application" action,
> BuildForMaster() has to be used.
>      This independence of threads has a similar impact on sensitive
> detectors : for these, the G4VUserDetectorConstruction class has a new
> method : ConstructSDandField(). Again, this method is called for each
> thread, so that sensitive detectors and fields live independent lives in
> the various threads.
>      This looks quite work, but is not that heavy in practice.
>
>      In practice also, what has to be taken care of in your code are
> "static" variables : at each occurence of a static variable, you have to
> think if this variable has to be common to the entire application -a
> "true" static-, or if it is common the thread only : "thread local"
> static. In most of the cases, static variables are static to the thread.
> For the case of a true "static", be aware that this means that each
> thread may access the variable, at any time. If this variable is read &
> write during the processing, it will have a quite unpredictable
> behavior, and this is a source of debugging headaches ;) . Any random
> crash -which are often non-reproducible between two processing- signs
> this sort of conflict.
>
>      Most of the G4 examples (basic, extended) are provided in MT mode,
> and are good starting points.
>
>      Hope this helps.
>
> Cheers,
> Marc (a G4 member)
>
>
> On 03/26/2015 01:02 PM, Alex Vergara Gil wrote:
>> Dear All
>>
>> I started this thread to unify all those enthusiast people who want to
>> add G4MTRunManager support into GATE. The advantages of a Multi
>> Threading Run Manager are obvious but I will explain it here anyway, I
>> will send you also my first patch and the problems I am facing.
>>
>> Advantages
>> 1. You will not depend on an external cluster software to run on a
>> single multi cpu PC.
>> 2. The time you need for simulation scales linearly with the number of
>> cpus
>> 3. You doesn't need to merge the outputs, since this is performed
>> automatically
>> 4. Any other you may add
>>
>> My first patch
>>
>> <start of the code>
>>
>> Autor: Alex Vergara Gil <alexvergaragil at gmail.com>  2015-03-25 17:13:45
>> Committer: Alex Vergara Gil <alexvergaragil at gmail.com>  2015-03-25
>> 17:13:45
>> Padre: db6875e64d60ad1e0f2d100c496843632acb23c8 (Merge
>> https://github.com/OpenGATE/Gate)
>> Hija:  28c338cd3263108df3927db14c6975f4cdcc31b4 (Agregado el
>> UserActionInitialization)
>> Rama: partopc
>> Sigue-a:
>> Precede-a:
>>
>>      trying g4mtRunManager
>>
>> ------------------- source/general/include/GateRunManager.hh
>> -------------------
>> index c4164d9..b72327b 100644
>> @@ -28,12 +28,19 @@
>>   #define GateRunManager_h 1
>>
>>   #include "G4RunManager.hh"
>> +#ifdef G4MULTITHREADED
>> + #include "G4MTRunManager.hh"
>> +#endif
>>   #include "GateHounsfieldToMaterialsBuilder.hh"
>>
>>   class GateRunManagerMessenger;
>>   class GateDetectorConstruction;
>>
>> +#ifdef G4MULTITHREADED
>> +class GateRunManager : public G4MTRunManager
>> +#else
>>   class GateRunManager : public G4RunManager
>> +#endif
>>   {
>>   public:
>>     //! Constructor
>> @@ -60,8 +67,11 @@ public:
>>
>>     //! Return the instance of the run manager
>>     static GateRunManager* GetRunManager()
>> +  #ifdef G4MULTITHREADED
>> +  {	return
>> dynamic_cast<GateRunManager*>(G4MTRunManager::GetRunManager()); }
>> +  #else
>>     {	return dynamic_cast<GateRunManager*>(G4RunManager::GetRunManager());
>> }
>> -
>> +  #endif
>>     bool GetGlobalOutputFlag() { return mGlobalOutputFlag; }
>>     void EnableGlobalOutput(bool b) { mGlobalOutputFlag = b; }
>>     void SetUserPhysicList(G4VUserPhysicsList * m) { mUserPhysicList = m;
>> }
>>
>> --------------------- source/general/src/GateRunManager.cc
>> ---------------------
>> index 2604e47..75b3fb5 100644
>> @@ -8,6 +8,9 @@
>>
>>
>>   #include "GateRunManager.hh"
>> +#ifdef G4MULTITHREADED
>> + #include "G4MTRunManager.hh"
>> +#endif
>>   #include "GateDetectorConstruction.hh"
>>   #include "GateRunManagerMessenger.hh"
>>   #include "GateHounsfieldToMaterialsBuilder.hh"
>> @@ -27,7 +30,11 @@
>>   #endif
>>
>>
>> //----------------------------------------------------------------------------------------
>> +#ifdef G4MULTITHREADED
>> +GateRunManager::GateRunManager():G4MTRunManager()
>> +#else
>>   GateRunManager::GateRunManager():G4RunManager()
>> +#endif
>>   {
>>     pMessenger = new GateRunManagerMessenger(this);
>>     mHounsfieldToMaterialsBuilder = new
>> GateHounsfieldToMaterialsBuilder();
>> @@ -112,7 +119,11 @@ void GateRunManager::InitializeAll()
>>
>> G4ProductionCutsTable::GetProductionCutsTable()->GetHighEdgeEnergy());
>>
>>       // Initialization
>> +#ifdef G4MULTITHREADED
>> +    G4MTRunManager::SetUserInitialization(mUserPhysicList);
>> +#else
>>       G4RunManager::SetUserInitialization(mUserPhysicList);
>> +#endif
>>
>>       //To take into account the user cuts (steplimiter and special cuts)
>>   #if (G4VERSION_MAJOR > 9)
>> @@ -126,7 +137,11 @@ void GateRunManager::InitializeAll()
>>     } // End if (mUserPhysicListName != "")
>>
>>     // InitializePhysics
>> +#ifdef G4MULTITHREADED
>>     G4RunManager::InitializePhysics();
>> +#else
>> +  G4MTRunManager::InitializePhysics();
>> +#endif
>>
>>     // Take into account the em option set by the user (dedx bin etc)
>>     GatePhysicsList::GetInstance()->SetEmProcessOptions();
>> @@ -169,7 +184,11 @@ void GateRunManager::InitGeometryOnly()
>>     if (!geometryInitialized)
>>       {
>>         GateMessage("Core", 1, "Initialization of geometry" << G4endl);
>> +#ifdef G4MULTITHREADED
>> +      G4MTRunManager::InitializeGeometry();
>> +#else
>>         G4RunManager::InitializeGeometry();
>> +#endif
>>       }
>>     else
>>       {
>> @@ -189,7 +208,11 @@ void GateRunManager::InitGeometryOnly()
>>
>> //----------------------------------------------------------------------------------------
>>   void GateRunManager::InitPhysics()
>>   {
>> +  #ifdef G4MULTITHREADED
>> +  G4MTRunManager::InitializePhysics();
>> +#else
>>     G4RunManager::InitializePhysics();
>> +#endif
>>   }
>>
>> //----------------------------------------------------------------------------------------
>>
>> @@ -205,7 +228,11 @@ void GateRunManager::RunInitialization()
>>
>>     // GateMessage("Core", 0, "Initialization of the run " << G4endl);
>>     // Perform a regular initialisation
>> +  #ifdef G4MULTITHREADED
>> +  G4MTRunManager::RunInitialization();
>> +#else
>>     G4RunManager::RunInitialization();
>> +#endif
>>
>>     // Initialization of the atom deexcitation processes
>>     // must be done after all other initialization
>>
>> </end of the code>
>>
>> This patch compiles without any special warnings, however when I try
>> to run it it explodes with the following message
>>
>> <start of message>
>> [G4]
>> [G4] *************************************************************
>> [G4]  Geant4 version Name: geant4-10-01 [MT]   (5-December-2014)
>> [G4]   << in Multi-threaded mode >>
>> [G4]                       Copyright : Geant4 Collaboration
>> [G4]                       Reference : NIM A 506 (2003), 250-303
>> [G4]                             WWW : http://cern.ch/geant4
>> [G4] *************************************************************
>> [G4]
>>          [G4-cerr]
>> -------- EEEE ------- G4Exception-START -------- EEEE -------
>> *** G4Exception : Run3011
>>        issued by : G4MTRunManager::SetUserAction()
>> For multi-threaded version, define G4UserEventAction in
>> G4VUserActionInitialization.
>> *** Fatal Exception *** core dump ***
>> -------- EEEE -------- G4Exception-END --------- EEEE -------
>>
>>          [G4-cerr]
>>          [G4-cerr] *** G4Exception: Aborting execution ***
>> Abortado (`core' generado)
>> </end of message>
>>
>> So I wonder if some of you have ever face this situation and how to help
>> me
>>
>> Best Regards
>> Alex
>> _______________________________________________
>> Gate-users mailing list
>> Gate-users at lists.opengatecollaboration.org
>> http://lists.opengatecollaboration.org/mailman/listinfo/gate-users
>
> _______________________________________________
> Gate-users mailing list
> Gate-users at lists.opengatecollaboration.org
> http://lists.opengatecollaboration.org/mailman/listinfo/gate-users
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: gateMT.patch
Type: text/x-patch
Size: 9534 bytes
Desc: not available
URL: <http://lists.opengatecollaboration.org/mailman/private/gate-users/attachments/20150331/6662a918/attachment-0001.bin>


More information about the Gate-users mailing list