[Gate-users] Simulation crashes unexpectedly

David Boersma david.boersma at acmit.at
Wed Feb 3 00:09:32 CET 2021


Hi Matthew and Cassandra,

Segfaults are embarrassing and should be fixed. The backtrace sometimes helps, but I didn't see any clear hints in the one you provided. In order to have a chance to solve it we'd need a minimal Gate macro that enables us to reproduce this problem. Use a fixed seed and tell us after how long it runs before it crashes for you. It can sometimes help if in addition the versions of Gate, Geant4 and ROOT, you also specify which OS (name and release), which compiler and some specs of the hardware you run it on. Please post all these details in a GitHub #issue so that it won't get buried in history.

Debugging this sort of problems is very necessary (because they are so embarrassing!), but it is also hard and time consuming (run it under "valgrind" and study the very verbose logs), so I don't guarantee that it will be fixed any time soon, even if we have a GitHub #issue with the full information. Sorry for the inconvenience!

/David


________________________________
From: Gate-users <gate-users-bounces at lists.opengatecollaboration.org> on behalf of Matthew Strugari <matthew.strugari at dal.ca>
Sent: Tuesday, February 2, 2021 4:23 PM
To: Cassandra Miller <cassandramiller at phas.ubc.ca>; gate-users at lists.opengatecollaboration.org <gate-users at lists.opengatecollaboration.org>
Subject: Re: [Gate-users] Simulation crashes unexpectedly


Hi Cassandra,



Very strange. I also think it may be a memory issue, although maybe not related to simulation size in my case. My total simulation consumes ~4.6G of RAM out of 64G available. The issue has also never occurred during dosimetry-related experiments.



I noticed that the issue began after I made some changes to my macro file in order to ensure that whole runs were distributed amongst the jobs. This required GATE to perform a number of internal alias comparisons and assignments. I have since cleaned up the alias and doif commands so hopefully that eliminates the issue since there were never any problems before adding those commands. Does your SPECT configuration utilize a lot of alias and doif commands as well?



As another developer pointed out to me, the stderr printout means that a pointer to a physical volume pointed to an area of unused memory. I have always used the auto seed generator with the MersenneTwister option. If the issue persists, then I will manually set the seed to check if it crashes repeatedly. If it doesn’t crash with the same seed, then I am told it may be a problem with my computer.



Thank you for your input!

Matthew





From: Cassandra Miller <cassandramiller at phas.ubc.ca>
Date: Monday, February 1, 2021 at 19:35
To: Matthew Strugari <matthew.strugari at dal.ca>
Subject: Re: [Gate-users] Simulation crashes unexpectedly

CAUTION: The Sender of this email is not from within Dalhousie.

Hi Matthew,

I don't have any insight into why this happens, but it happens to me ALL the time. The frequency it happens depends on the size of the simulation, I think it's a memory thing somehow. It's never happened to me using GATE actors for dosimetry, but it seems to exclusively happen when I'm doing SPECT imaging.

The more jobs I split my simulation into, the less this happens. So sometimes in the past when I've ran simulations that are ridiculous in size or when I'm using specific sources (for example it happens way more with the fastY90 source than an ion source) I've split my sim into a stupid amount of jobs and then summed them up at the end. And wrote some scripts to automatically open and list the jobs that have crashed, go inside and change the engine seed number (I find if I change the engine seed it will not crash when I rerun it, but if I don't change the engine seed it will crash again), and re-run them.

It sounds a bit ridiculous but.. I've opted to do that instead of figure out what causes the problem in the first place! So I'm sorry I can't help you with the root of the problem but there are ways around it at least.

Best,
Cassandra

--



Cassandra Miller

PhD Student, Physics

Quantitative Radiomolecular Imaging and Therapy Lab

Department of Physics and Astronomy, University of British Columbia

On 2021-02-01 2:36 p.m., Matthew Strugari wrote:

Hi all,



I am having difficulty determining why my simulation is crashing (see attached log file). I am using the task spooler to split my simulation into 12 jobs. Sometimes, one or more jobs will crash at the beginning of a run, but the run that GATE crashes on seems to be random (jobs have either 5 or 6 runs/projection images). Other times, all of the jobs will run to completion. After a job has crashed, I can reissue the command and the job will run to completion without error.



I cannot figure out why the job(s) seemingly crash at random. Does anyone have any insight into the issue? Your assistance is greatly appreciated.



Regards,

Matthew



--

Matthew Strugari

Biomedical Translational Imaging Centre - BIOTIC,
5890 University Ave,
Halifax, NS, B3K 6R8



_______________________________________________

Gate-users mailing list

Gate-users at lists.opengatecollaboration.org<mailto:Gate-users at lists.opengatecollaboration.org>

http://lists.opengatecollaboration.org/mailman/listinfo/gate-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opengatecollaboration.org/pipermail/gate-users/attachments/20210202/380496fd/attachment-0001.html>


More information about the Gate-users mailing list