[Gate-users] Simulation crashes unexpectedly

[Matthew Strugari]

Hi Cassandra,

Very strange. I also think it may be a memory issue, although maybe not related to simulation size in my case. My total simulation consumes ~4.6G of RAM out of 64G available. The issue has also never occurred during dosimetry-related experiments.

I noticed that the issue began after I made some changes to my macro file in order to ensure that whole runs were distributed amongst the jobs. This required GATE to perform a number of internal alias comparisons and assignments. I have since cleaned up the alias and doif commands so hopefully that eliminates the issue since there were never any problems before adding those commands. Does your SPECT configuration utilize a lot of alias and doif commands as well?

As another developer pointed out to me, the stderr printout means that a pointer to a physical volume pointed to an area of unused memory. I have always used the auto seed generator with the MersenneTwister option. If the issue persists, then I will manually set the seed to check if it crashes repeatedly. If it doesn’t crash with the same seed, then I am told it may be a problem with my computer.

Thank you for your input!
Matthew


From: Cassandra Miller <cassandramiller at phas.ubc.ca>
Date: Monday, February 1, 2021 at 19:35
To: Matthew Strugari <matthew.strugari at dal.ca>
Subject: Re: [Gate-users] Simulation crashes unexpectedly
CAUTION: The Sender of this email is not from within Dalhousie.

Hi Matthew,

I don't have any insight into why this happens, but it happens to me ALL the time. The frequency it happens depends on the size of the simulation, I think it's a memory thing somehow. It's never happened to me using GATE actors for dosimetry, but it seems to exclusively happen when I'm doing SPECT imaging.

The more jobs I split my simulation into, the less this happens. So sometimes in the past when I've ran simulations that are ridiculous in size or when I'm using specific sources (for example it happens way more with the fastY90 source than an ion source) I've split my sim into a stupid amount of jobs and then summed them up at the end. And wrote some scripts to automatically open and list the jobs that have crashed, go inside and change the engine seed number (I find if I change the engine seed it will not crash when I rerun it, but if I don't change the engine seed it will crash again), and re-run them.

It sounds a bit ridiculous but.. I've opted to do that instead of figure out what causes the problem in the first place! So I'm sorry I can't help you with the root of the problem but there are ways around it at least.

Best,
Cassandra

--



Cassandra Miller

PhD Student, Physics

Quantitative Radiomolecular Imaging and Therapy Lab

Department of Physics and Astronomy, University of British Columbia
On 2021-02-01 2:36 p.m., Matthew Strugari wrote:
Hi all,

I am having difficulty determining why my simulation is crashing (see attached log file). I am using the task spooler to split my simulation into 12 jobs. Sometimes, one or more jobs will crash at the beginning of a run, but the run that GATE crashes on seems to be random (jobs have either 5 or 6 runs/projection images). Other times, all of the jobs will run to completion. After a job has crashed, I can reissue the command and the job will run to completion without error.

I cannot figure out why the job(s) seemingly crash at random. Does anyone have any insight into the issue? Your assistance is greatly appreciated.

Regards,
Matthew

--
Matthew Strugari
Biomedical Translational Imaging Centre - BIOTIC,
5890 University Ave,
Halifax, NS, B3K 6R8



_______________________________________________

Gate-users mailing list

Gate-users at lists.opengatecollaboration.org<mailto:Gate-users at lists.opengatecollaboration.org>

http://lists.opengatecollaboration.org/mailman/listinfo/gate-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.opengatecollaboration.org/pipermail/gate-users/attachments/20210202/84c85791/attachment.html>