<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman",serif;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body bgcolor="white" lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Hi Luke,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Yes, we are aware of this issue. The benchmark tests are very rigid, they require the output to be identical to the reference result. This does not make a lot
of sense, physics-wise. The benchmark directories also contain analysis scripts with which you can investigate the results yourself, and you’ll see that at least for the RT cases the simulation results are statistically compatible with the reference.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">I think that at the recent Gate collaboration meeting(s) there was a consensus to replace these identity tests with statistical tests, or at least tests that
are more physically meaningful. E.g. for the proton and carbon beam tests, we could determine the range (and related observables) from the simulated dose distribution and check that it is statistically compatible with the range obtained from the reference
dose distribution. And/or we could run a gamma index test. We could include tests that mimic the typical validation/commissioning tests that are done at clinics; such tests may take some time to run, but for validating a GATE release it should be OK to run
some thorough tests that may last a couple of hours (or maybe even days!).<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">There are many more problems with the current benchmark tests. For starters, there should be *<b>many</b>* more tests, in order to offer a minimal validation
of a good portion (ideally *<b>all</b>*) of the functionality of Gate. I am working on this issue, and I hope that in a few weeks I can commit some prototype of a new testing suite to the develop branch. Expanding the testing suite to perform meaningful tests
of a large part of Gate will be a pretty big undertaking, I hope that there more users/developers who also care about this, enough to join the effort and develop & contribute some good tests.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">Kind regards,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D">David<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri",sans-serif;color:#1F497D"><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri",sans-serif"> Gate-users [mailto:gate-users-bounces@lists.opengatecollaboration.org]
<b>On Behalf Of </b>Luke Gilling<br>
<b>Sent:</b> Tuesday, June 12, 2018 02:26<br>
<b>To:</b> gate-users@lists.opengatecollaboration.org<br>
<b>Subject:</b> [Gate-users] Gate v8.1 failing validation tests<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p><span style="font-family:"Calibri",sans-serif;color:black">Hello everyone,<o:p></o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black"><o:p> </o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black">I have just finished installing gate v 8.1 and the command 'Gate' runs the software. However when I run the benchmark tests via the 'make test' command I get the following output:<o:p></o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black"><o:p> </o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black;background:#BEBEBE">Running tests...</span><span style="font-family:"Calibri",sans-serif;color:black"><br>
<span style="background:#BEBEBE">Test project /home/physics/Progs/gate_v8.1-build</span><br>
<span style="background:#BEBEBE"> Start 1: benchRT_gamma</span><br>
<span style="background:#BEBEBE">1/7 Test #1: benchRT_gamma ....................***Failed 36.44 sec</span><br>
<span style="background:#BEBEBE"> Start 2: benchRT_proton</span><br>
<span style="background:#BEBEBE">2/7 Test #2: benchRT_proton ...................***Failed 2.59 sec</span><br>
<span style="background:#BEBEBE"> Start 3: benchRT_carbon</span><br>
<span style="background:#BEBEBE">3/7 Test #3: benchRT_carbon ...................***Failed 15.95 sec</span><br>
<span style="background:#BEBEBE"> Start 4: benchImaging_ct</span><br>
<span style="background:#BEBEBE">4/7 Test #4: benchImaging_ct ..................***Failed 49.31 sec</span><br>
<span style="background:#BEBEBE"> Start 5: benchImaging_optical</span><br>
<span style="background:#BEBEBE">5/7 Test #5: benchImaging_optical .............***Failed 6.35 sec</span><br>
<span style="background:#BEBEBE"> Start 6: benchImaging_spect</span><br>
<span style="background:#BEBEBE">6/7 Test #6: benchImaging_spect ...............***Failed 70.11 sec</span><br>
<span style="background:#BEBEBE"> Start 7: benchImaging_pet</span><br>
<span style="background:#BEBEBE">7/7 Test #7: benchImaging_pet .................***Failed 0.97 sec</span><br>
<br>
<span style="background:#BEBEBE">0% tests passed, 7 tests failed out of 7</span><br>
<br>
<span style="background:#BEBEBE">Total Test time (real) = 181.77 sec</span><o:p></o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black"><o:p> </o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black">Any ideas on what I can do from here? Is this a problem with the physics data?<o:p></o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black"><o:p> </o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black">Kind regards,<o:p></o:p></span></p>
<p><span style="font-family:"Calibri",sans-serif;color:black">Luke Gilling<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-family:"Calibri",sans-serif;color:black">This electronic message, together with any attachments is confidential and may be privileged. If you are not the intended recipient: 1.do not copy, disclose or use the contents
in any way. 2.please let me know by return email immediately and then destroy the message. Waikato DHB is not responsible for any changes made to this message and/or any attachments after sending by Waikato DHB. Before opening or using attachments, check them
for viruses and effects. Waikato DHB takes no responsibility for affected attachments. Click on link
<a href="http://www.waikatodhb.health.nz/disclaimer">www.waikatodhb.health.nz/disclaimer</a> to view the company policy website. If you are not redirected to the company policy website then copy and paste the URL into a new browser window.
<o:p></o:p></span></p>
<div>
<div style="border-top:solid black 1.0pt;border-left:none;border-bottom:solid black 1.0pt;border-right:none;padding:8.0pt 0in 8.0pt 0in;margin-top:15.0pt;margin-bottom:15.0pt">
<p class="MsoNormal"><span style="font-size:9.0pt;font-family:"Verdana",sans-serif;color:black">This email has been filtered by SMX. For more information visit
<a href="https://emea01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fsmxemail.com%2F&data=02%7C01%7Cdavid.boersma%40medaustron.at%7C0fbed36fc0854fc6ed7e08d5cffb54e0%7Cadb749e06dfd4f6fbea8cd9f8e0c9b89%7C0%7C0%7C636643600733921199&sdata=m90mds%2Bng5BtJHChgKLHOLlJYBLPISjQVNwCBLxedSU%3D&reserved=0">
smxemail.com</a><o:p></o:p></span></p>
</div>
</div>
</div>
</div>
<div class="WordSection1">
<p class="MsoNormal"><small><span style=""><o:p></o:p></span><b><span style="font-family: "Verdana","sans-serif"; color: rgb(17, 35, 89);" lang="EN-GB">Disclaimer:</span></b><br>
<span style="font-family: "Verdana","sans-serif"; color: rgb(17, 35, 89);" lang="EN-GB">Please notice our E-Mail Disclaimer <a href="http://www.medaustron.at/email-disclaimer/"><span style="color: rgb(17, 35, 89);">http://www.medaustron.at/email-disclaimer/</span></a></span></small><span style=""><o:p></o:p></span><o:p> </o:p></p>
</div>
<br>
</body>
</html>