Using Endeavour for the first time today (QCB), and I ran into an issue which I have never experienced on Discovery. A bunch of my jobs crashed, and it seems like the reason is that they write a temporary file to the directory
$TMPDIR (which I assumed was unique to that job), and delete it when they are done. On Endeavour, the crashed jobs were complaining that this temporary file didn’t exist when they went to remove it.
I can only reason that this is because jobs which are sharing the same node on Endeavour are also sharing the same
$TMPDIR. If so, this results in clashes between jobs sharing the same node.
This this correct that
$TMPDIR is shared by different jobs running on the same node? If so, is this behavior different on Discovery? What is the intention?