Using the Scratch Labeled by Job Number

  • The scratch directory is probably the hardest to use portion of the cluster, but the idea of it is quite simple: it is a directory where files and functions can be temporarily stored for quicker access by the nodes.
  • There are two ways to use the scratch. Either it can be used as a purely temporary file location (detailed here) or as a longer-term storage location for larger files and functions that would cause your local directory to run out of storage space (next).
  • This time, instead of creating a matrix and then saving it, we’re going to copy an existing matrix from our personal directory to a temporary scratch location, load it into our function, take its transpose, and save the new matrix back in our personal directory. This is a simple thing to do in MATLAB, but it is complicated enough that once you understand how this works you’ll be able to run just about anything that you want using the scratch.
  • The .serial file has a few new lines, so it is a good place to start.
    1. Most notably at the top is -R, which defines how much memory you want to reserve for the codes and files to be copied to this temporary folder. It is measured in MB, so -R “rusage[mem=4000]” gives us 4 GB of storage space.
    2. This .serial file also defines a variable, MYSANSCRATCH, to be a folder within /sanscratch/ that is automatically created when the job starts. The next line changes directories to this folder.
    3. The next thing to do is copy everything that you’re going to need over to the scratch. Don’t forget this step when if you have a function that calls other functions: everything that you’re going to try to access needs to be copied to the sanscratch unless another path is specified within the function.
  • After that, you need to make sure that your function knows how to access any new files that might have been copied over. This is why $MYSANSCRATCH is an input to the function, as is the size of the matrix that was copied to the scratch folder.
  • Once you have changed the save path within the MATLAB function to be somewhere in your home directory, run the .serial file and you should see the new matrix appear in the specified folder.