...
Sign up for a free bitbucket.org account. Once you have an account, email yantosca@seasColin Thackray (thackray @ g.harvard.edu or elundgren@seas.harvard.edu to ) to be added to the Harvard BGC group on Bitbucket. All users of the Harvard BGC group will automatically have access to the MITgcm_code repository.
...
You should always use an interactive session for any work when working on Odyssey rather than work on the login node. Working on the login node reduces your available memory and generally slows down the network for other users.
You can use the "interactive" script (downloaded to your bin/ directory during initial setup) to request an interactive session. The requested number of CPUS, nodes, memory (MB), time duration (min), and partition are all configurable. For example, to start an 8-hr interactive session with 13 8 CPUs, 1 node, and 4 GB on the interact partition, type:
interactive 13 8 1 4000 480 interact
You should request at least 2 3 hours for your interactive session when first setting up with MITgcm since the initial input download will take over an hour. If you plan on running MITgcm interactively rather than submit your job to slurm, you should request a multiple of 13 cores.
Once your interactive session starts up, we recommend that you also run the awake.sh script in the background (type 'awake.sh &'). This will periodically print. a blank line to keep your terminal from freezing. Alternatively, place awake.sh into your .my_personal_settings file and configure your .bashrc to source that file. It will then run every time you start an interactive session.
Step 2: Download the "MITgcm_code" repository
Navigate First, navigate to your /n/regal/sunderland_lab/your_user_name folder . You since you must store the MITgcm run _code directory within your regal directory for maximum performance of the model.
Type Next, navigate to whatever subdirectory in your regal directory you want to work in. Finally, type the following commands to download the MITgcm repository to local directory " MITgcm_code ":repository.
git clone https://bitbucket.org/harvard-bgc/mitgcm_code MITgcm_code
Step 3: Set up the initial PCB run directory
Change directories into MITgcm_code which you just cloned. Execute shell script make_initial_pcb_rundir.sh. Note that downloading input files from the Sunderland_lab space to your regal directory will take a while. For this reason, it is only done once and additional run directories that you set up later in the last step below will symbolically link to files in this directory.
Pay attention to the text output to learn what the script is doing. After it is complete instructions are printed out to clone the pkg/ source code repository developed by Helen Amos. Follow these instructions before proceeding.
The MITgcm_code directory will serve as the local storage location for all MITgcm input files, including those used in additional MITgcm directories that you create. For this reason, once you clone it and set it up using the instructions above, do not rename it or move it. Due to the 90-day storage retention on Odyssey, you will need to periodically perform this setup step again in the future to prevent the files from being deleted.
Step 4: Download the "hamos_mitgcm_pkg" repository
...
Execute python script change_PCBpcb_congener.py to set your congener in config files, source code (pcb_param.F), and run scripts. For example, "./change_PCB_congener.py PCB101". If you add any congener-specific source code outside of pcb_param.F, make sure that you incorporate setting the correct settings into change_PCB_congener.py.
...
Once you have one run directory you can very quickly create new ones without having the re-download all of the input files. This saves a lot of time and resources. Go back to step 2 and clone the repository to a new directory with a different name than your first one. Make sure it is stored in the same directory as your original clone "MITgcm_code". Change directories into your new clone, and execute the shell script make_new_pcb_rundir.sh to set it up. Additional directories can have descriptive names of what you plan on doing with them, e.g. MITgcm_PCB28_{update_description}. Follow the instructions prompted by the script to complete the setup. Then continue through steps above to get the source code, change congeners, compile, and run.
Step 10: Clean an existing run directory for re-use
If you would like to re-use a run directory it is a good idea to remove all files leftover from previous runs. You can do this by executing the clean_pcb_rundir.sh script located in the top-level of any of your MITgcm_code cloned repositories after run directory setup. All output and log files will be removed with the exception of files and directories in the results_pcb/ directory.