MITgcm / GEOS-Chem Coupler

Description of coupler

The coupler is a Python interface that passes information between the MITgcm ocean and GEOS-Chem atmosphere, allowing you to capture the temporal dynamics of air-sea exchange. The coupler was initially developed as part of a project to model the global fate and transport of polychlorinated biphenyls (PCBs), a semi-volatile industrial chemical. 

Credit & citation

Colin Pike-Thackray (MIT) wrote the Python and shell scripts making up the essential skeleton of the coupler. Helen M. Amos (Harvard) developed the PCB ocean simulation and wrote the MATLAB scripts handling all of the regridding.

Citation: Amos et al. (2015), Temporal trends in air-sea exchange of polychlorinated biphenyls (PCBs) and implications for the Arctic, in prep.

Disclaimer

As of 22 September 2015, the coupler has only been tested on Svante for the following model versions:

• MITgcm ECCOv4 (lat-lon-cap grid configuration) PCB simulation
• GEOS-Chem v9-01-03 PCB simulation

The coupler has never been tested (1) on Odyssey, (2) for the Hg or PFOS simulations, (3) for other GEOS-Chem versions, (4) for MITgcm ECCO v1. 

Prerequisites for running the coupler

• An account on Odyssey or Svante
• Space on a file/storage server
• A copy of GEOS-Chem that compiles and runs standalone (i.e., not as part of the coupler)
• A copy of the MITgcm that compiles and runs standalone (i.e., not as part of the coupler)
• A GitHub account
• Python
• MATLAB
• gcmfaces MATLAB package up and running (tip: go through the demo)

Step 1: Download the coupler

Put the coupler on a file/storage server. The coupler will run GEOS-Chem and MITgcm together, generating large output files from both models. If you run it from your home directory, you will quickly hit your storage quota. On Svante, I put the coupler on file server #2: /net/fs02/d2/geos_harvard/helen/coupler/. 

Open up a terminal and login to your head node (i.e., login node). For example, this is how I log into Svante

ssh -Y geos_harvard@svante.mit.edu

Go to the location on the file server where you want to put the coupler. For example, this is where I want my coupler on Svante 

cd /net/fs02/d2/geos_harvard/helen/

Clone a copy of the coupler from Colin's GitHub page 

git clone https://github.com/thackray/coupler.git

Step 2: Initial set up and test run

After cloning a copy of the coupler, type this series of commands into the terminal to set up your coupler. You should be logged into the head node.

cd /your_directory/coupler/

ssh -Y <username>@svante2.mit.edu             (ssh to a file server, here's how on Svante)

python setup.py                                                (generates a bunch of subdirectories)

exit                                                                    (exit the file server, go back to head node)

source startup.sh                                              (grabs nodes to run coupled job)

python example.py                                            (run job)

The following message will appear in your terminal when the test run is done:

All Done! Thanks for using coupler.py

As of 22 September 2015, example.py will run a 1-yr coupled PCB simulation with a passing frequency of 21 days on 13 cores. Because it's on 13 cores, it's slow. The default example.py takes 7 hours on Svante. When the job is running, the coupler spits a status update to the terminal window every 10 seconds. When the job is finished, you'll see the message, "Thanks for using the coupler!"

Step 3: Modifying the coupler

Rename a copy of example.py. You don't have to do this, but I recommend it because then you have the original example.py you can refer to if your modifications are unsuccessful. 

cp example.py myrun.py 

To modify the length of your run, open myrun.py and edit "start_time" and/or "end_time". To modify the passing frequency, open myrun.py and edit "step". For example,

start_time, end_time = datetime(2000,1,1,0),datetime(2001,1,1,0)                 (run 1 yearr, from Jan 01, 2000 to Jan 01, 2001)

step = 21* 24                                                                                                    (pause models and pass information every 21 days)

 

start_time, end_time = datetime(2000,1,1,0),datetime(2000,1,5,0)                 (run 4 days, from Jan 01, 2000 to Jan 05, 2000)

step = 1* 24                                                                                                    (pause models and pass information every day)

Before you submit a new coupled job, remove files "DONE" and "SENT" from your run directory:

cp <your_directory>/coupler/GEOS-Chem/run/

rm DONE SENT

cp <your_directory>/coupler/MITgcm/verification/global_pcb_llc90/run/

rm DONE SENT

Return to your coupler parent directory:

cp <your_directory>/coupler/

Type the following into the command line to submit your new job:

source startup.sh                                              (grabs nodes to run coupled job)

python myrun.py                                               (run job)

The following message will appear in your terminal when the job is done:

All Done! Thanks for using coupler.py