1. General introduction

This wiki introduces the processes to run the Global Terrestrial Mercury Model (GTMM) coupled to the GEOS-Chem Hg simulation. Here I only introduce the content that is immediately related to set up a Hg/POPs simulation on the Harvard machine, as this is a work in progress.

To proceed, you need to first obtain a FAS Research Computing account from here.

See other relevant wiki page for environment set-up scripts.

On odyssey, please always do work like this in an interactive session. To get an interactive session, enter command:

interactive 4 1 4000 240 test

The following might work with any compiler but has only been tested using the GEOS-Chem settings for the Intel comiler version 11, so do the following command:

load_if11

2 Obtain source code

2.1 Obtain GTMM/GEOS-Chem code

For users from the Harvard BGC group, a copy of the source code can be obtained from my home directory:

cp -r /n/home02/thackray/GCgtmm YOUR_SOURCE_DIRECTORY

Github link to follow when public version is complete.

2.2 Obtain GTMM data

Again, you can copy my version:

cp -r /n/home02/thackray/GTMM YOUR_GTMM_DIRECTORY

This also contains a run directory for your pre-GEOS-Chem spin up.

3 Edit and compile GTMM code

3.1 Edit GTMM code

Edit the file YOUR_SOURCE_DIRECTORY/GTMM/defineConstants.F90 to point "filepath" and "outputpath" to the

relevant directories in YOUR_GTMM_DIRECTORY.

Note** you must also change "f_len" and "f_len_output" to match your defined paths (this silliness will be removed in the future).

3.2 Compile GTMM code

For now, GTMM must be compiled before GEOS-Chem to make its library available for the GEOS-Chem code. In the future, this will be built into the GEOS-Chem build sequence.

cd YOUR_SOURCE_DIRECTORY/GTMM

bash compile.sh

cp gtmm YOUR_GTMM_DIRECTORY/output/

3.3 Edit GEOS-Chem code

Until the GEOS-Chem build sequence is updated, edit the file YOUR_SOURCE_DIRECTORY/Makefile_header.mk

change line 896 to read:

GTMM_NEEDED             :=1

3.4 Compile GEOS-Chem code

At this point the best way to proceed is to make a 4x5 degree Hg run directory using the GEOS-Chem unit tester.

(If you'd like an easier option, you can also do:

cp -r /n/home02/thackray/gtmmrun YOUR_RUN_DIRECTORY

)

Once this is done, edit input.geos so that line 104 reads:

Use GTMM soil model?    : T

Now you are ready to compile GEOS-Chem in this run directory:

cd YOUR_RUN_DIRECTORY

make -j4 mpbuild

Note** if you did not use the unit tester to make your run directory, you will have to edit the path that "CODE_DIR" points to in YOUR_RUN_DIRECTORY/Makefile

If compilation completes normally, an executable "geos.mp" will be created.

4. Run the simulation

A quick guide for submitting and managing jobs on Odyssey is available here.

4.1 Run GTMM standalone (optional?)

If you need to do a spin up of GTMM uncoupled from GEOS-Chem, you can do this:

cd YOUR_GTMM_DIRECTORY/output

Edit rungtmm.sh to point to your directory

sbatch rungtmm.sh

When it's done, you can do the following to make GEOS-Chem happy:

cp HgPools HgPools.0

4.2 Run GEOS-Chem coupled to GTMM

An example runscript is available at /n/home02/thackray/gtmmrun/dorun.sh

Edit this run script to use YOUR_RUN_DIRECTORY instead of the one defined by me.

You can submit your job by doing:

sbatch dorun.sh

Sit back and wait for output!