This page documents the steps to run WRF and WRF-Chem on OdysseyHarvard cluster.  

We invite Archana Dayalu, Packard Chan, Lee Miller , and Jiahua Guo , and Archana Dayalu to to co-edit this page. 

Glossary

...

tail rsl.error.0000*  # expect to see "SUCCESS COMPLETE WRF"

Running

...

REAL cases

PART I. Setting up/Configuration/Compilation.

• Instructions in this part assume you want to compile and run your own version of WRF. However, note that a compiled usable version of WRF/WRF-Chem v3.6.1 including all external utilities and supplementary geography datasets that you can copy to your preferred run directory is already located at: 

  /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/WRF_CHEM_3-6-1

  This folder (hereafter $CLIMATE_MODELS) contains the WRF-ARW model, the WRF Pre-processing system (WPS; used for real test cases), the chemistry module add-on, the complete WRF geography dataset (for use with WPS and WRF-Chem), and other utilities needed for WRF-Chem. Note: WPS, WRF-Chem not relevant for idealized cases.

  Note 2: With the exception of the geography data set which is really big, copy the WRF_CHEM_3-6-1 folder to a location you are going to run it from. Soft link to the geography data set in the $CLIMATE_MODELS folder.
  .

• If you want to use another version, you will first need to register as a user to download WRF source codes
  WRF: http://www2.mmm.ucar.edu/wrf/users/download/get_source.html
  
  
• Once you've registered, you will be able to navigate to the downloads section. Select the WRF components relevant to you. For example:
  http://www2.mmm.ucar.edu/wrf/src/WPSV3.6.1.TAR.gz #WPS 3.6.1
  http://www2.mmm.ucar.edu/wrf/src/WRFV3.6.1.TAR.gz #WRF 3.6.1
  http://www2.mmm.ucar.edu/wrf/src/WRFV3-Chem-3.6.1.TAR.gz #Chem module add-on for WRF 3.6.1
  
  
• Be sure to read the online user manuals:
  WRF-ARW: http://www2.mmm.ucar.edu/wrf/users/docs/user_guide_V3/contents.html
  WRF-Chem: https://ruc.noaa.gov/wrf/wrf-chem/Users_guide.pdf #This is for a different WRF-Chem version (3.9), but it's still a relevant guide.
  https://ruc.noaa.gov/wrf/wrf-chem/Emission_guide.pdf #This is a separate supplementary WRF-Chem guide to chemical input data processing.
  https://ruc.noaa.gov/wrf/wrf-chem/wrf_tutorial_nepal/talks/Setup.pdf  #Some helpful WRF-Chem slides from NOAA
  
  
• If you're going to be using WRF meteorology output to drive the STILT LPDM (http://stilt-model.org/index.php/Main/HomePage) you will need to make some changes to the Registry file before compiling. Some basic instructions for this are located in $CLIMATE_MODELS folder as a README_WRF-STILT_modifications.txt file. 
  
  
• Once you've downloaded the necessary tar.gz files, confirm your bashrc file looks something like:

  # (1) Load required modules (here we use Intel and Intel MPI) 

     module load intel/17.0.4-fasrc01

     module load impi/2017.2.174-fasrc01

     module load netcdf/4.1.3-fasrc02 

     module load libpng/1.6.25-fasrc01

     module load jasper/1.900.1-fasrc02

     module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 ncview/2.1.7-fasrc01 

     module load ncl_ncarg/6.4.0-fasrc01

  # (2) Define required environment variables

     export NETCDF=${NETCDF_FORTRAN_HOME:-${NETCDF_HOME}}

     export JASPERLIB=${JASPER_LIB}

     export JASPERINC=${JASPER_INCLUDE}

     export WRFIO_NCD_LARGE_FILE_SUPPORT=1

     export HDF5=${HDF5_HOME}

     unset MPI_LIB  #unset for WPS, where WPS is used for real WRF simulations

  ### ...... For WRF-Chem: ...... ###

     export WRF_EM_CORE=1

     export WRF_NMM_CORE=0

     export WRF_CHEM=1

   

• Now, configure and compile. WRF must be compiled before WPS.

...

#(12)  The MOZBC utility for mapping chemical boundary conditions to your WRF-Chem  domain has already been compiled and saved in the $CLIMATE_MODELS WRF-Chem folder, following the instructions in the README_mozbc file. You can use that, or if you wanted, download and compile MOZBC on your own. The initial files are based on MOZART 4-GEOS 5 output (6hr, 1.9x2.5deg, 56-level) datasets (https://www.acom.ucar.edu/wrf-chem/mozart.shtml). Read the README file for compilation instructions if you're doing this on your own; on odyssey the Harvard cluster you might have to do the following: export NETCDF_DIR=$NETCDF_HOME before compilation, and same with MEGAN instructions (#13 below). Otherwise you can use the compiled version located at:

...

Regardless of whether you are running WRF or WRF-Chem, it is important that you do the following first and in this order (detailed instructions follow, including in the examples in Part III and IV):

(1) Run WPS (geogrid.exe, ungrib.exe, metgrid.exe) to create real data-based initialization files with of form met_em.d0X.YYYY-MM-DD_hh:mm:ss.nc

(2) Run real.exe to generate input and boundary files of form wrfinput_d0*, wrfbdy_d01 (and optionally wrffdda_d0*) to initialize WRF model 

...

      3. You should see met_em.d01(or d02 or d03....).YYYY-MM-DD_hh:mm:ss.nc files created. These are your WPS final output files that real.exe ingests.

Step 2. Running WRF

• Navigate to your WRF/WRFV3/run folder

• Step 2b. PROGRAM REAL: real data initialization program to set up the model domain and initialize WRF

...

PART IV. Running WRF-Chem for real cases in Large Eddy Simulation (LES) Mode: A Beijing PM2.5 Case Study

Running

...

IDEALIZED cases

Miscellaneous links

Google Docs: https://docs.google.com/document/d/1Jls4FlWIOIhMlCzMPWm6_aBZqx_Axxe8RMcKjdILDFg/

...