Content Comparison

Table of Contents

this page document the steps to run WRF and WRF-Chem on Odyssey.

...

Instructions in this part assume you want to compile and run your own version of WRF. However, note that a compiled usable version of WRF/WRF-Chem v3.6.1 including all external utilities and supplementary geography datasets that you can copy to your preferred run directory is already located at:
```
/n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/WRF_CHEM_3-6-1/WRF
```
This folder (hereafter $CLIMATE_MODELS) contains the WRF-ARW model, the WRF Pre-processing system (WPS; used for real test cases), the chemistry module add-on, the complete WRF geography dataset (for use with WPS and WRF-Chem), and other utilities needed for WRF-Chem. Note: WPS, WRF-Chem not relevant for idealized cases.
Note 2: With the exception of the geography data set which is really big, copy the WRF_CHEM_3-6-1 folder to a location you are going to run it from. Soft link to the geography data set in the $CLIMATE_MODELS folder.
.
If you want to use another version, you will first need to register as a user to download WRF source codes
WRF: http://www2.mmm.ucar.edu/wrf/users/download/get_source.html
Once you've registered, you will be able to navigate to the downloads section. Select the WRF components relevant to you. For example:
http://www2.mmm.ucar.edu/wrf/src/WPSV3.6.1.TAR.gz #WPS 3.6.1
http://www2.mmm.ucar.edu/wrf/src/WRFV3.6.1.TAR.gz #WRF 3.6.1
http://www2.mmm.ucar.edu/wrf/src/WRFV3-Chem-3.6.1.TAR.gz #Chem module add-on for WRF 3.6.1
Be sure to read the online user manuals:
WRF-ARW: http://www2.mmm.ucar.edu/wrf/users/docs/user_guide_V3/contents.html
WRF-Chem: https://ruc.noaa.gov/wrf/wrf-chem/Users_guide.pdf #This is for a different WRF-Chem version (3.9), but it's still a relevant guide.
https://ruc.noaa.gov/wrf/wrf-chem/Emission_guide.pdf #This is a separate supplementary WRF-Chem guide to chemical input data processing.
https://ruc.noaa.gov/wrf/wrf-chem/wrf_tutorial_nepal/talks/Setup.pdf #Some helpful WRF-Chem slides from NOAA
If you're going to be using WRF meteorology output to drive the STILT LPDM (http://stilt-model.org/index.php/Main/HomePage) you will need to make some changes to the Registry file before compiling. Some basic instructions for this are located in $CLIMATE_MODELS folder as a README_WRF-STILT_modifications.txt file.

Once you've downloaded the necessary tar.gz files, confirm your bashrc file looks something like:

# (1) Load required modules (here we use Intel and Intel MPI)

   module load intel/17.0.4-fasrc01

   module load impi/2017.2.174-fasrc01

   module load netcdf/4.1.3-fasrc02

   module load libpng/1.6.25-fasrc01

   module load jasper/1.900.1-fasrc02

   module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 ncview/2.1.7-fasrc01

   module load ncl_ncarg/6.4.0-fasrc01

# (2) Define required environment variables

   export NETCDF=${NETCDF_FORTRAN_HOME:-${NETCDF_HOME}}

   export JASPERLIB=${JASPER_LIB}

   export JASPERINC=${JASPER_INCLUDE}

   export WRFIO_NCD_LARGE_FILE_SUPPORT=1

   export HDF5=${HDF5_HOME}

   unset MPI_LIB  #unset for WPS, where WPS is used for real WRF simulations

### ...... For WRF-Chem: ...... ###

   export WRF_EM_CORE=1

   export WRF_NMM_CORE=0

   export WRF_CHEM=1

Now, configure and compile. WRF must be compiled before WPS.

...

Regardless of whether you are running WRF or WRF-Chem, it is important that you do the following first and in this order (detailed instructions follow, including in the examples in Part III and IV):

(1) Run WPS (geogrid.exe, ungrib.exe, metgrid.exe) to create real data-based initialization files with of form met_em.d0X.YYYY-MM-DD_hh:mm:ss.nc

(2) Run real.exe to generate input and boundary files of form wrfinput_d0*, wrfbdy_d01 (and optionally wrffdda_d0*) to initialize WRF model

...

3. You should see met_em.d01(or d02 or d03....).YYYY-MM-DD_hh:mm:ss.nc files created. These are your WPS final output files that real.exe ingests.

Step 2. Running WRF

Navigate to your WRF/WRFV3/run folder
Step 2b. PROGRAM REAL: real data initialization program to set up the model domain and initialize WRF

...

Get in the habit of using ncview for sanity-check visuals
Use the dust map from the WPS high-res geographical data set (namelist.input dust_opt=3), although this is less important for this winter exercise.
Use prep-chem-sources and convert_emis to format a gridded anthropogenic PM2.5 emissions file to the WRF forecast domain
Use gridded chemical data for biogenic emissions and chemical boundary conditions
Run WRF-Chem with PM2.5-radiation feedbacks to simulate PM2.5 concentrations in Beijing in January 2013.
Extract relevant components from wrfout netcdf files and examine/visualize using NCL.

...

And you should see something like :below. Once this is done, navigate to your geogrid dump folder that you specified in your namelist.wps file and visualize the three geo_em.<domain>.nc files using ncview. Make sure things look reasonable.

Parsed 22 entries in GEOGRID.TBL
Processing domain 1 of 3
 Processing XLAT and XLONG
 Processing MAPFAC

[...etc for other variables, domains ...]

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
! Successful completion of geogrid. !
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

...

ln -sf ungrib/Variable_Tables/Vtable.GFS_new Vtable

./link_grib.csh /n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130106* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130107* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130108* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130109* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130110* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130111* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130112* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130113* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130114* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130115* \

/n/holylfs/LABS/kuang_lab/adayalu/WRF_GRIB2_NCEP_FNL/fnl_20130116*

Examine a sample GRIB2 files using the g2print.exe tool

...

If the program ran correctly, you should get no messages and you should see three files in your MEGAN folder

...

. Check them out with ncview.

wrfbiochemi_d01

wrfbiochemi_d02

wrfbiochemi_d03

...

Now that you have your bio emissions, it's time to get your anthropogenic emissions in the right format. We're going to use the EDGAR-HTAP PM2.5 emissions on a 0.1x0.1 deg annual global grid.
This option has been successfully tested if you're using EDGAR-HTAP emissions or IPCC emissions (for use with CAM-Chem) and seems a little less complicated than prep-chem-sources. In these examples, we will go this route. If you are starting with other emissions, you probably need to go the prep-chem route. I haven't spent much time figuring out how to use it; the anthro_emis option seemed most straightforward for now because I already had some familiarity with utilities in the same family (MEGAN, MOZBC).
Navigate to your ANTHRO folder

cd src

Link the EDGAR_HTAP_emi_PM2.5_2010.0.1x0.1.nc file from the anthro_data/MOZCART folder to this folder, keeping the filename the same

ln -sf yourpath_toWRF/anthro_data/MOZCART/EDGAR_HTAP_emi_PM2.5_2010.0.1x0.1.nc .

Create a new text file called anthro_emis.inp. This will be your namelist file. Check out the README file for detailed instructions. Your namelist file should look like the following. This is constructed based on a hybrid of following instructions in the README file and the input file located in the anthro_data/EDGAR-HTAP/INP-Examples folder. Make sure you understand the purpose of the entries.

&CONTROL

 anthro_dir = '/n/holylfs/LABS/kuang_lab/adayalu/WRF/ANTHRO/src'

 domains = 3

 wrf_dir    = '/n/holylfs/LABS/kuang_lab/adayalu/WRF/UTILINP'

 src_file_prefix = 'EDGAR_HTAP_emi_'

 src_file_suffix = '_2010.0.1x0.1.nc'

 src_names = 'PM2.5(1)'

 sub_categories  = 'emis_tot'

 serial_output   = .false.

 start_output_time = '2010-12-01_00:00:00'

 emissions_zdim_stag = 1

 emis_map = 'PM_25(a)->PM2.5',

...

Note that the src_file_prefix and src_file suffix are concatenated with whatever is specified in src_names (here, PM2.5) to generate the full filename string.
Now run anthro_emis.

./anthro_emis < anthro_emis.inp > anthro_emis.out

and you should see six new files in the ANTHRO/src directory, one for each of the three domains. Check out the contents with ncview.

wrfchemi_00z_<domain>

wrfchemi_12z_<domain>

Step 4. OPTION 2 (FYI for now): Prep your anthropogenic emissions using prep-chem-sources

TBD. Watch this space.
Use this utility if you're using something other than EDGAR-HTAP or things not listed in the anthro_emis README/instructions. In theory, anthro_emis could work for other emissions fields, but they have just not been tested.
Now that you have your bio emissions, it's time to get your anthropogenic emissions in the right format. Navigate to your PREP-CHEM-SRC-1.5 folder.

cd bin

Edit the prep_chem_sources.inp namelist file. Check out the README file for detailed instructions.

...

prep_chem_sources.inp namelist file. Check out the README file for detailed instructions.

Step 5. Run real.exe again, with chem_opt turned on

Navigate to your WRFV3/test/em_real/ directory
Link your bio and anthro files to your WRFV3/test/em_real/ directory.
Open up the the namelist.input file and turn chem_opt back on (set it to 10).
Make sure kemit=1 (vertical levels in anthro emissions files...in this case it is 1...surface data).
Call up an interactive shell if you don't have one running. Run real.exe again. This incorporates the chem variables into the initial and boundary condition files so that MOZBC can populate them.

mpirun -np 1 ./real.exe > run_real.log

Step 6. Prep the gridded chemical data initial and boundary conditions using MOZBC

This step modifies the wrf initial and boundary condition files that now have space for chemical data (since we re-ran real.exe with chem_opt on and anthro/bio fields)
Go to https://www.acom.ucar.edu/wrf-chem/mozart.shtml and submit a data request. This can take some time to process depending on your domain size request. Submit your bounding box, and times. For this exercise you can use what was already downloaded. This is available in the mozbc directory in the $CLIMATE_MODELS folder as a mozart4geos5-ZZZZ.nc file.
Navigate to the your mozbc folder from the $CLIMATE_MODELS folder.

...

do_bc = .true.
do_ic = .true.
domain = 3
#To set dir_wrf, I recommend calling it something like mozbcin, and link your required files. #FYI, I've found mozbc can be unhappy when the set directory path is too long. e.g.,
dir_wrf = '/n/holylfs/LABS/kuang_lab/adayalu/WRF/mozbcinUTILINP/' #obviously this should be your specific path.
dir_moz = '/n/holylfs/LABS/kuang_lab/adayalu/WRF/MOZBC/' #obviously this should be your specific path.
# fn_moz should look something like 'ha0004.nc'. #you will have to rename your mozart4geos5-ZZZZ.nc file whatever this is.
#In the species map (spc_map) section, delete the entry 'op2 -> C2H5OOH'. This isn't in the mozart4geos5 file, and leads to an error if it remains in there.
#(This knowledge is from trial and error.)

...

Version	Old Version 68	New Version 69
Changes made by	Archana Dayalu (Unlicensed)	Archana Dayalu (Unlicensed)
Saved on	Aug 01, 2018	Aug 02, 2018

Versions Compared

Key

Step 2. Running WRF