Table of Contents

We invite Archana Dayalu, Packard Chan, Lee Miller and Jiahua Guo to co-edit this page.

Glossary

WRF: Weather Research and Forecasting Model
ARW / EM: Advanced Research WRF. As opposed to NMM.
NMM: Nonhydrostatic Mesoscale Model
WPS: WRF Preprocessing System
DA / VAR: Variational Data Assimilation
KPP: Kinetic Pre-Processor (related to chemistry)
chem, ..
real
ideal
grib: Gridded Binary
grid

...

Exit datamover01 node. You can use login node for this default case.
cd /n/holylfs/LABS/`id -gn`/$USER/new-wrf-root/410-WPS/
source source_wrf # 25s on holylogin03
vim namelist.wps # no edits
./geogrid.exe # 8s on holylogin03, v410, Jan00
ln -sf ungrib/Variable_Tables/Vtable.GFS-PRMSL Vtable # for ungrib

...

./ungrib.exe # 2s on holylogin03, v410, Jan00
./metgrid.exe # 1s on holylogin03, v410, Jan00

...

cd /n/holylfs/LABS/`id -gn`/$USER/new-wrf-root/412-WRF/
cp -ai run 00run # 5s on holylogin03
cd 00run/
# make sure you have sourced source_wrf
ln -s ../../410-WPS/met_em.d01.2000-01-2* ./
vim namelist.input # no edits
./real.exe # 3s on holylogin03
tail rsl.error.0000 # expect to see "SUCCESS COMPLETE REAL_EM INIT"
vim job_wrf.sbatch # no required edits
sbatch job_wrf.sbatch # 2m36s with 4 huce_intel cpus
tail rsl.error.0000* # expect to see "SUCCESS COMPLETE WRF"

...

REAL cases

PART I. Setting up/Configuration/Compilation.

Instructions in this part assume you want to compile and run your own version of WRF. However, note that a compiled usable version of WRF/WRF-Chem v3.6.1 including all external utilities and supplementary geography datasets that you can copy to your preferred run directory is already located at:

Choose versions of WRF & WPS:

Latest releases on official GitHub: Image Added Image Added

WRF v4.1.2 (precompiled without chem) is downloaded in /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/wrf/412-WRF
WPS v4.1 (precompiled) is downloaded in /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/wrf/410-WPS

WRF v3.6.1 (precompiled with chem) is downloaded in /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/WRF_CHEM_3-6-1

...

This folder (hereafter $CLIMATE_MODELS) contains the WRF-ARW model, the WRF Pre-processing system (WPS; used for real test cases), the chemistry module add-on, the complete WRF geography dataset (for use with WPS and WRF-Chem), and other utilities needed for WRF-Chem. Note: WPS, WRF-Chem not relevant for idealized cases.

Note 2: With the exception of the geography data set which is really big, copy the WRF_CHEM_3-6-1 folder to a location you are going to run it from. Soft link to the geography data set in the $CLIMATE_MODELS folder.
.

...

/WRFV3
WPS v3.6.1 (precompiled) is downloaded in /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/WRF_CHEM_3-6-1/WPS

WRF v3.0beta with RAVE (pls recompile) is in ~kuang/Model/WRFV3
WPS v3.2.1 (pls recompile) is in ~dingma/Model2/WPSv3

Read the user's guide:
WRF (ARW) User's Guides: v3, v4
WRF-Chem: https://ruc.noaa.gov/wrf/wrf-chem/Users_guide.pdf #This is for a different WRF-Chem version (3.9), but it's still a relevant guide.
https://ruc.noaa.gov/wrf/wrf-chem/Emission_guide.pdf #This is a separate supplementary WRF-Chem guide to chemical input data processing.
https://ruc.noaa.gov/wrf/wrf-chem/wrf_tutorial_nepal/talks/Setup.pdf #Some helpful WRF-Chem slides from NOAA

PART I. Setting up/Configuration/Compilation.

Instructions in this part assume you want to compile and run your own version of WRF. However, note that a compiled usable version of WRF/WRF-Chem v3.6.1 including all external utilities and supplementary geography datasets that you can copy to your preferred run directory is already located at:
```
/n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/WRF_CHEM_3-6-1
```
This folder (hereafter $CLIMATE_MODELS) contains the WRF-ARW model, the WRF Pre-processing system (WPS; used for real test cases), the chemistry module add-on, the complete WRF geography dataset (for use with WPS and WRF-Chem), and other utilities needed for WRF-Chem. Note: WPS, WRF-Chem not relevant for idealized cases.
Note 2: With the exception of the geography data set which is really big, copy the WRF_CHEM_3-6-1 folder to a location you are going to run it from. Soft link to the geography data set in the $CLIMATE_MODELS folder.
.
If you want to use another version, you will first need to register as a user to download WRF source codes
WRF: http://www2.mmm.ucar.edu/wrf/users/docsdownload/user_guide_V3/contentsget_source.html
WRF-Chem: https://ruc.noaa.gov/wrf/wrf-chem/Users_guide.pdf #This is for a different WRF-Chem version (3.9), but it's still a relevant guide.
https://ruc.noaa.gov/wrf/wrf-chem/Emission_guide.pdf #This is a separate supplementary WRF-Chem guide to chemical input data processing.
https://ruc.noaa.gov/wrf/wrf-chem/wrf_tutorial_nepal/talks/Setup.pdf #Some helpful WRF-Chem slides from NOAA
Once you've registered, you will be able to navigate to the downloads section. Select the WRF components relevant to you. For example:
http://www2.mmm.ucar.edu/wrf/src/WPSV3.6.1.TAR.gz #WPS 3.6.1
http://www2.mmm.ucar.edu/wrf/src/WRFV3.6.1.TAR.gz #WRF 3.6.1
http://www2.mmm.ucar.edu/wrf/src/WRFV3-Chem-3.6.1.TAR.gz #Chem module add-on for WRF 3.6.1
If you're going to be using WRF meteorology output to drive the STILT LPDM (http://stilt-model.org/index.php/Main/HomePage) you will need to make some changes to the Registry file before compiling. Some basic instructions for this are located in $CLIMATE_MODELS folder as a README_WRF-STILT_modifications.txt file.

Once you've downloaded the necessary tar.gz files, confirm your bashrc file looks something like:

# (1) Load required modules (here we use Intel and Intel MPI)

   module load intel/17.0.4-fasrc01
   module load impi/2017.2.174-fasrc01
   module load netcdf/4.1.3-fasrc02 
   module load libpng/1.6.25-fasrc01
   module load jasper/1.900.1-fasrc02
   module load intel/17.0.4-fasrc01 impi/2017.2.174-fasrc01 ncview/2.1.7-fasrc01 
   module load ncl_ncarg/6.4.0-fasrc01

# (2) Define required environment variables

   export NETCDF=${NETCDF_FORTRAN_HOME:-${NETCDF_HOME}}
   export JASPERLIB=${JASPER_LIB}
   export JASPERINC=${JASPER_INCLUDE}
   export WRFIO_NCD_LARGE_FILE_SUPPORT=1
   export HDF5=${HDF5_HOME}
   unset MPI_LIB  #unset for WPS, where WPS is used for real WRF simulations

### ...... For WRF-Chem: ...... ###

   export WRF_EM_CORE=1
   export WRF_NMM_CORE=0
   export WRF_CHEM=1

Now, configure and compile. WRF must be compiled before WPS.

...

./compile &> compile.output

...

#(1)-(8)

...

are adapted

...

from

...

p.6 of Plamen's

...

advice

...

for

...

v3.9.1

...

(6/8/2018)

Now that the main WRF and WPS programs are compiled, it's time to think about utilities specific WRF-Chem. If you are planning on running WRF-Chem, you will additionally need to compile some utilities to make sure your chemical data is in the right format. Specifically, you will likely need to compile convert_emiss.exe; prep-chem-src, and mozbc. These are just examples; you may find that you may need additional ones.

...

#(10) Compile PREP-CHEM-SOURCES (available HERE), a mapping utility for converting raw anthropogenic chemical fields to binary intermediates that are then fed into convert_emiss.exe. Unzip the tar.gz file into your main WRF folder. There are some typos, and missing details in the pdf guide above, so a modified version of the instructions (and Paul Edmon's help rebuilding HDF5 to fix an error message) enabled successful compilation of the utility. The modified instructions are located here:

...

Regardless of whether you are running WRF or WRF-Chem, it is important that you do the following first and in this order (detailed instructions follow, including in the examples in Part III and IV):

(1) Run WPS (geogrid.exe, ungrib.exe, metgrid.exe) to create real data-based initialization files with of form met_em.d0X.YYYY-MM-DD_hh:mm:ss.nc

(2) Run real.exe to generate input and boundary files of form wrfinput_d0*, wrfbdy_d01 (and optionally wrffdda_d0*) to initialize WRF model

...

3. You should see met_em.d01(or d02 or d03....).YYYY-MM-DD_hh:mm:ss.nc files created. These are your WPS final output files that real.exe ingests.

Step 2. Running WRF

Navigate to your WRF/WRFV3/run folder
Step 2b. PROGRAM REAL: real data initialization program to set up the model domain and initialize WRF

...

wrfout_d0#_YYYY-MM-DD_hh:mm:ss

...

Detailed examples

WRF-Chem: A PM2.5 example

The purpose of this example is to take the general steps listed above and actually run a three nested domain WRF-Chem PM2.5 simulation over Beijing during the January 2013 severe haze event and compare with observations. We are going to run WRF-Chem for a total of 10 days from Jan 6 2013 00:00UTC to Jan 16 2013 00:00UTC. We establish 5 days for model spin-up such that the usable simulation time period is 5 days. Make sure you have a local copy of the /n/holylfs/INTERNAL_REPOS/CLIMATE_MODELS/WRF_CHEM_3-6-1/WRF directory and contents. You don't need to copy the geography data set.

...

Now get ready to run ungrib. Since we will be using NCEP fnl data (GRIB2 format) for this run, we do some sleuthing to confirm the correct Vtable to use. Following the discussion here we download the appropriate Vtable separately (this one isn't included with the standard set of Vtables in WRFv3.6.1 because they recently updated the dataset). Note: I have already downloaded it and saved it: ungrib/Variable_Tables/Vtable.GFS_new

...

You will need access to the relevant MEGAN initial files, from https://www.acom.ucar.edu/wrf-chem/download.shtml
Make sure you are still using an interactive shell ('srun -n 1 --mem=10000 --pty --x11=first -p test -t 200 bash' should be sufficient). Navigate to the MEGAN directory in the $CLIMATE_MODELS directory (or wherever you have saved your local copy of everything)
Create a new text file called megan_bio_emiss.inp. This is your MEGAN namelist file. Note that as the README instructs, the leaf area index (lai) months requires the simulation month and the previous month such that for January (as our example here is) we have to simulate all months. Following Following the instructions in the README, you should populate to look like follows ... with the paths obviously reflecting where your WRF + external utility directories are located.

&control
domains = 3,
start_lai_monthmnth = 1,
end_lai_monthmnth = 12,
wrf_dir = '/n/holylfs/LABS/kuang_lab/adayalu/WRF/UTILINP',
megan_dir = '/n/holylfs/LABS/kuang_lab/adayalu/WRF/MEGAN'
/

You should be ready to run the MEGAN bio emission utility

...

Create a new text file called anthro_emis.inp. This will be your namelist file. Check out the README file for detailed instructions. Your namelist file should look like the following. This is constructed based on a hybrid of following instructions in the README file and the input file located in the anthro_data/EDGAR-HTAP/INP-Examples folder. Make sure you understand the purpose of the entries.

&CONTROL
anthro_dir

=

'/n/holylfs/LABS/kuang_lab/adayalu/WRF/ANTHRO/src'
domains
domains =

3
wrf_dir

=

'/n/holylfs/LABS/kuang_lab/adayalu/WRF/UTILINP'
src_file_prefix

=

'EDGAR_HTAP_emi_'
src_file_suffix

=

'_2010.0.1x0.1.nc'
src_names

=

'PM2.5(1)'
sub_categories

=

'emis_tot'
serial_output

=

.false.
start_output_time

=

'2010-12-01_00:00:00'
emissions_zdim_stag

=

1
emis_map

=

'PM_25(a)
->PM2
→PM2.5',
/

Note that the src_file_prefix and src_file suffix are concatenated with whatever is specified in src_names (here, PM2.5) to generate the full filename string.
Now run anthro_emis.

...

In any case, this exercise should at least get you familiarity with the process of running WRF-Chem and set you up for being able to do second-order troubleshooting (like the more important question of why these values are unrealistic!).

...

Running WRF-Chem for real cases in Large Eddy Simulation (LES) Mode: A Beijing PM2.5 Case Study

...

IDEALIZED cases

WRF (ARW) User's Guides: v3, v4

Miscellaneous links

WRF (ARW) User's Guides: v3, v4

Google Docs: https://docs.google.com/document/d/1Jls4FlWIOIhMlCzMPWm6_aBZqx_Axxe8RMcKjdILDFg/

...

Version	Old Version 92	New Version Current
Changes made by	Packard Chan (Unlicensed)	Packard Chan (Unlicensed)
Saved on	Nov 12, 2019	Jun 19, 2020

Versions Compared

Key

Glossary

REAL cases

PART I. Setting up/Configuration/Compilation.

PART I. Setting up/Configuration/Compilation.

Step 2. Running WRF

Detailed examples

WRF-Chem: A PM2.5 example

Running WRF-Chem for real cases in Large Eddy Simulation (LES) Mode: A Beijing PM2.5 Case Study

IDEALIZED cases

Miscellaneous links

Content Comparison

Versions Compared

Key

Glossary

REAL cases

PART I. Setting up/Configuration/Compilation.

PART I. Setting up/Configuration/Compilation.

Step 2. Running WRF

Detailed examples

WRF-Chem: A PM2.5 example

Running WRF-Chem for real cases in Large Eddy Simulation (LES) Mode: A Beijing PM2.5 Case Study

IDEALIZED cases

Miscellaneous links