Last update: 5 Sep 2019 *** PLEASE READ THIS README BEFORE RUNNING SMOKE. *** == 1. Introduction == These packages contain scripts, inventories, and ancillary files related to EPA's 2016 beta platform for air quality modeling. This package includes updates to the beta platform for use in modeling in support of Regional Haze applications. The beta platform was referred to as trhe "2016ff_16j" case, while this case is the "2016fg_16j" case. The basis of this platform is the 2014 National Emissions Inventory, Version 2 (2014NEIv2). This package is an addendum to the previously published beta platform package located here: ftp://newftp.epa.gov/Air/emismod/2016/beta/ Because this addendum package includes only files updated for 2016fg, you *must* install that package in its entirety first, and then install this package on top of that. This package consists of two zip files: - 2016fg_inventory_ancillary_addendum_to_2016ff.zip includes emissions inventories and other SMOKE inputs. - 2016fg_scripts_ancillary_addendum_to_2016ff.zip includes SMOKE scripts for sectors that have been updated for 2016fg. This package also includes a series of new scripts in support of creating emissions for CAMx 2-way nested modeling in CAMx version 7. The following sectors are updated for the 2016fg Regional Haze case: ptfire, othafdust, othptdust, ptegu, ptnonipm, and othpt. Scripts are also provided for ptfire_othna because the methodology for creating CAMx-ready emissions for this sector has changed. CMAQ model-ready emissions generated using these packages with SMOKE v4.6 should be identical to those used in EPA's 2016 platform, with some exceptions as noted in this README. There may be additional emissions differences resulting from differences in the Linux operating system, hardware platform, and other system-specific differences. == 2. Requirements for processing emissions for air quality modeling == If you are only reviewing inventories and not developing emissions for air quality modeling, you do not need to install SMOKE or to follow the instructions below. Instead, unzip the files with the data of interest and examine those and the corresponding reports that are provided. If you plan to develop emissions for air quality modeling, SMOKE v4.6 is REQUIRED to process this case. SMOKE v4.6 includes additional updates and bug fixes as compared to version 4.5, and so SMOKE v4.6 should be used when processing these emissions cases. The smoke_2016beta_platform_core.zip package includes precompiled executables of SMOKE v4.6 programs. These executables reflect SMOKE v4.6 as of 3 Oct 2018 and is equivalent to the official public release of SMOKE v4.6 as of that date. The same version of SMOKE can be used for both 2016ff_16j and 2016fg_16j, so the SMOKE executables provided with the beta platform package are sufficient. Python: We also recommend (if not require) python version 3.0 or later, along with select python libraries. Many of the helper scripts included in this package use python. The python scripts within this package reference '#!/usr/bin/env python'; you may need to change this on your computing platform. All of the scripts have been updated for Python 3.5 to match the configuration on EPA's cluster. You should still be able to use Python 2.7 if you install the "future" modules. In most cases that can be done by running: sudo pip install future == 3. Installation of data files and scripts == This readme covers the installation of the SMOKE inventories, scripts, and ancillary files used for the 2016 beta platform with updates for Regional Haze modeling. The addendum packages should be installed to the same directory as the original beta platform packages. Choose an install directory on your system; we will refer to this directory as "INSTALL_DIR". To review/reproduce emissions for all sectors, unzip all the .zip files into INSTALL_DIR. The packages have subdirectories embedded within them, so it is important that all files be unpacked in the same place in order for the scripts know where to find the inputs. If you are only interested in reproducing or examining emissions for specific sectors, you may download and unzip only the data for those sectors from the inventory directories, but for SMOKE processing you should also include the files in the ancillary_inputs and spatial_surrogates directories, and also the contents of smoke_2016beta_platform_core.zip. Precompiled SMOKE executables and I/O API utilities are available in the SMOKE zip. A separate SMOKE "utilities" zip includes additional scripts and files that are not necessary to run SMOKE, but are for developing SMOKE inputs or processing SMOKE outputs. All SMOKE inventories, scripts, and ancillary files used for the beta platform are provided, including emission factor tables for onroad processing via SMOKE-MOVES. Prior to running SMOKE, you will need to edit the INSTALL_DIR (your install directory) and MET_ROOT (location of MCIP meteorology data) environment variable definitions in the "directory_definitions.csh" script located in each CASE/scripts directory. This script is sourced by each of the individual run scripts for each sector. Regarding the MCIP data, SMOKE only uses the GRIDCRO2D, METCRO2D, and METCRO3D files. If you are computing plume rise within SMOKE using the Laypoint program, SMOKE also needs the METDOT3D file. MCIP meteorology data is not included in the package. This is used for the onroad, onroad_ca_adj, dust (afdust/othafdust/othptdust), and biogenics (beis) sectors. == 4. Case description and instructions for each sector == For most sectors, scripts are provided to process emissions for a 12km national grid (12US1) using CB6 speciation for CMAQ version 5.2. The CB6 for CMAQ mechanism is slightly different than the CB6 mechanism for the CAMx model from the 2011 emissions platform: it includes a new tracer species (SOAALK), naphthalene is now explicit in the mechanism (NAPH), and the XYL species is replaced by XYLMN (XYL minus naphthalene). These emissions can be converted to CB6-for-CAMx speciation when converting to CAMx format. The scripts can be adapted to run for other grids. First, in the directory_definitions.csh script, edit REGION_ABBREV and REGION_IOAPI_GRIDNAME. You also may need to change other inputs in each individual sector script, such as spatial surrogates (SRGPRO / SRGDESC), transportable fractions (XPORTFRAC), gridded meteorology for onroad (METMOVES), ocean chlorine (for the sector merge), and biogenic land use and seasons (BELD4 and BIOSEASON). If you are changing the grid to 12US2, you do not need to change the spatial surrogates, since 12US2 is a subset of 12US1. You do need to window the other grid-specific input files, however. CAMx two-way nested modeling requires that you run most sectors at both 36km resolution (36US3 grid) and 12km resolution (12US2 grid; we typically run at 12US1 and window the results to 12US2 for CAMx applications). Scripts are provided for 12US1 only for most sectors and must be adapted to also be run for 36US3. Separate directory_definitions.csh scripts have been provided for both 36US3 and 12US2 to make this easier. The default directory_definitions.csh script is for 12US1. This package includes ptfire_othna3D scripts for both 12US2 and 36US3, and othpt and ptnonipm scripts for 36US3 only. The onroad, onroad_ca_adj, and afdust_adj sectors are not processed directly through SMOKE for 36US3. Instead, the 12US1 emissions for those sectors are aggregated to 36US3 resolution. This is done because these sectors depend on gridded meteorology, and so emissions are different for different grids; converting 12US1 to 36US3 eliminates any discrepancy in emissions totals between resolutions for these sectors. A tool is provided in the 2016fg package for regridding 12km resolutions to 36km resolution called the "regrid" tool. To run the regrid tool, copy one of the other run scripts for the sector, and change the bottom (execution) line to: $INSTALL_DIR/smoke4.6/scripts/emf/regrid_emf.csh 12US1 36US3 -m "$RUN_MONTHS" 0 This package includes tools for converting the emissions to the format needed for CAMx modeling, as described in see Section 8 of this README. Emissions processing is split into "sectors". Each sector has its own run scripts for processing, with one (or more) run scripts per case. See Section 7 of this README for information about the run script zips. All sectors are US-only unless otherwise noted. The sectors are: AFDUST: Particulate emissions from fugitive dust sources. This sector is processed in two steps. The first (Annual_afdust_12US1_*) processes the annual inventory, and the second (Annual_afdust_adj_12US1*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "afdust_adj". The afdust scripts must be run in that order. This sector is unchanged from 2016ff beta platform and not included in this package. AG: Agricultural emissions. This is mostly ammonia, but unlike in prior modeling platforms, this sector now includes other pollutants from agricultural sources as well. This sector is unchanged from 2016ff beta platform and not included in this package. BEIS: Biogenic emissions generated using the BEIS model, version 3.6.1. This sector is unchanged from 2016ff beta platform and not included in this package. CMV_C1C2: Emissions from C1 and C2 commercial marine sources, including ports and navigable waterways. Includes offshore C1/C2 marine emissions in the Atlantic and Pacific Oceans, and the Gulf of Mexico. This sector is unchanged from 2016ff beta platform and not included in this package. CMV_C3: Emissions from C3 commercial marine sources, including ports and navigable waterways. Includes offshore C3 marine emissions in the Atlantic and Pacific Oceans and the Gulf of Mexico. Canadian C3 emissions are in the othar sector. The 2016ff emissions from this sector are derived from 2014NEIv2 Nonpoint, but we process as a point sector in order to support plume rise in CMAQ. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). This sector is unchanged from 2016ff beta platform and not included in this package. NONPT: Area source emissions not included in other sectors. This sector is unchanged from 2016ff beta platform and not included in this package. NONROAD: Off highway mobile source emissions. This sector is unchanged from 2016ff beta platform and not included in this package. NP_OILGAS: Area source oil and gas emissions. This sector is unchanged from 2016ff beta platform and not included in this package. ONROAD: On highway mobile source emissions, excluding California. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B. This sector is unchanged from 2016ff beta platform and not included in this package. ONROAD_CA_ADJ: On highway mobile source emissions, California only. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B. This sector is unchanged from 2016ff beta platform and not included in this package. OTHAFDUST: Particulate emissions from fugitive dust sources in Canada. Just like with afdust, this sector is processed in two steps. The first (Annual_othafdust_12US2_*) processes the annual inventory, and the second (Annual_othafdust_adj_12US2*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "othafdust_adj". The othafdust scripts must be run in that order. Fugitive dust emissions in Mexico are included in the othar sector and do not need the same transportable fraction and meteorological adjustments that the Canada fugitive dust emissions in othafdust do. This sector is changed from 2016ff beta platform and included in this package. OTHPTDUST: Point source particulate emissions from fugitive dust sources in Canada. In the new 2015 Canadian inventory, dust emissions are in area source format for some sources (othafdust sector) and point source format for other sources (othptdust sector). This is a new sector starting with beta platform. This is a 'point' sector with additional adjustments, and is processed via THREE scripts: the 'onetime' script, the 'daily' script, and then the adjust script (othptdust_adj), in that order. All emissions in this sector are low-level only (no inline files). This sector is changed from 2016ff beta platform and included in this package. OTHAR: Area source emissions from Canada and Mexico, including mobile nonroad. This sector is unchanged from 2016ff beta platform and not included in this package. ONROAD_CAN: Mobile onroad source emissions from Canada. This sector is unchanged from 2016ff beta platform and not included in this package. ONROAD_MEX: Mobile onroad source emissions from Mexico. The onroad Mexico emissions inventory includes pre-speciated VOC emissions for the CB6-for-CAMx mechanism, so there is an extra script for this sector to convert those emissions to the CB6 mechanism needed for CMAQ. This extra script is called *_part2_combine.csh and uses the combine utility to perform the CB6-for-CMAQ conversion. The combine program is included, pre-compiled, in the SMOKE package along with pre-compiled SMOKE executables and I/O API utilities. To help make the distinction between CB6-for-CAMx and CB6-for-CMAQ emissions, CB6-for-CAMx emissions use the sector name "onroad_mex_cb6orig". The CB6-for-CMAQ post-processing step creates emissions files with the final sector name "onroad_mex". This sector is unchanged from 2016ff beta platform and not included in this package. OTHPT: Point source emissions from Canada and Mexico. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). This sector is changed from 2016ff beta platform and included in this package. PTAGFIRE: Point source agricultural burning emissions. The ptagfire sector uses a daily point source inventory. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). This sector is unchanged from 2016ff beta platform and not included in this package. PTEGU: Electric generating unit emissions. This sector incorporates CEM (Continuous Emissions Monitoring) hourly emissions for a majority of sources. This is a 'point' sector, and like all 'point' sectors, is processed via a 'onetime' script first, followed by a 'daily' script. For ptegu there are two 'daily' scripts for different months of the year: 'summer' (May through September), and 'winter' (October through April). For sources without hourly CEM emissions, summer and winter use different hourly temporalization, and so they are run with separate inputs. All emissions in this sector are elevated (no low-level contribution). This sector is changed from 2016ff beta platform and included in this package. PTNONIPM: Point source emissions from industrial activities. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. Some emissions in this sector are elevated while others are low-level, so both 2-D emissions files (emis_mole) and inline emissions files (inln_mole) are generated. This sector is changed from 2016ff beta platform and included in this package. PTFIRE: Point source emissions from year specific controlled burning and wild fires. Fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. This sector is changed from 2016ff beta platform and included in this package. PTFIRE_OTHNA: Point source emissions from year specific controlled burning and wild fires in the rest of North America ('OTHNA' = OTHer North America), including Canada and Mexico. In addition to Canada and Mexico, fire emissions for Central America and the Caribbean are also included. Emissions from those areas are ultimately not modeled due to being outside of the 12US1 modeling domain, but they are provided for possible use in larger grids. These fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. This sector is unchanged from 2016ff beta platform, but scripts are included in this package because the methodology has changed for CAMx 2-way nested modeling. PT_OILGAS: Point source oil and gas emissions, including emissions from offshore oil rigs in the Gulf of Mexico. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. Some emissions in this sector are elevated while others are low-level, so both 2-D emissions files (emis_mole) and inline emissions files (inln_mole) are generated. This sector is unchanged from 2016ff beta platform and not included in this package. RAIL: Area source railway emissions. This sector is unchanged from 2016ff beta platform and not included in this package. RWC: Area source residential wood combustion emissions. This sector is unchanged from 2016ff beta platform and not included in this package. == 4B. Notes regarding onroad == Since the onroad sector is unchanged from beta platform, this addendum package includes no onroad scripts or inputs. This portion of the README applies to the beta platform package. Onroad emissions are processed using SMOKE-MOVES. The processing is split into multiple run scripts. In the beta platform, SMOKE-MOVES inputs were prepared using MOVES2014a. As in recent platforms, speciation of VOC emissions is handled within the MOVES model. At the time MOVES2014a was run for this emissions case, MOVES included the CB6 for CAMx mechanism, but not the CB6 for CMAQ mechanism needed for CMAQ 5.2. So, SMOKE-MOVES is run for the CB6-for-CAMx mechanism, and then converted to CB6-for-CMAQ in an additional post-processing step. To help make the distinction between CB6-for-CAMx and CB6-for-CMAQ emissions, CB6-for-CAMx emissions from SMOKE-MOVES use the sector names "onroad_cb6orig" and "onroad_ca_adj_cb6orig" instead of "onroad" and "onroad_ca_adj". The CB6-for-CMAQ post-processing step creates emissions files with the final sector names "onroad" and "onroad_ca_adj". As described in the SMOKE online documentation, SMOKE-MOVES handles onroad emissions separately for four types of processes: - On-network emissions (RatePerDistance, or RPD) - Off-network emissions, fuel vapor venting (RatePerProfile, or RPP) - Off-network emissions, extended idling (RatePerHour, or RPH) - Off-network emissions, non-venting, non-extended idle (RatePerVehicle, or RPV) For each of the two onroad sectors (onroad, onroad_ca_adj), there are separate run scripts for RPD, RPP, RPH, and RPV, plus a merge script that combines emissions from RPD, RPP, RPH, and RPV into a single emissions file per day, and a "part2" merge script that converts the emissions from CB6-for-CAMx to CB6-for-CMAQ. The CB6-for-CMAQ conversion is performed using the combine utility which is included, pre-compiled, in the SMOKE package along with pre-compiled SMOKE executables and I/O API utilities. These scripts may take a particularly long time to run, especially RPD. Therefore, consideration should be given to running multiple RPD jobs in parallel, such as one job per quarter. The reason onroad has been split into two sectors - onroad and onroad_ca_adj - is in order to match SMOKE-MOVES annual emission totals with those provided by California. To do this, we split California into a separate sector, and run SMOKE-MOVES with a control factor file (CFPRO) which nudges the emissions so that the annual totals post-SMOKE-MOVES equal those provided by CARB. CARB provided separate onroad emissions inventories for use in the beta platform, and we match their provided emissions at the county/SCC level, except that the CARB inventory does not distinguish between different on-network road types (but does distinguish on-network emissions from off-network). For the onroad sector, a CFPRO is provided in the ancillary_inputs/ge_dat_*_other.zip package. This CFPRO zeroes out refueling emissions in 52 Colorado counties. We zero out these emissions in order to prevent a double count with the ptnonipm sector, since that sector includes refueling emissions in these counties. This may be changed for 2016 v1 platform. The portion of this CFPRO pertaining to Diesel PM is only relevant for multipollutant modeling (e.g. NATA). To help identify which species and pollutants to expect from each of the onroad components (RPD, RPH, RPP, RPV), sample Movesmrg reports for one day are provided in: reports/onroad_sample_movesmrg_reports.zip The sample reports are for California, but for the purposes of checking the existence of species, these reports can be used nationwide. == 4B1. DAYS_PER_RUN == SMOKE-MOVES can be run more efficiently if running more than one day at a time. For example, Movesmrg can create one 7-day emissions file more quickly than it can create seven individual 1-day emissions files. To turn on this feature, use the DAYS_PER_RUN variable, set to the number of days you wish to run in a single Movesmrg instance. The recommended value for DAYS_PER_RUN is 7. The onroad scripts include a setting called "DAYS_PER_RUN", set to 1 as the default. If DAYS_PER_RUN > 1, after Movesmrg is run, the run scripts will use the I/O API utility m3xtract to split up the multi-day emissions file into single day (25-hour) emissions files. Multi-day Movesmrg runs will never cross months. For example, if DAYS_PER_RUN = 7, then the last Movesmrg run of January will start on January 29th and end on January 31st (3 days), and the first Movesmrg run of February will start on February 1st and end on February 7th. Using the multi-day Movesmrg functionality requires multi-day MCIP files. For example, if DAYS_PER_RUN = 7, your METCRO2D files must also be 7 days (169 hours) long. These multi-day MCIP files should be stored in MET_ROOT_${Xday}/, where X = DAYS_PER_RUN (i.e. /7day for DAYS_PER_RUN = 7). For example, if the single day MCIP files are in /foo/foo/mcip_dir/, then 7-day MCIP files should be in /foo/foo/mcip_dir_7day/. The primary drawback to using this multi-day Movesmrg functionality is an increase in the memory usage. == 4C. Sector merge == After all sectors have been processed, the Sector_merge script merges the low-level emissions from all sectors into a single CMAQ-ready emissions file per day. Merged model-ready emissions will be output to: INSTALL_DIR/$CASE/smoke_out/$CASE/$GRID/$SPC/ Inline emissions and stack_groups files will be output to the same directory, except in subdirectories by sector name (e.g. .../$SPC/ptnonipm/). By default, the sector merge scripts are configured to include biogenics. Some CMAQ modelers may wish to process biogenic emissions inline within CMAQ and not include biogenic emissions in the gridded emissions files. To run the sector merge without biogenics, edit the sectorlist file in the $CASE/scripts directory, and set the mergesector column to N for the 'beis' sector. To merge in alternative biogenic emissions files, edit the sectorlist by changing the 'beis' sector name to the sector name of your choice, and make sure your biogenic emissions files exist in the $CASE/premerged/[sector name] directory with filenames adhering to the file name convention used by other sectors. == 5. Description of inventory packages == All inventories updated for the 2016fg case are included in: 2016fg_inventory_ancillary_addendum_to_2016ff.zip This includes inventories for the following sectors: othafdust, othpt, othptdust, ptfire, ptnonipm, ptegu. The ptegu sector also uses the hourly CEMs files from the beta platform package (2016ff_inventory_cem_17dec2018.zip). Even though the scripts package includes scripts for ptfire_othna, the inventories are unchanged from 2016ff beta platform and must be installed from the beta platform package (2016ff_inventory_fires_02jan2019.zip). See Section 4 of this README for a description of each modeling sector. == 6. Description of ancillary file packages == All ancillary files updated for the 2016fg case are included in: 2016fg_inventory_ancillary_addendum_to_2016ff.zip Only the temporal cross-reference files for ptegu sector are updated for 2016fg. All other ancillary files should be installed from the 2016ff beta platform package. == 7. Description of script packages == The 2016fg_scripts_addendum_to_2016ff.zip package should be unpacked in INSTALL_DIR. This includes updated scripts and other utilities needed to process the 2016fg case. The scripts in the 2016fg_16j subdirectory are the scripts you run directly in order to replicate our emissions. Separate script(s) are provided for each sector. See section 4 for information pertinent to each sector. In general, you edit the directory_definitions.csh file, in particular INSTALL_DIR and MET_ROOT, and then run each sector. Sector scripts are organized into subdirectories within CASE/scripts by sector category: biogenics, nonpoint, onroad, point, merge, and camx. This addendum case does not include any biogenics or onroad scripts. For afdust sectors, run afdust/othafdust first, then afdust_adj/othafdust_adj. For point sectors, run "onetime" first, and then "daily". Othptdust has an additional "adj" script which comes after the daily script. For onroad sectors, run RPV/RPD/RPH/RPP first, then the merge script, then the "part2_combine" script. The onroad_mex sector also has a separate part2_combine script. For ptfire_othna3D, there is a third script for 36US3 only, necessary for 2-way nested CAMx modeling, in which the region of the 36US3 domain which overlaps the 12US2 domain is zeroed out, or "masked". This is described in further detail in the CAMx conversion section of this README. The scripts, programs, and other inputs in the camx and smoke4.6 subdirectories are all "helper" scripts and inputs, and generally never need to be run directly. This package includes a full set of CAMx helper scripts and a new tool to mask the 12US2 portion of 36US3 gridded emissions. == 8. Preparing emissions for CAMx == Preparing emissions for the CAMx model instead of the CMAQ model requires converting the emissions to CAMx-ready format after SMOKE is run. Conversion scripts are included in the scripts addendum zip, and output to the INSTALL_DIR/CASE/scripts/camx/ directory. Those scripts use helper scripts and programs which are also included in the core SMOKE zip and output to INSTALL_DIR/camx/. This package includes a full set of scripts needed to convert CMAQ-ready emissions to CAMx format for 2-way nested CAMx modeling. Preparation of emissions for 2-way nested modeling differs from that of regular CAMx modeling as follows: - 2-D gridded emissions files must be generated for both 36US3 and 12US2. - A "buffer cell" must be added to the edges of the 12US2 domain. - The mrgpt file only needs to include point source emissions processed for 36US3 or 12US1 or 12US2. If a sector has no emissions that are outside 12US1 but are inside 36US3, then a 12US1 file can be used; otherwise a 36US3 file must be used. In this case, ptegu is processed for 12US1 while other sectors are processed for 36US3. - Conversion of fire emissions to CAMx format is different from other point sectors because CAMx does not support inline plume rise for fires. Instead, fires must be layered by SMOKE (3-D processing with laypoint) and converted to CAMx ptsr format. Since these are gridded emissions, the final mrgpt file must include fire emissions at 12km resolution within the 12US2 domain, plus fire emissions at 36km resolution in the region outside 12US2 and inside 36US3. For this case, this means running ptagfire3D and ptfire3D through SMOKE at 12US2, and ptfire_othna3D through SMOKE at both 36US3 and 12US2. Then, the 36US3 ptfire_othna3D must be run through a "masking" program (Annual_ptfire_othna3D_mask_36US3_2016fg_16j.csh) which edits the 36US3 emissions by zeroing out the portion of the domain which overlaps 12US2, in order to prevent a double count. The mrgpt file then will include ptfire3D 12US2, ptagfire3D 12US2, ptfire_othna3D 12US2, and ptfire_othna3D masked 36US3. - CAMx v7 also supports use of multiple input files, instead of having to merge all point sources together. For this project, we kept ptegu files separate and merged all other point sources together, resulting in two mrgpt emissions files per day: ptegu, and all other point sources including fires. Converting CMAQ-ready emissions to CAMx-ready emissions consists of several steps. First, fire emissions must be processed through SMOKE in 3-D format instead of inline format so that the SMOKE outputs include layered emissions. This will need to be done for three sectors: ptfire, ptfire_othna, and ptagfire. As described above, for 2-way nesting applications, ptfire3D and ptagfire3D must be processed for 12US2, and ptfire_othna3D must be processed for both 36US3 and 12US2 with the 36US3 emissions then "masked". Scripts for processing these three sectors as 3-D layered sectors are provided in the CASE/scripts/point/ directory, alongside the regular scripts for processing inline emissions for CMAQ. The 3-D scripts specify '3D' in the filename, and '3D' has also been appended to the sector names to distinguish these runs from the inline runs (ptfire3D, ptagfire3D, ptfire_othna3D). These scripts will produce gridded and layered 3-D emissions in the 2016fg_16j/premerged directory, and will not produce any inline emissions files for CMAQ. Second, those 3-D fire emissions and all other point source emissions must be separately converted to CAMx format. Scripts for this step are provided in the 2016fg_16j/scripts/camx/ directory: camx_convert_2016ff_36US3.ptsr.cmv_c3_pt_oilgas.csh camx_convert_2016fg_12US1.ptsr.ptegu.csh camx_convert_2016fg_36US3.ptsr.othpt_ptnonipm.csh camx_convert_2016ff_12US2.ptagfire3D.csh camx_convert_2016fg_12US2.ptfire3D.csh camx_convert_2016fg_12US2.ptfire_othna3D.csh camx_convert_2016fg_36US3.ptfire_othna3D_masked.csh Third, convert the merged 2-D gridded emissions to CAMx format for both 36US3 and 12US2: camx_convert_2016fg_12US2.emis2D.csh camx_convert_2016fg_36US3.emis2D.csh New for this case, sea salt emissions are processed using a new processor called the "oceanic" processor. 2016fg_scripts_addendum_to_2016ff.zip includes this processor and it is called by the emis2D.csh scripts. After that, add a row and column of buffer cells to the edges of the 12US2 domain, as is necessary for 2-way nested modeling. The outputs from the "addbuffer" script will be labeled with the grid name "12US2b": camx_convert_2016fg_12US2.emis2D_addbuffer.csh Finally, merge the ptsr files processed in Step 2 together, EXCEPT for ptegu: camx_convert_2016fg_36US3.mrgpt.csh The mrgpt files will be the combination of point source emissions from ptnonipm 36US3, othpt 36US3, cmv_c3 36US3, pt_oilgas 36US3, ptfire3D 12US2, ptagfire3D 12US2, ptfire_othna3D 12US2, and ptfire_othna3D masked 36US3. All CAMx-ready files will go to: INSTALL_DIR/$CASE/smoke_out/$CASE/$GRID/camx_v7/ Mrgpt files will go under 36US3, and the 2-D files will go under 36US3 and 12US2b. The full set of files needed to be provided to the CAMx model for 2-way nested modeling using these scripts is: - emis2d file for 36US3 - emis2d file for 12US2b - mrgpt file for 36US3 - ptegu ptsr file for 12US1 or 12US2 (output to INSTALL_DIR/$CASE/smoke_out/$CASE/12US1/camx_v7/sector/ptegu/) == 8A. FORTRAN programs used by the CAMx converter == The CAMx converter uses several FORTRAN programs. The package contains both our compiled FORTRAN executables (which may or may not work on your system), and the original FORTRAN source code and Makefiles, so that you may recompile these helper programs yourself. The source code is in INSTALL_DIR/camx/src/, and our executables are in INSTALL_DIR/camx/bin/. If you have to recompile the code, this is where the recompiled executables should ultimately go. There are multiple versions of the cmaq2uam program, each of which is used. == 8B. Other files used by the CAMx converter == The subdirectories 'dates', 'land', 'oceanic_v4.1.1', 'Species_Mapping_Tables', and 'window' in INSTALL_DIR/camx/ include other files used by the converter. The addition of sea salt emissions requires CAMx-formatted meteorological data from WRF. Specifically, this requires a 'met3d' file and a 'met2d' file. The WRF_ROOT parameter in the final CAMx conversion script should point to the location of this data. Like the MCIP meteorology, these files are not included in the package. == 9. Log analyzer == The platform scripts include a Python-based tool called the log analyzer that runs automatically at the conclusion of each SMOKE job. The purpose of the log analyzer is to scan all log files from the sector, search for errors and warnings, and filter out common or "acceptable" warnings. Log analyzer output goes into this directory: $INSTALL_DIR/$CASE/reports/log_analyzer There are two types of output files: Level 1 and Level 3. Level 3 lists every instance of each error and warning individually, while Level 1 combines repeats of common warnings. It is sufficient to look at only the Level 1 output. The Level 1 output includes a "priority" code, the error/warning message, and the log file in which the message appears. The priority code is a number from 0 to 3. If priority = 2 or 3, the message has been identified as common or acceptable. This is based on a file called known_messages.txt, which is located here: $INSTALL_DIR/smoke4.5/scripts/log_analyzer/known_messages.txt If priority = 1, the message is included in known_messages and has been identified as NOT acceptable - however, many priority 1 warnings are in fact SOMETIMES acceptable and sometimes not, which is why they are not given priority 2 or 3. Some common priority 1 warnings are listed below. If priority = 0, the message is not included in known_messages. Priority 0 or priority 1 messages which are acceptable include: - WARNING: [*] is not found in both inventory pollutant and model species lists. These are common onroad messages resulting from the CFPRO file including more species than is always necessary. - WARNING: Could not read [BEGHOUR/ENDHOUR] from file "PDAY" This is a common ptfire warning and is acceptable. - WARNING: resetting surrogates ratio of Co/St/Ct (FIPS): ... This warning is in known_messages but is not always picked up by the log analyzer for reasons unknown. - WARNING: Speciation profile "?? " is not in profiles This warning is common/acceptable for onroad_mex. - WARNING: Total annual toxic emissions greater than annual [*]__VOC emissions for source: This warning is common for sectors with monthly inventories such as nonroad. - WARNING: Applying default time zone 5 to country/state/county code: [*] This warning is acceptable for Alaska FIPS and FIPS 85005. - WARNING: Duplicate entry in AR2PT x-ref file: This warning is acceptable but is not in known_messages to let us know we should correct it in the future. == 10. SMOKE reports (Smkreport) == By default, the run scripts run the Smkreport program, output to the $CASE/reports/inv directory. For sectors with annual inventories,reports are annual. For sectors with monthly inventories (e.g. nonroad), reports are monthly. Most reports include all inventory pollutants and model species, although PM10 usually appears as zero due to a SMOKE quirk; to get PM10, sum PM2_5 and PMC in the report. For onroad, these reports reflect activity, not emissions, and include some double counting due to how SMOKE allocates activity to different processes; therefore, you should not use the reports/inv reports for the onroad or onroad_ca_adj sectors. The following types of reports are generated. Note that not all types of reports are generated for all sectors: *state.txt: State totals. *county.txt: County totals. *state_scc.txt: State/SCC totals. *county_scc.txt: County/SCC totals. *state_naics.txt: State/NAICS totals. *cell_${GRID}.txt: Totals by grid cell. *cell_county_${GRID}.txt: Totals by grid cell and county. *state_grid_${GRID}.txt: State totals after gridding. *state_scc_srgid_${GRID}.txt: State/SCC totals after gridding, and also including the spatial surrogate assignment. *plant_cell_${GRID}.txt: Totals by grid cell and facility. *plant_oris.txt: Totals by EIS facility and ORIS facility. *state_scc_pm25prof.txt: State/SCC totals of PM2.5, and also including the PM2.5 speciation profile assignment. *state_scc_vocprof.txt: State/SCC totals of PM2.5, and also including the VOC speciation profile assignment. This report only includes no-integrate VOC; this will likely be changed for 2016 v1 platform. For some sectors, there are separate reports by mode (e.g. exhvocprof.txt). As part of beta platform, we have been generating other types of SMOKE reports for community distribution. It is expected that the "default" SMOKE report configurations will be changed for 2016 v1 platform in support of modeling community needs. For all sectors except those processed with SMOKE-MOVES, Smkmerge generates daily county total reports in the $CASE/reports/smkmerge directory. These are then summed to annual by state and county, output to the $CASE/reports/annual_report directory. Emissions totals in the reports/annual_report directory (post-temporalization) should be within 1-2% of the totals in the reports/inv directory (pre-temporalization). For SMOKE-MOVES sectors, the $CASE/reports/smkmerge directory includes daily (or weekly if DAYS_PER_RUN=7) totals by county/SCC. Scripts to aggregate these totals to monthly or annual by state, county, state/SCC, and county/SCC are provided in the SMOKE utilities zip, movesmrg_report_postproc/ directory. == 11. Combine Utility == The SMOKE core zip includes a pre-compiled program called combine which can manipulate and merge I/O API-formatted netCDF files. When unpacking the SMOKE core zip, the combine utility is unzipped to: INSTALL_DIR/combine/combine-terra.exe This utility is used within the emissions modeling platform for the onroad, onroad_ca_adj, and onroad_mex sectors to convert emissions speciated for CB6-CAMx to CB6-CMAQ, as described in earlier sections. However, the program can be used independently for custom purposes, as described below. To use the combine program, you must first set up a species definition file. There are several example files in the same directory as the combine-terra.exe executable. To create a new species definition file, first start with this column header: /orig species ,units ,expression Then, enter a formula for each species that you wish to be included in the output from combine. This list must include ALL species, even those which are not being manipulated. The "orig species" column is the output species. The units are typically g/s (PM) or moles/s (most other species). The "expression" column is a formula which references one or more input files. Some examples are provided below. When running combine, the following environment variables should be set: SPECIES_DEF: point to species definition file OUTFILE: output file name INFILE1: first input file name (indexed with [1] in SPECIES_DEF) INFILE2: second input file name, if necessary (indexed with [2] in SPECIES_DEF) INFILE3: third input file name, if necessary (indexed with [3] in SPECIES_DEF) ...and so on, with as many INFILEs as needed. Here is one example use of combine. Let's say you have a merged emissions file, and would like to replace only the ag emissions from that dataset with an alternative dataset without having to remerge all of the sectors together again. You could set the INFILE variables as follows: INFILE1 = original merged emissions INFILE2 = original ag sector emissions INFILE3 = alternative ag sector emissions For each species which is in the ag sector, use this formula, in which new merged = old merged - old ag + new ag: NH3, moles/s, NH3[1]-NH3[2]+NH3[3] For each species which is NOT in the ag sector, pass through the original merged emissions without modification: PEC, g/s, PEC[1] These species must all be explicitly listed in the SPECIES_DEF, otherwise they will not be passed through to the output. One final note: Emissions files output with the combine utility may not immediately work with all visualization tools and utilities. To remedy this, we pass emissions files output by combine through the m3xtract I/O API utility, with all timesteps and species output as is. This reformats the file slightly so that it is compatible with other tools.