Last update: June 28 2019 *** DISCLAIMER ***** *** THESE DATA ARE BEING PROVIDED BEFORE AIR QUALITY MODEL RESULTS *** USING THESE DATA HAVE BEEN REVIEWED AND/OR PRODUCED *** THEY ARE BEING PROVIDED TO ALLOW FOR PRELIMINARY MODELING OF 2014-16 *** *** PLEASE READ THIS README BEFORE RUNNING SMOKE. *** == 1. Introduction == These packages contain scripts, inventories, and ancillary files related to EPA's 2014v7.1 platform for air quality modeling. This package includes inputs for three modeling years: 2014, 2015, and 2016. The year-specific emissions files are here: ftp://newftp.epa.gov/air/emismod/2014/v2/2014fd/emissions/ ftp://newftp.epa.gov/air/emismod/2015/alpha/2015fd/emissions/ ftp://newftp.epa.gov/air/emismod/2016/alpha/2016fd/emissions/ The ancillary data for all years are stored with the 2016 alpha platform: ftp://newftp.epa.gov/air/emismod/2016/alpha/ CMAQ model-ready emissions generated using these packages with SMOKE v4.5 should be identical to those used in EPA's 2014v7.1 platform, with some exceptions as noted in this README. There may be additional emissions differences resulting from differences in the Linux operating system, hardware platform, and other system-specific differences. The data files are divided into several directories: - 2014fd/emissions contains emission inventories for the year 2014, including national CEMS emissions, nonroad inventories and onroad activity data, and point and nonpoint inventories. - 2015fd/emissions and 2016fd/emissions contain similar files to 2014fd/emissions for years 2015 and 2016. As of 24 Apr 2018, 2016 point emissions are now available. On 31 May 2018, the nonroad inventories for 2015 and 2016 were updated due to errors in the originally posted inventories for those two years. - ancillary_inputs contains general ancillary files (ge_dat), including those related to speciation, spatial allocation (gridding), temporalization, and gridded ocean chlorine emissions files, which are included in the final model-ready emissions. - smoke_2014v7_1_platform_core.zip contains scripts to run the core SMOKE programs along with precompiled SMOKE executables from SMOKE version 4.5. - smoke_2014v7_1_platform_utilities.zip contains sample scripts for running other SMOKE programs, or programs that create SMOKE inputs. - spatial_surrogates contains 12km spatial surrogates for the US, Canada, and Mexico, and 4km spatial surrogates for the US and Canada. Section 4 of this README includes information about the modeling sectors used in the platform. Section 5 of this README includes information about the inventories provided. Section 6 of this README includes information about the ancillary (non-inventory) files included. == 2. Requirements for processing emissions for air quality modeling == If you are only reviewing inventories and not developing emissions for air quality modeling, you do not need to install SMOKE or to follow the instructions below. Instead, unzip the files with the data of interest and examine those and the corresponding reports that are provided. If you plan to develop emissions for air quality modeling, SMOKE v4.5 is REQUIRED to process this case. SMOKE v4.5 includes additional updates and bug fixes as compared to versions 3.7 and 4.0, and so SMOKE v4.5 should be used when processing these emissions cases. The smoke_2014v7_1_platform_core.zip package includes precompiled executables of SMOKE v4.5 programs. These executables reflect SMOKE v4.5 as of 7 Mar 2018 and may not reflect the official public release of SMOKE v4.5. Python: We also recommend (if not require) python version 3.0 or later, along with select python libraries. Many of the helper scripts included in this package use python. The python scripts within this package reference '#!/usr/bin/env python'; you may need to change this on your computing platform. All of the scripts have been updated for Python 3.5 to match the configuration on EPA's cluster. You should still be able to use Python 2.7 if you install the "future" modules. In most cases that can be done by running: sudo pip install future == 3. Installation of data files and scripts == This readme covers the installation of the SMOKE inventories, scripts, and ancillary files used for the v7.1 platform. Choose an install directory on your system; we will refer to this directory as "INSTALL_DIR". To review/reproduce emissions for all sectors, unzip all the .zip files into INSTALL_DIR. The packages have subdirectories embedded within them, so it is important that all files be unpacked in the same place in order for the scripts know where to find the inputs. If you are only interested in reproducing or examining emissions for specific sectors, you may download and unzip only the data for those sectors from the inventory directories, but for SMOKE processing you should also include the files in the ancillary_inputs and spatial_surrogates directories, and also the contents of smoke_2014v7_1_platform_core.zip. Precompiled SMOKE executables and I/O API utilities are available in the SMOKE zip. A separate SMOKE "utilities" zip includes additional scripts and files that are not necessary to run SMOKE, but are for developing SMOKE inputs or processing SMOKE outputs. All SMOKE inventories, scripts, and ancillary files used for the v7.1 platform are provided, except emission factor tables for onroad processing via SMOKE-MOVES. The full set of emission factor tables is too large to permanently store on the FTP server and instead are furnished upon request. Prior to running SMOKE, you will need to edit the INSTALL_DIR (your install directory) and MET_ROOT (location of MCIP meteorology data) environment variable definitions in the "directory_definitions.csh" script located in each CASE/scripts directory. This script is sourced by each of the individual run scripts for each sector. Regarding the MCIP data, SMOKE only uses the GRIDCRO2D, METCRO2D, and METCRO3D files. MCIP meteorology data is not included in the package. This is used for the onroad, onroad_ca_adj, and biogenics (beis) sectors. == 4. Case description and instructions for each sector == Scripts are provided to process emissions for a 12km national grid (12US2) using CB6 speciation for CMAQ version 5.2. The CB6 for CMAQ mechanism is slightly different than the CB6 mechanism for the CAMx model from the 2011 emissions platform: it includes a new tracer species (SOAALK), naphthalene is now explicit in the mechanism (NAPH), and the XYL species is replaced by XYLMN (XYL minus naphthalene). The emissions from this package will work with base CMAQ v5.2 but not the Multi-Pollutant version used for the National Air Toxics Assessment (NATA). CMAQ versions 5.1 and later include support for the CB6 mechanism. These emissions can also be converted to CB6-for-CAMx speciation when converting to CAMx format. Speciation profiles for the CB05 and SAPRC mechanisms are posted on the FTP server under the 2011v6.2 platform and 2011v6.3 platform. Those GSPRO files do not include newer speciation profiles that debuted in the 2014 platform, however. GSPRO files including all new 2014 platform speciation profiles for these other mechanisms may be posted in the future. Emissions for the v7.1 platform have thus far only been processed for CMAQ. This package does not include tools for converting the emissions to the format needed for CAMx modeling like in the 2011 platform. See Section 8 of this README for instructions on how to adapt the CAMx conversion scripts from the 2011 platform to these new modeling cases. Emissions processing is split into "sectors". Each sector has its own run scripts for processing, with one (or more) run scripts per case. See Section 7 of this README for information about the run script zips. All sectors are US-only unless otherwise noted. The sectors are: AFDUST: Particulate emissions from fugitive dust sources. This sector is processed in two steps. The first (Annual_afdust_12US2_*) processes the annual inventory, and the second (Annual_afdust_adj_12US2*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "afdust_adj". The afdust scripts must be run in that order. AG: Agricultural emissions. This is mostly ammonia, but unlike in prior modeling platforms, this sector now includes other pollutants from agricultural sources as well. BEIS: Biogenic emissions generated using the BEIS model, version 3.6.1. CMV_C1C2: Emissions from C1 and C2 commercial marine sources, including ports and navigable waterways. Includes offshore C1/C2 marine emissions in the Atlantic and Pacific Oceans, and the Gulf of Mexico. CMV_C3: Emissions from C3 commercial marine sources, including ports and navigable waterways. Includes offshore C3 marine emissions in the Atlantic and Pacific Oceans and the Gulf of Mexico. Canadian C3 emissions are in the othar sector. The 2014fd emissions from this sector equal those from 2014NEIv2 Nonpoint, but we process as a point sector in order to support plume rise in CMAQ. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). NONPT: Area source emissions not included in other sectors. NONROAD: Off highway mobile source emissions. NP_OILGAS: Area source oil and gas emissions. ONROAD: On highway mobile source emissions, excluding California. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B. ONROAD_CA_ADJ: On highway mobile source emissions, California only. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B. OTHAFDUST: Particulate emissions from fugitive dust sources in Canada. Just like with afdust, this sector is processed in two steps. The first (Annual_othafdust_12US2_*) processes the annual inventory, and the second (Annual_othafdust_adj_12US2*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "othafdust_adj". The othafdust scripts must be run in that order. Fugitive dust emissions in Mexico are included in the othar sector and do not need the same transportable fraction and meteorological adjustments that the Canada fugitive dust emissions in othafdust do. OTHAR: Area source emissions from Canada and Mexico, including mobile nonroad. ONROAD_CAN: Mobile onroad source emissions from Canada. ONROAD_MEX: Mobile onroad source emissions from Mexico. The onroad Mexico emissions inventory includes pre-speciated VOC emissions for the CB6-for-CAMx mechanism, so there is an extra script for this sector to convert those emissions to the CB6 mechanism needed for CMAQ. This extra script is called *_part2_combine.csh and uses the combine utility to perform the CB6-for-CMAQ conversion. The combine program is included, pre-compiled, in the SMOKE package along with pre-compiled SMOKE executables and I/O API utilities. To help make the distinction between CB6-for-CAMx and CB6-for-CMAQ emissions, CB6-for-CAMx emissions use the sector name "onroad_mex_cb6orig". The CB6-for-CMAQ post-processing step creates emissions files with the final sector name "onroad_mex". OTHPT: Point source emissions from Canada and Mexico. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). PTAGFIRE: Point source agricultural burning emissions. The ptagfire sector uses a daily point source inventory. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. The 2014fd emissions from this sector equal those from 2014NEIv2 nonpoint, but we process as a daily point sector in order to include more resolved temporalization and support plume rise in CMAQ. All emissions in this sector are elevated (no low-level contribution). PTEGU: Electric generating unit emissions. This sector incorporates CEM (Continuous Emissions Monitoring) hourly emissions for a majority of sources. This is a 'point' sector, and like all 'point' sectors, is processed via a 'onetime' script first, followed by a 'daily' script. For ptegu there are two 'daily' scripts for different months of the year: 'summer' (May through September), and 'winter' (October through April). For sources without hourly CEM emissions, summer and winter use different hourly temporalization, and so they are run with separate inputs. All emissions in this sector are elevated (no low-level contribution). PTNONIPM: Point source emissions from industrial activities. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. Some emissions in this sector are elevated while others are low-level, so both 2-D emissions files (emis_mole) and inline emissions files (inln_mole) are generated. PTFIRE: Point source emissions from year specific controlled burning and wild fires. Fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PTFIRE_OTHNA: Point source emissions from year specific controlled burning and wild fires in the rest of North America ('OTHNA' = OTHer North America), including Canada and Mexico. For 2014, fire emissions for only Canada and Mexico are included. For 2015 and 2016, in addition to Canada and Mexico, fire emissions for Central America and the Caribbean are also included. Emissions from those areas are ultimately not modeled due to being outside of the 12US2 modeling domain, but they are provided for possible use in larger grids. These fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PT_OILGAS: Point source oil and gas emissions, including emissions from offshore oil rigs in the Gulf of Mexico. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. Some emissions in this sector are elevated while others are low-level, so both 2-D emissions files (emis_mole) and inline emissions files (inln_mole) are generated. RAIL: Area source railway emissions. RWC: Area source residential wood combustion emissions. == 4B. Notes regarding onroad == Onroad emissions are processed using SMOKE-MOVES. The processing is split into multiple run scripts. In the v7.1 platform, SMOKE-MOVES inputs were prepared using the latest version of MOVES: MOVES2014a. As in recent platforms, speciation of VOC emissions is handled within the MOVES model. At the time MOVES2014a was run for this emissions case, MOVES included the CB6 for CAMx mechanism, but not the CB6 for CMAQ mechanism needed for CMAQ 5.2. So, SMOKE-MOVES is run for the CB6-for-CAMx mechanism, and then converted to CB6-for-CMAQ in an additional post-processing step. To help make the distinction between CB6-for-CAMx and CB6-for-CMAQ emissions, CB6-for-CAMx emissions from SMOKE-MOVES use the sector names "onroad_cb6orig" and "onroad_ca_adj_cb6orig" instead of "onroad" and "onroad_ca_adj". The CB6-for-CMAQ post-processing step creates emissions files with the final sector names "onroad" and "onroad_ca_adj". As described in the SMOKE online documentation, SMOKE-MOVES handles onroad emissions separately for four types of processes: - On-network emissions (RatePerDistance, or RPD) - Off-network emissions, fuel vapor venting (RatePerProfile, or RPP) - Off-network emissions, extended idling (RatePerHour, or RPH) - Off-network emissions, non-venting, non-extended idle (RatePerVehicle, or RPV) For each of the two onroad sectors (onroad, onroad_ca_adj), there are separate run scripts for RPD, RPP, RPH, and RPV, plus a merge script that combines emissions from RPD, RPP, RPH, and RPV into a single emissions file per day, and a "part2" merge script that converts the emissions from CB6-for-CAMx to CB6-for-CMAQ. The CB6-for-CMAQ conversion is performed using the combine utility which is included, pre-compiled, in the SMOKE package along with pre-compiled SMOKE executables and I/O API utilities. These scripts may take a particularly long time to run, especially RPD. Therefore, consideration should be given to running multiple RPD jobs in parallel, such as one job per quarter. The reason onroad has been split into two sectors - onroad and onroad_ca_adj - is in order to match SMOKE-MOVES annual emission totals with those provided by California. To do this, we split California into a separate sector, and run SMOKE-MOVES with a control factor file (CFPRO) which nudges the emissions so that the annual totals post-SMOKE-MOVES equal those provided by CARB. CARB provided updated onroad emissions inventories for use in the v7.1 platform, and we match their provided emissions at the county/SCC level, except that the CARB inventory does not distinguish between different on-network road types (but does distinguish on-network emissions from off-network). For the onroad sector, a CFPRO is provided in the ancillary_inputs/ge_dat_*_other.zip package. This CFPRO zeroes out refueling emissions in 52 Colorado counties. We zero out these emissions in order to prevent a double count with the ptnonipm sector, since that sector includes refueling emissions in these counties. The portion of this CFPRO pertaining to Diesel PM is only relevant for multipollutant modeling (e.g. NATA). To help identify which species and pollutants to expect from each of the onroad components (RPD, RPH, RPP, RPV), sample Movesmrg reports for one day are provided in: reports/onroad_sample_movesmrg_reports.zip The sample reports are for California, but for the purposes of checking the existence of species, these reports can be used nationwide. == 4B1. DAYS_PER_RUN == SMOKE-MOVES can be run more efficiently if running more than one day at a time. For example, Movesmrg can create one 7-day emissions file more quickly than it can create seven individual 1-day emissions files. To turn on this feature, use the DAYS_PER_RUN variable, set to the number of days you wish to run in a single Movesmrg instance. The recommended value for DAYS_PER_RUN is 7. The onroad scripts include a setting called "DAYS_PER_RUN", set to 1 as the default. If DAYS_PER_RUN > 1, after Movesmrg is run, the run scripts will use the I/O API utility m3xtract to split up the multi-day emissions file into single day (25-hour) emissions files. Multi-day Movesmrg runs will never cross months. For example, if DAYS_PER_RUN = 7, then the last Movesmrg run of January will start on January 29th and end on January 31st (3 days), and the first Movesmrg run of February will start on February 1st and end on February 7th. Using the multi-day Movesmrg functionality requires multi-day MCIP files. For example, if DAYS_PER_RUN = 7, your METCRO2D files must also be 7 days (169 hours) long. These multi-day MCIP files should be stored in MET_ROOT_${Xday}/, where X = DAYS_PER_RUN (i.e. /7day for DAYS_PER_RUN = 7). For example, if the single day MCIP files are in /foo/foo/mcip_dir/, then 7-day MCIP files should be in /foo/foo/mcip_dir_7day/. The primary drawback to using this multi-day Movesmrg functionality is an increase in the memory usage. == 4C. Sector merge == After all sectors have been processed, the Sector_merge script merges the low-level emissions from all sectors into a single CMAQ-ready emissions file per day. Merged model-ready emissions will be output to: INSTALL_DIR/$CASE/smoke_out/$CASE/$GRID/$SPC/ Inline emissions and stack_groups files will be output to the same directory, except in subdirectories by sector name (e.g. .../$SPC/ptnonipm/). By default, the sector merge scripts are configured to include biogenics. Some CMAQ modelers may wish to process biogenic emissions inline within CMAQ and not include biogenic emissions in the gridded emissions files. To run the sector merge without biogenics, edit the sectorlist file in the $CASE/scripts directory, and set the mergesector column to N for the 'beis' sector. To merge in alternative biogenic emissions files, edit the sectorlist by changing the 'beis' sector name to the sector name of your choice, and make sure your biogenic emissions files exist in the $CASE/premerged/[sector name] directory with filenames adhering to the file name convention used by other sectors. == 5. Description of inventory packages == Inventories for the 2014fd case are included in the following files, all of which should be unpackaged in INSTALL_DIR. Inventories for 2015fd and 2016fd are organized similarly. Note that some sectors, in particular nonpoint sectors, use the same inventories across all three years. 2014fd/emissions/2014fd_inputs_biogenics.zip 2014fd/emissions/2014fd_inputs_cem.zip 2014fd/emissions/2014fd_inputs_nonpoint.zip 2014fd/emissions/2014fd_inputs_nonroad.zip 2014fd/emissions/2014fd_inputs_onroad.zip 2014fd/emissions/2014fd_inputs_oth.zip 2014fd/emissions/2014fd_inputs_point.zip 2014fd/emissions/2014fd_inputs_ptfire.zip 2014fd/emissions/2014fd_SMOKE_MOVES_onroad_ff10.zip For 2015 and 2016, nonroad zip file has "corrected" in the filename, because these inventories were updated on 31 May 2018 due to errors in the previously posted nonroad inventories for those two years. The "biogenics" package includes gridded land use, BIOSEASON, and biogenic emission factor files for input to BEIS 3.6.1. The latest version of BEIS uses different land use and emission factor inputs compared to prior versions, such as BEIS 3.14. The v7.1 platform includes updated land use: BELD 4.1. This package also includes a b3grd file for 12US2. This file is output by the Normbeis program, and is included in the package for those people who do not wish to run Normbeis and generate their own b3grd using the provided land use and emission factor files. This b3grd file is not speciation-specific and can be used with any speciation mechanism. For some years, the biogenics zip also includes b3grd, BIOSEASON, and land use files for 12US1. Land use and b3grd files can be used interchangeably between years, but BIOSEASON is year-specific. The "cem" package includes the hourly CEM (Continuous Emissions Monitoring) emissions used by the ptegu sector. This is the same data that is available on EPA's Air Markets Program Data website (ampd.epa.gov), except that we've split the data into months and days as needed for our scripts, and run the data through a "CEM correct" program. The "inputs_nonpoint" package includes inventories for the following sectors: afdust, ag, cmv_c1c2, cmv_c3, nonpt, np_oilgas, rail, rwc. Even though cmv_c3 is processed as a point sector, we included its inventories in the nonpoint zip because its emissions come from NEI Nonpoint. Agricultural fire emissions also come from NEI Nonpoint, but those inventories are included in the ptfire zip. The "inputs_nonroad" package include the inventories for the nonroad sector. To reduce the file size, pollutants not needed for normal CMAQ modeling were removed prior to packaging, such as metals, PAHs, and dioxins and furans. The "inputs_onroad" package includes the activity data for the onroad and onroad_ca_adj sectors. It does not include the emission factor tables also required to run SMOKE-MOVES; these are not stored on the FTP site due to their large size and instead are furnished upon request. Emission factors are only available for CB6. The "inputs_oth" package includes all inventories for Canada and Mexico, except fires. The "inputs_point" package includes the inventories for the following sectors: ptnonipm, ptegu, pt_oilgas. As of 24 Apr 2018, point inventories for 2016 are now available. The "inputs_ptfire" package includes the inventories for the ptfire, ptfire_othna, and ptagfire sectors. The "SMOKE_MOVES_onroad_ff10" package includes an FF10-formatted inventory representing emissions totals for the onroad sector as calculated by SMOKE-MOVES. This is provided for those who are interested in an onroad emissions inventory or report. It is not needed for SMOKE modeling. See Section 4 of this README for a description of each modeling sector. == 6. Description of ancillary file packages == The following packages should be unpacked in INSTALL_DIR: ancillary_data/ge_dat_for_2014_15_16_alpha_platform_gridding.zip ancillary_data/ge_dat_for_2014_15_16_alpha_platform_other.zip ancillary_data/ge_dat_for_2014_15_16_alpha_platform_speciation.zip ancillary_data/ge_dat_for_2014_15_16_alpha_platform_temporal.zip ancillary_data/ocean_chlorine.zip spatial_surrogates/surrogates_CONUS12_2013CAN_2010MEX_19jun2017.zip spatial_surrogates/surrogates_CONUS12_2014_v1.3_29nov2016.zip The "surrogates" packages contain the spatial surrogates for 12US2 for the US, and for Canada and Mexico. 4km surrogates for the US and Canada are also provided in separate zips. The "gridding" package includes all SMOKE inputs related to spatial allocation other than the surrogates, including cross-references, surrogate descriptions, and gridded transportable fractions used in afdust_adj and othafdust_adj. The "speciation" package includes speciation profiles, cross-references, VOC-to-TOG conversion factors, NHAPEXCLUDE files (concerns VOC HAP integration), and inventory tables. This .zip includes files for the CB6 mechanism for CMAQ v5.2. The "temporal" package includes temporal profiles and cross-references, including daily and hourly temporal profiles developed by the SMOKE program Gentpro for use in the rwc and ag sectors. The "other" ge_dat package includes all other SMOKE ancillary files not included in the above packages, including: - SMOKE-MOVES ancillary files, including the reference county (MCXREF) and fuel month (MFMREF) cross-references, pollutant (MEPROC) and emission factor table (MRCLIST) lists, activity SCC to full SCC cross-references (SCCXREF), hourly speed profiles (SPDPRO), daily temperature data (METMOVES), and Movesmrg adjustment factors (CFPRO) - Other miscellaneous SMOKE inputs, such as the ARTOPNT, COSTCY, HOLIDAYS, MACTDESC, NAICSDESC, ORISDESC, PELVCONFIG, PSTK, SCCDESC - Smkreport configuration files (REPCONFIG, all in ge_dat/smkreport/repconfig) This ocean_chlorine.zip package contains gridded ocean chlorine emissions, which are included in the sector merge, for 12US1, 12US2, and 36US1. The run scripts (see section 7) are already set up to use the proper ancillary files and inventories for each sector and case. There is also additional zip file in the ancillary_data directory concerning onroad. This extra zip is not needed to process the 2014/15/16 emissions: ancillary_data/metmoves_12US2_2011_2012_2013_2014_2015_2016.zip - Includes gridded met4moves outputs for SMOKE for the 12US2 grid for all years 2011 through 2016. The files for 2014 through 2016 are also included in ge_dat_for_2014_15_16_alpha_platform_other.zip. == 7. Description of script packages == The smoke_2014v7_1_platform_core.zip package should be unpacked in INSTALL_DIR. This includes scripts and precompiled executables for running SMOKE in general, and for running the 2014fd/2015fd/2016fd modeling cases in particular. The scripts in the 2014fd_cb6_14j, 2015fd_cb6_15j, and 2016fd_cb6_16j subdirectories are the scripts you run directly in order to replicate our emissions. Separate script(s) are provided for each sector. See section 4 for information pertinent to each sector. In general, you edit the directory_definitions.csh file, in particular INSTALL_DIR and MET_ROOT, and then run each sector. Sector scripts are organized into subdirectories within CASE/scripts by sector category: biogenics, nonpoint, onroad, point, and merge. For afdust sectors, run afdust/othafdust first, then afdust_adj/othafdust_adj. For point sectors, run "onetime" first, and then "daily". For onroad sectors, run RPV/RPD/RPH/RPP first, then the merge script, then the "part2_combine" script. The onroad_mex sector also has a separate part2_combine script. The scripts, programs, and other inputs in the combne, ioapi, and smoke4.5 subdirectories are all "helper" scripts and inputs, and generally never need to be run directly. == 7B. Other miscellaneous programs == The smoke_2014v7_1_platform_utilities.zip package includes the following scripts for miscellaneous pre-SMOKE and post-SMOKE applications. Unzipping the full contents of the utilities zip creates the following directories, each concerning one specific application: afdust_xportfrac/: Afdust emissions are adjusted by applying a transportable fraction file (XPORTFRAC) to reduce the emissions after SMOKE is run. The ge_dat zips include XPORTFRAC files for 12US1 and 12US2. Sample scripts and inputs to generate XPORTFRAC files for other grids are provided here. beld4.1_land_use/: Land use files for processing biogenic emissions with BEIS for 12US1 and 12US2 are included in the ge_dat zips. To generate land use files for other grids, inputs and sample scripts are provided here. Includes the most recent BELD 4.1 update from February 2017. bioseason/: Includes a sample script for creating a BIOSEASON file for use with BEIS. camx/: Helper scripts for converting SMOKE outputs to CAMx-ready emissions (see Section 8). cem_correct/: A program is provided which corrects anomalous values in the continuous emissions monitoring (CEM) hourly emissions data available on the CAMD website. This program has already been run on the hourly CEM data provided in these packages. gentpro_scripts/: For those wishing to generate their own meteorology-dependent temporal profiles for residential wood combustion (RWC) or agricultural emissions using the SMOKE program Gentpro, sample Gentpro scripts and inputs have been provided here. Note that in this platform, all ag sector emissions - both livestock and fertilizer, NH3 and other pollutants - are using hourly profiles developed from Gentpro. This is a change from prior platforms where only NH3 from livestock was temporalized as such. met4moves/: The ge_dat zips include met4moves outputs used by SMOKE based on EPA meteorological data. For alternate sets of meteorological data, met4moves should be run separately. SMOKE uses the "SMOKE_DAILY" gridded files output by met4moves. Sample scripts for running met4moves are here. moves2smk_postprocessing/: For processing onroad emissions, we run MOVES to create a set of emission factor tables to be used by SMOKE. MOVES does not actually create the emission factor tables directly; instead, MOVES creates a series of databases, which are converted to emission factor tables using a MOVES post-processor script called "moves2smk". The moves2smk script and its inputs are provided here. movesmrg_report_postproc/: Onroad processing with SMOKE-MOVES does not use an input emissions inventory with county/SCC-level emissions already defined. Instead, we need SMOKE-MOVES outputs to serve as the emissions "inventory" for report and summary purposes. Therefore, Movesmrg is set up to create full county /SCC-level daily emissions reports. A script to aggregate the Movesmrg daily reports to more digestable annual and monthly reports by state, state/SCC, county, and county/SCC, is provided here. smk_merge_dates/: Our emissions platforms process sectors with "representative days" - for example, one day per week for a month, rather than every day of the month. This reduces computational requirements. Our Mrggrid sector merge script uses a set of "merge dates" files when running Mrggrid for each calendar day, which tells the run script which representative day maps to each calendar day. Merge dates files for 2014, 2015, and 2016 are part of the core package. To create merge dates files for other years, a script is provided here. == 8. Preparing emissions for CAMx == Preparing emissions for the CAMx model instead of the CMAQ model requires converting the emissions to CAMx-ready format after SMOKE is run. For the v7.1 platform we have not yet prepared emissions for CAMx, so scripts to handle the conversion are not provided. But, CAMx conversion scripts from the 2011v6.3 platform release can be adapted to the v7.1 platform. Instructions on how to do that are provided below. 2011v6.3 platform CAMx conversion scripts can be retrieved from: ftp://newftp.epa.gov/air/emismod/2011/v3platform/smoke_2011en_2023en.zip All we need from this package are the scripts in 2011en_cb6v2_v6_11g/scripts/camx/. The scripts in INSTALL_DIR/camx/* are mostly the same as those in smoke_2014v7_1_platform_utilities.zip. First, fire emissions must be processed through SMOKE in 3-D format instead of inline format so that the SMOKE outputs include layered emissions. This will need to be done for three sectors: ptfire, ptfire_othna, and ptagfire. To use the existing ptfire/ptfire_othna/ptagfire scripts for 3-D processing, change the following script settings: - append '3D' to the end of the SECTOR parameter (e.g. setenv SECTOR "ptfire_othna3D") - setenv ELEVPOINT_DAILY "" - setenv INLINE_MODE "off" - setenv SMK_SPECELEV_YN "N" - setenv SMK_EMLAYS [X], where [X] = the number of layers in your 3-D meteorology. Common settings include 25 and 35. - make sure your meteorology as defined by directory_definitions.csh (MET_ROOT) includes layered METCRO3D and METDOT3D files Second, those 3-D fire emissions must be separately converted to CAMx format. Start with the fire-specific scripts in 2011en_cb6v2_v6_11g/scripts/camx/. Change the sector name (if needed), start and end dates, grid specs (if not 12US2), and YEAR. Also, set SPECIES_MAPPING to: $INSTALL_DIR/camx/Species_Mapping_Tables/MAPTBL.CAMX_CB6+AERO6_to_CAMXv6_CB6+CF+RTRACPM_3jul2017_bnza_from_CB6CMAQ This new species mapping table was not included in the v6.3 package but is included in smoke_2014v7_1_platform_utilities.zip. Do this the same way for the three fire sectors: ptfire, ptfire_othna, and ptagfire. Third, create a new mergedates file for your year of interest in the INSTALL_DIR/camx/dates directory. Concatenate the smk_dates files from smoke4.5/scripts/smk_dates/$YEAR for your year and reformat similarly to the 2011 file that is provided in the v6.3 package. Finally, run the final CAMx conversion script. Starting from camx_convert_2011en_cb6v2_12US2.csh, change: - start_date, end_date, YEAR - WRF_ROOT (point to your WRF output, needed for sea salt calculation) - grid specs (if not 12US2) - SECTORLIST (point to the sectorlist file in $CASE/scripts) - set the ptfire path variables to point to the output from the second step; needed for ptfire, ptfire_othna, and ptagfire - where it says 'set firesectors' and 'set firepaths', set firesectors to (ptfire3D ptfire_othna3D ptagfire3D) and set firepaths equal to the three paths you defined earlier; make sure the paths in the same order as the sectors - for the SECOND 'set sectors' (which includes fire sectors) definition, make sure all three fire sectors are on this list == 9. Log analyzer == The platform scripts include a Python-based tool called the log analyzer that runs automatically at the conclusion of each SMOKE job. The purpose of the log analyzer is to scan all log files from the sector, search for errors and warnings, and filter out common or "acceptable" warnings. Log analyzer output goes into this directory: $INSTALL_DIR/$CASE/reports/log_analyzer There are two types of output files: Level 1 and Level 3. Level 3 lists every instance of each error and warning individually, while Level 1 combines repeats of common warnings. It is sufficient to look at only the Level 1 output. The Level 1 output includes a "priority" code, the error/warning message, and the log file in which the message appears. The priority code is a number from 0 to 3. If priority = 2 or 3, the message has been identified as common or acceptable. This is based on a file called known_messages.txt, which is located here: $INSTALL_DIR/smoke4.5/scripts/log_analyzer/known_messages.txt If priority = 1, the message is included in known_messages and has been identified as NOT acceptable - however, many priority 1 warnings are in fact SOMETIMES acceptable and sometimes not, which is why they are not given priority 2 or 3. Some common priority 1 warnings are listed below. If priority = 0, the message is not included in known_messages. Priority 0 or priority 1 messages which are acceptable include: - WARNING: [*] is not found in both inventory pollutant and model species lists. These are common onroad messages resulting from the CFPRO file including more species than is always necessary. - WARNING: Could not read [BEGHOUR/ENDHOUR] from file "PDAY" This is a common ptfire warning and is acceptable. - WARNING: resetting surrogates ratio of Co/St/Ct (FIPS): ... This warning is in known_messages but is not always picked up by the log analyzer for reasons unknown. - WARNING: Speciation profile "?? " is not in profiles This warning is common/acceptable for onroad_mex. - WARNING: Total annual toxic emissions greater than annual [*]__VOC emissions for source: This warning is common for sectors with monthly inventories such as nonroad. - WARNING: Applying default time zone 5 to country/state/county code: [*] This warning is acceptable for Alaska FIPS and FIPS 85005. - WARNING: Duplicate entry in AR2PT x-ref file: This warning is acceptable but is not in known_messages to let us know we should correct it in the future.