Last update: 20 Mar 2023 *** PLEASE READ THIS BEFORE RUNNING SMOKE. *** WE ARE INTERESTED TO KNOW HOW THESE DATA ARE BEING USED. PLEASE EMAIL eyth.alison@epa.gov with a subject: 2019 data usage AND A BRIEF NOTE DESCRIBING HOW YOU ARE USING THE DATA. THANK YOU. This package contains files relevant to the 2019 emissions modeling platform. It includes scripts and executables for processing 2019ge sectors through SMOKE. This package includes precompiled SMOKE 4.8.1. As of March 2023, The Smkreport executable which is part of SMOKE 4.8.1 is not stable and does not work for all sectors. Therefore, the default Smkreport executable included in this package is from SMOKE 4.8. == 1. Introduction == These packages contain scripts, inventories, and ancillary files related to EPA's 2019 platform for air quality modeling. This package includes inputs and scripts for modeling year 2019. The basis of this platform is the 2017 National Emissions Inventory (2017NEI). The year-specific emissions files are here: https://gaftp.epa.gov/air/emismod/2019/2019emissions/ CMAQ model-ready emissions generated using these packages with SMOKE v4.8.1 should be identical to those used in EPA's 2019 platform except for the speciation. EPA modeled this 2019 emissions case as a "HAP-CAP" case including many toxics. The scripts and inputs packaged here are for standard CAP-focused modeling. In addition to there being fewer HAPs included in this run compared to in HAP-CAP modeling, the treatment of certain VOCs is different in this case versus the equivalent HAP-CAP case. This package is intended to support a typical modeling application rather than a specialized HAP-CAP application. The data files are divided into several directories: - 2096emissions/ contains emission inventories for the year 2019, including national CEMS emissions, nonroad inventories and onroad activity data, and point and nonpoint inventories. - ancillary_data/ contains general ancillary files (ge_dat), including those related to speciation, spatial allocation (gridding), temporalization, and gridded ocean chlorine emissions files, which are included in the final model-ready emissions. - smoke_2019_platform_core.zip contains scripts to run the core SMOKE programs along with precompiled SMOKE executables from SMOKE version 4.8.1. - spatial_surrogates contains 12km spatial surrogates for the US, Canada, and Mexico, Spatial surrogates for other domains are available in other platform releases, such as 2016 v3 platform. Section 4 of this README includes information about the modeling sectors used in the platform. Section 5 of this README includes information about the inventories provided. Section 6 of this README includes information about the ancillary (non-inventory) files included. == 2. Requirements for processing emissions for air quality modeling == If you are only reviewing inventories and not developing emissions for air quality modeling, you do not need to install SMOKE or to follow the instructions below. Instead, unzip the files with the data of interest and examine those and the corresponding reports that are provided. If you plan to develop emissions for air quality modeling, SMOKE v4.8.1 is REQUIRED to process this case. SMOKE v4.8.1 includes new features and bug fixes required for the platform. The SMOKE core script package includes precompiled executables of SMOKE v4.8.1 programs. These executables reflect SMOKE v4.8.1 as of June 2021. The run scripts included in this package use C-shell, referenced at the top of all scripts as: "#/bin/csh". If /bin/csh is not available on your system, but /bin/tcsh is, you may need to either edit all run scripts to say /bin/tcsh at the top, or create a softlink in which /bin/csh -> /bin/tcsh. Python: We also recommend (if not require) python version 3.0 or later, along with select python libraries. Many of the helper scripts included in this package use python. The python scripts within this package reference '#!/usr/bin/env python'; you may need to change this on your computing platform. All of the scripts have been updated for Python 3.5 to match the configuration on EPA's cluster. You should still be able to use Python 2.7 if you install the "future" modules. In most cases that can be done by running: sudo pip install future == 3. Installation of data files and scripts == This readme covers the installation of the SMOKE inventories, scripts, and ancillary files used for the 2019 platform. Choose an install directory on your system; we will refer to this directory as "INSTALL_DIR". To review/reproduce emissions for all sectors, unzip all the .zip files into INSTALL_DIR. The packages have subdirectories embedded within them, so it is important that all files be unpacked in the same place in order for the scripts know where to find the inputs. If you are only interested in reproducing or examining emissions for specific sectors, you may download and unzip only the data for those sectors from the inventory directories, but for SMOKE processing you should also include the files in the ancillary_inputs and spatial_surrogates directories, and also the contents of the SMOKE core script package. Precompiled SMOKE executables and I/O API utilities are available in the SMOKE zip. If you would like to use an alternate set of SMOKE executables instead of the execs in this package, then in each sector run script in #INSTALL_DIR/[CASE]/scripts/*/, setenv SMOKE_LOCATION [location of your execs]. Similarly, to use an alternate set of I/O API utilities (e.g. m3xtract, m3tshift), setenv IOAPI_LOCATION [location of your execs]. All SMOKE inventories, scripts, and ancillary files used for the platform are provided. The full set of emission factor tables is not always permanently stored on the FTP server due to their large size; if they are not posted, they are furnished upon request. Prior to running SMOKE, you will need to edit the INSTALL_DIR (your install directory) and MET_ROOT (location of MCIP meteorology data) environment variable definitions in the "directory_definitions.csh" script located in each CASE/scripts directory. This script is sourced by each of the individual run scripts for each sector. Regarding the MCIP data, SMOKE only uses the GRIDCRO2D, METCRO2D, and METCRO3D files. If you are computing plume rise within SMOKE using the Laypoint program, SMOKE also needs the METDOT3D file. MCIP meteorology data is not included in the package. This is used for the onroad, onroad_ca_adj, dust (afdust/othafdust/othptdust), and biogenics (beis4) sectors. == 4. Case description and instructions for each sector == Scripts are provided to process emissions for a 12km national grid (12US1) using CB6R3AE7 speciation for CMAQ. The CB6R3AE7 mechanism includes new species IVOC, APIN, AACD, and FACD which were not in the previous version of CB6. These new species do not impact CAMx modeling. The emissions from this package will work with base CMAQ, but not the Multi-Pollutant version used for HAP-CAP modeling. The scripts can be adapted to run for other grids. First, in the directory_definitions.csh script, edit REGION_ABBREV and REGION_IOAPI_GRIDNAME. You also may need to change other inputs in each individual sector script, such as spatial surrogates (SRGPRO / SRGDESC), transportable fractions (XPORTFRAC), gridded meteorology for onroad (METMOVES), ocean chlorine (for the sector merge), and biogenic land use. Emissions for the platform may be processed for CMAQ modeling. This package does not include scripts for converting SMOKE outputs (CMAQ-ready emissions) to the format required by CAMx. Emissions processing is split into "sectors". Each sector has its own run scripts for processing, with one (or more) run scripts per case. See Section 7 of this README for information about the run script zips. All sectors are US-only unless otherwise noted. The sectors are: AFDUST: Particulate emissions from fugitive dust sources. This sector is processed in two steps. The first (Annual_afdust_12US1_*) processes the annual inventory, and the second (Annual_afdust_adj_12US1*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "afdust_adj". The afdust scripts must be run in that order. AIRPORTS: Emissions from airports. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are low-level only (no inline files). BEIS: Biogenic emissions generated using the BEIS model, version 3.6.1. CANADA_AG: Point source emissions from Canada agricultural sources. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are low-level only (no inline files). CANADA_OG2D: Point source emissions from low-level Canada upstream oil and gas sources. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are low-level only (no inline files). CMV_C1C2: Emissions from C1 and C2 commercial marine sources, including ports and navigable waterways. Includes C1/C2 marine emissions in the entire domain, including US, Canada, Mexico, and all bodies of water which lie outside the boundaries of those countries. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. Inventories for this sector are grid-specific and designed for the 12US1 grid, or 12km grids which are a subset of 12US1 (e.g. 12US2). Therefore, emissions are output under the sector name "cmv_c1c2_12". All emissions in this sector are elevated (no low-level contribution). CMV_C3: Emissions from C3 commercial marine sources, including ports and navigable waterways. Includes C3 marine emissions in the entire domain, including US, Canada, Mexico, and all bodies of water which lie outside the boundaries of those countries. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. Inventories for this sector are grid-specific and designed for the 12US1 grid, or 12km grids which are a subset of 12US1 (e.g. 12US2). Therefore, emissions are output under the sector name "cmv_c3_12". All emissions in this sector are elevated (no low-level contribution). LIVESTOCK: Agricultural emissions from livestock. This is mostly ammonia, but includes other pollutants from livestock sources as well. Note that agricultural emissions from fertilizer are not included in this package, because CMAQ was run with the bidirectional flux option, negating the need for SMOKE-processed NONPT: Area source emissions not included in other sectors. NONROAD: Off highway mobile source emissions. EPA processed this sector using a new SMOKE feature supporting output of multiple subsector emissions files in a single SMOKE run. Specifcially, SMOKE output diesel and gas (including all non-diesel fuels such as LPG and CNG) as separate inputs, in addition to the total onroad emissions. By default, the scripts included in this package will output separate emissions for gas and diesel (in addition to the total nonroad emissions). To turn off this option and have SMOKE only generate a single nonroad emissions file per day, set environment variable SMK_SUB_SECTOR_OUTPUT_YN to N. NP_OILGAS: Area source oil and gas emissions. NP_SOLVENTS: Area source emissions from solvents. ONROAD: On highway mobile source emissions, excluding California. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B. EPA processed this sector using a new SMOKE feature supporting output of multiple subsector emissions files in a single SMOKE run. Specifcially, SMOKE-MOVES output diesel and gas (including all non-diesel fuels such as E-85 By default, the scripts included in this package will output separate emissions for gas and diesel (in addition to the total sector emissions). To turn off this option and have SMOKE only generate a single emissions file per day, set environment variable SMK_SUB_SECTOR_OUTPUT_YN to N. ONROAD_CA_ADJ: On highway mobile source emissions, California only. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4B. EPA processed this sector using a new SMOKE feature supporting output of multiple subsector emissions files in a single SMOKE run. Specifcially, SMOKE-MOVES output diesel and gas (including all non-diesel fuels such as E-85 and CNG) as separate inputs, in addition to the total onroad emissions. By default, the scripts included in this package will output separate emissions for gas and diesel (in addition to the total sector emissions). To turn off this option and have SMOKE only generate a single emissions file per day, set environment variable SMK_SUB_SECTOR_OUTPUT_YN to N. OTHAFDUST: Particulate emissions from fugitive dust sources in Canada. Just like with afdust, this sector is processed in two steps. The first (Annual_othafdust_12US1_*) processes the annual inventory, and the second (Annual_othafdust_adj_12US1*) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "othafdust_adj". The othafdust scripts must be run in that order. Fugitive dust emissions in Mexico are included in the othar sector and do not need the same transportable fraction and meteorological adjustments that the Canada fugitive dust emissions in othafdust do. OTHPTDUST: Point source particulate emissions from fugitive dust sources in Canada. In Canada, dust emissions are in area source format for some sources (othafdust sector) and point source format for other sources (othptdust sector). This is a 'point' sector with additional adjustments, and is processed via THREE scripts: the 'onetime' script, the 'daily' script, and then the adjust script (othptdust_adj), in that order. All emissions in this sector are low-level only (no inline files). OTHAR: Area source emissions from Canada and Mexico, including mobile nonroad. ONROAD_CAN: Mobile onroad source emissions from Canada. ONROAD_MEX: Mobile onroad source emissions from Mexico. The onroad Mexico emissions inventory includes pre-speciated VOC emissions for an older CB6 mechanism, so there is an extra script for this sector to convert those emissions to the newer CB6 mechanism needed for CMAQ. This extra script is called *_part2_combine.csh and uses the combine utility to perform the CB6 mechanism conversion. The combine program is included, pre-compiled, in the SMOKE package along with pre-compiled SMOKE executables and I/O API utilities. To help make the distinction between emissions for the two CB6 mechanisms, older CB6 emissions use the sector name "onroad_mex_cb6orig". The combine post-processing step creates emissions files with the final sector name "onroad_mex". OTHPT: Point source emissions from Canada and Mexico. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). PTAGFIRE: Point source agricultural burning emissions. The ptagfire sector uses a daily point source inventory. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). PTEGU: Electric generating unit emissions. This sector incorporates CEM (Continuous Emissions Monitoring) hourly emissions for a majority of sources. This is a 'point' sector, and like all 'point' sectors, is processed via a 'onetime' script first, followed by a 'daily' script. For ptegu there are two 'daily' scripts for different months of the year: 'summer' (May through September), and 'winter' (October through April). For sources without hourly CEM emissions, summer and winter use different hourly temporalization, and so they are run with separate inputs. All emissions in this sector are elevated (no low-level contribution). PTNONIPM: Point source emissions from industrial activities. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). PTFIRE-RX: Point source emissions from year specific controlled (prescribed) burning. Fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and is processed via multiple scripts: the 'onetime' script, and then the 'daily' scripts. The 'onetime' script must be run first. The two daily scripts, "lowactivity" and "highactivity", are run for different months of the year, and both must be run. PTFIRE-WILD: Point source emissions from year specific wild fires. Fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and is processed via multiple scripts: the 'onetime' script, and then the 'daily' scripts. The 'onetime' script must be run first. The two daily scripts, "lowactivity" and "highactivity", are run for different months of the year, and both must be run. PTFIRE_OTHNA: Point source emissions from year specific controlled burning and wild fires in the rest of North America ('OTHNA' = OTHer North America), including Canada and Mexico. In addition to Canada and Mexico, fire emissions for Central America and the Caribbean are also included. Emissions from those areas are ultimately not modeled due to being outside of the 12US1 modeling domain, but they are provided for possible use in larger grids. These fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PT_OILGAS: Point source oil and gas emissions, including emissions from offshore oil rigs in the Gulf of Mexico. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). RAIL: Area source railway emissions. RWC: Area source residential wood combustion emissions. == 4B. Notes regarding onroad == Onroad emissions are processed using SMOKE-MOVES. The processing is split into multiple run scripts. In this platform, SMOKE-MOVES inputs were prepared using MOVES3. As in recent platforms, speciation of VOC emissions is handled within the MOVES model. This platform uses MOVES version MOVES3, which supports speciation for the CB6R3AE7 mechanism. As described in the SMOKE online documentation, SMOKE-MOVES handles onroad emissions separately for six types of processes: - On-network emissions (RatePerDistance, or RPD) - Off-network emissions, fuel vapor venting (RatePerProfile, or RPP) - Off-network emissions, extended idling (RatePerHour, or RPH) - On-network idling (ONI) emissions (RatePerHour-ONI, or RPHO) - Off-network emissions, starts (RatePerStart, or RPS) - All other evaporative off-network emissions (RatePerVehicle, or RPV) For each of the two onroad sectors (onroad, onroad_ca_adj), there are separate run scripts for RPD, RPP, RPH, RPHO, RPS, and RPV, plus a merge script that combines emissions from the six streams into a single emissions file per day. These scripts may take a particularly long time to run, especially RPD. Therefore, consideration should be given to running multiple RPD jobs in parallel, such as one job per quarter. The reason onroad has been split into two sectors - onroad and onroad_ca_adj - is in order to match SMOKE-MOVES annual emission totals with those provided by California. To do this, we split California into a separate sector, and run SMOKE-MOVES with a control factor file (CFPRO) which nudges the emissions so that the annual totals post-SMOKE-MOVES equal those provided by CARB. CARB provided separate onroad emissions inventories for use in the beta platform, and we match their provided emissions at the county/SCC level, except that the CARB inventory does not distinguish between different on-network road types (but does distinguish on-network emissions from off-network). == 4B1. DAYS_PER_RUN == SMOKE-MOVES can be run more efficiently if running more than one day at a time. For example, Movesmrg can create one 7-day emissions file more quickly than it can create seven individual 1-day emissions files. To turn on this feature, use the DAYS_PER_RUN variable, set to the number of days you wish to run in a single Movesmrg instance. The recommended value for DAYS_PER_RUN is 7. The onroad scripts include a setting called "DAYS_PER_RUN", set to 1 as the default. If DAYS_PER_RUN > 1, after Movesmrg is run, the run scripts will use the I/O API utility m3xtract to split up the multi-day emissions file into single day (25-hour) emissions files. Multi-day Movesmrg runs will never cross months. For example, if DAYS_PER_RUN = 7, then the last Movesmrg run of January will start on January 29th and end on January 31st (3 days), and the first Movesmrg run of February will start on February 1st and end on February 7th. Using the multi-day Movesmrg functionality requires multi-day MCIP files. For example, if DAYS_PER_RUN = 7, your METCRO2D files must also be 7 days (169 hours) long. These multi-day MCIP files should be stored in MET_ROOT_${Xday}/, where X = DAYS_PER_RUN (i.e. /7day for DAYS_PER_RUN = 7). For example, if the single day MCIP files are in /foo/foo/mcip_dir/, then 7-day MCIP files should be in /foo/foo/mcip_dir_7day/. The primary drawback to using this multi-day Movesmrg functionality is an increase in the memory usage. == 4B2. NMOG == In sectors other than onroad, an additional species called NMOG - non-methane organic gases; equivalent to TOG minus methane - is included in the speciation profiles and CMAQ-ready emissions. Onroad emissions from SMOKE-MOVES do not include NMOG. NMOG is not needed for CMAQ and is optional. For those who want to create NMOG emissions for onroad, an extra script is provided to be run after the onroad merge step. For example: Monthly_onroad_diesel_add_NMOG_12US1_2019ge_cb6_19k.csh This script uses the combine utility to add the NMOG species to the emissions. Emissions output with this script have the sector names "onroad_w_nmog" and "onroad_ca_adj_w_nmog", respectively. Edit the sector names in the SECTORLIST prior to running the overall sector merge (section 4C) to include NMOG-supplemented onroad emissions in your final merged model-ready emissions. == 4C. Sector merge == After all sectors have been processed, the Sector_merge script merges the low-level emissions from all sectors into a single CMAQ-ready emissions file per day. Merged model-ready emissions will be output to: INSTALL_DIR/$CASE/smoke_out/$CASE/$GRID/$SPC/merged_nobeis_norwc/ Inline emissions and stack_groups files will be output to the same directory, except in subdirectories by sector name (e.g. .../$SPC/ptnonipm/). By default, the sector merge scripts are configured to exclude biogenics and RWC. Some CMAQ modelers may wish to include biogenic emissions in the gridded emissions files and not have CMAQ compute biogenics inline. The newest versions of CMAQ include features which require the RWC sector emissions to be passed into CMAQ separately. So, both the output directory and the individual filenames of the merged emissions indicate whether beis and RWC are included. By default they are both excluded so the files are output to a directory called "merged_nobeis_norwc" and each filename says "nobeis_norwc". The sectorlist file controls which sectors are included in the sector merge. This file is included in the package along with the run scripts, and is located in the $CASE/scripts directory for each case. Within the sectorlist file, the 'mergesector' column controls which sectors are merged (Y or N). For example, to run the sector merge with biogenics or RWC, set the mergesector column to Y for the 'beis' and 'rwc' sectors. Many point sectors have mergesector set to N (e.g. ptegu, othpt) because they do not have any 2-D gridded emissions. To merge in alternative biogenic emissions files, edit the sectorlist by changing the 'beis' sector name to the sector name of your choice, and make sure your biogenic emissions files exist in the $CASE/premerged/[sector name] directory with filenames adhering to the file name convention used by other sectors. == 5. Description of inventory packages == Inventories for the 2019ge case are included in the following files, all of which should be unpackaged in INSTALL_DIR. 2019ge_CMV_c1c2_inventory_16mar2023.zip 2019ge_CMV_c3_inventory_16mar2023.zip 2019ge_cem_inventory_10mar2023.zip 2019ge_fires_inventory_10mar2023.zip 2019ge_nonpoint_inventory_10mar2023.zip 2019ge_nonroad_inventory_10mar2023.zip 2019ge_onroad_inventory_10mar2023.zip 2019ge_oth_inventory_17mar2023.zip 2019ge_point_inventory_10mar2023.zip 2019ge_onroad_SMOKE-MOVES_MOVES3_emissions_FF10_allHAPs_30nov2021_v0.zip Past platforms also included a biogenics package. The files from that package are now in the ancillary_data directory. The "cem" package includes the hourly CEM (Continuous Emissions Monitoring) emissions used by the ptegu sector. This is the same data that is available on EPA's Air Markets Program Data website (ampd.epa.gov), except that we've split the data into months and days as needed for our scripts, and run the data through a "CEM correct" program. The CEMSUM file is in the ancillary_data area. The "CMV" package includes annual and hourly inventories for the cmv_c1c2 and cmv_c3 sectors, 12US1 grid. The "fires" package includes the inventories for the ptfire-rx, ptfire-wild, ptfire_othna, and ptagfire sectors. The "nonpoint" package includes inventories for the following sectors: afdust, livestock, nonpt, np_oilgas, rail, rwc, np_solvents. Agricultural fire emissions also come from NEI Nonpoint, but those inventories are included in the fires zip. The "nonroad" package include the inventories for the nonroad sector. The "onroad_activity" package includes the activity data for the onroad and onroad_ca_adj sectors. It does not include the emission factor tables also required to run SMOKE-MOVES; those are in separate zips posted in the 2019emissions/moves_eftables/ area. Emission factors are only available for CB6R3AE7. The emission factor tables are provided on FTP with separate zips for each state, e.g. moves_eftables/2019ge_cb6_19k_eftables_cb6_st49.zip All of these EF tables already have a NOx humidity adjustment. The "oth" package includes all inventories for Canada and Mexico, except fires. The Mexico point source inventory is included in this package, but for proprietary data sharing reasons, does not include any facility names. The "point" package includes the inventories for the following sectors: ptnonipm, ptegu, pt_oilgas, airports. The "onroad_SMOKE_MOVES_emissions_ff10" package includes an FF10-formatted inventory representing emissions totals for the onroad sector as calculated by SMOKE-MOVES. This is provided for those who are interested in an onroad emissions inventory or report. It is not needed for SMOKE modeling. This FF10 contains HAPs which are included in HAP-CAP modeling as well as CAPs. See Section 4 of this README for a description of each modeling sector. == 6. Description of ancillary file packages == The following packages should be unpacked in INSTALL_DIR: ancillary_data/2019ge_beis_ge_dat_16mar2023.zip ancillary_data/2019ge_gridding_ge_dat_10mar2023.zip ancillary_data/2019ge_other_ge_dat_10mar2023.zip ancillary_data/2019ge_speciation_ge_dat_10mar2023.zip ancillary_data/2019ge_temporal_ge_dat_10mar2023.zip ancillary_data/ocean_chlorine.zip spatial_surrogates/CONUS12_2015CAN_2010MEX_surrogates_03dec2018.zip spatial_surrogates/CONUS12_2017NEI_with_2019_oilgas_04mar2021.zip The "beis" package includes gridded land use and biogenic emission factor files for input to BEIS version 3.6.1. This package also includes normalized emissions files for 12US1. The 12US1 file is extracted to the [CASE]/intermed/beis/ directory so that it can be used by the beis run script ([CASE]/scripts/biogenics/) to run BEIS. By default, the Normbeis3 program is turned off / not executed by the run script. These normalized emissions files are not speciation-specific and can be used with any speciation mechanism. The "surrogates" packages contain the spatial surrogates at 12km resolution for the US, and for Canada and Mexico. 12km surrogates for the np_oilgas sector for the year 2019 are included in a subdirectory within this zip. The oil and gas surrogates include a set of surrogates specifically for the WRAP region. The "gridding" package includes all SMOKE inputs related to spatial allocation other than the surrogates, including cross-references, surrogate descriptions, and gridded transportable fractions used in afdust_adj, othafdust_adj, and othptdust_adj. The "speciation" package includes speciation profiles, cross-references, VOC-to-TOG conversion factors, and inventory tables. This .zip includes files for the CB6R3AE7 mechanism for CMAQ v5.3. The "temporal" package includes temporal profiles and cross-references, including daily and hourly temporal profiles developed by the SMOKE program Gentpro for use in the rwc and livestock sectors. The "other" ge_dat package includes all other SMOKE ancillary files not included in the above packages, including: - SMOKE-MOVES ancillary files, including the reference county (MCXREF) and fuel month (MFMREF) cross-references, pollutant (MEPROC) and emission factor table (MRCLIST) lists, activity SCC to full SCC cross-references (SCCXREF), hourly speed profiles (SPDPRO), daily temperature data (METMOVES), and Movesmrg adjustment factors (CFPRO) - NHAPEXCLUDE files (concerns VOC HAP integration) - Other miscellaneous SMOKE inputs, such as the ARTOPNT, COSTCY, HOLIDAYS, MACTDESC, NAICSDESC, ORISDESC, PELVCONFIG, PSTK, SCCDESC - Smkreport configuration files (REPCONFIG, all in ge_dat/smkreport/repconfig) This ocean_chlorine.zip package contains gridded ocean chlorine emissions, which are included in the sector merge, for 12US1, 12US2, and 36US1, as well as a national-scale 4km domain. The run scripts (see section 7) are already set up to use the proper ancillary files and inventories for each sector and case. == 7. Description of script packages == The SMOKE core script package should be unpacked in INSTALL_DIR. This includes scripts and precompiled executables for running SMOKE in general, and for running the 2019 modeling case in particular. The scripts in the 2019ge_cb6_19k subdirectory are the scripts you run directly in order to replicate our emissions. Separate script(s) are provided for each sector. See section 4 for information pertinent to each sector. In general, you edit the directory_definitions.csh file, in particular INSTALL_DIR and MET_ROOT, and then run each sector. Sector scripts are organized into subdirectories within CASE/scripts by sector category: biogenics, nonpoint, onroad, point, and merge. For afdust sectors, run afdust/othafdust first, then afdust_adj/othafdust_adj. For point sectors, run "onetime" first, and then "daily". Othptdust has an additional "adj" script which comes after the daily script. For onroad sectors, run RPV/RPD/RPH/RPHO/RPP/RPS first, then the merge script. The onroad NMOG script is optional (see section 4B2). For onroad_mex, run the normal script and also the part2_combine script after that. The scripts, programs, and other inputs in the combine, ioapi, and smoke4.8.1 subdirectories are all "helper" scripts and inputs, and generally never need to be run directly. == 7B. Other miscellaneous programs == The beta platform public package includes additional SMOKE utilities in the package smoke_2016beta_platform_utilities.zip. These utilities have not been updated since beta platform, and so they have not been reposted for this platform. Additional information can be found in the README for beta platform. == 8. Preparing emissions for CAMx == CAMx modeling has not been performed for the 2019 platform, so support for CAMx emissions processing is not included in this package. == 9. Log analyzer == The platform scripts include a Python-based tool called the log analyzer that runs automatically at the conclusion of each SMOKE job. The purpose of the log analyzer is to scan all log files from the sector, search for errors and warnings, and filter out common or "acceptable" warnings. Log analyzer output goes into this directory: $INSTALL_DIR/$CASE/reports/log_analyzer There are two types of output files: Level 1 and Level 3. Level 3 lists every instance of each error and warning individually, while Level 1 combines repeats of common warnings. It is sufficient to look at only the Level 1 output. The Level 1 output includes a "priority" code, the error/warning message, and the log file in which the message appears. The priority code is a number from 0 to 3. If priority = 2 or 3, the message has been identified as common or acceptable. This is based on a file called known_messages.txt, which is located here: $INSTALL_DIR/smoke4.9/scripts/log_analyzer/known_messages.txt If priority = 1, the message is included in known_messages and has been identified as NOT acceptable - however, many priority 1 warnings are in fact SOMETIMES acceptable and sometimes not, which is why they are not given priority 2 or 3. Some common priority 1 warnings are listed below. If priority = 0, the message is not included in known_messages. Priority 0 or priority 1 messages which are acceptable include: - WARNING: [*] is not found in both inventory pollutant and model species lists. These are common onroad messages resulting from the CFPRO file including more species than is always necessary. - WARNING: Could not read [BEGHOUR/ENDHOUR] from file "PDAY" This is a common ptfire warning and is acceptable. - WARNING: resetting surrogates ratio of Co/St/Ct (FIPS): ... This warning is in known_messages but is not always picked up by the log analyzer for reasons unknown. - WARNING: Speciation profile "?? " is not in profiles This warning is common/acceptable for onroad_mex. - WARNING: Total annual toxic emissions greater than annual [*]__VOC emissions for source: This warning is common for sectors with monthly inventories such as nonroad. - WARNING: Applying default time zone 5 to country/state/county code: [*] This warning is acceptable for Alaska FIPS and FIPS 85005. - WARNING: Duplicate entry in AR2PT x-ref file: This warning is acceptable but is not in known_messages to let us know we should correct it in the future. - WARNING: Hour-specific ending date/time WARNING: Could not read "INDXH" from file "PHOUR " netCDF error number -40 Error reading netCDF time step flag for PHOUR These four messages may appear in the Temporal logs for January and February for the CMV sectors. - WARNING: Dropping SCC .* not listed in SCCXREF file This is an acceptable warning for the RPP process in onroad. In other platforms which use MOVES3, this warning is also acceptable for RPV. (Platforms which uses MOVES3 also have an "RPS" process.) == 10. SMOKE reports (Smkreport) == By default, the run scripts run the Smkreport program, output to the $CASE/reports/inv directory. For sectors with annual inventories, reports are annual. For sectors with monthly inventories (e.g. nonroad), reports are monthly. Most reports include all inventory pollutants and model species, although PM10 usually appears as zero due to a SMOKE quirk; to get PM10, sum PM2_5 and PMC in the report. For onroad, these reports reflect activity, not emissions, and include some double counting due to how SMOKE allocates activity to different processes; therefore, you should not use the reports/inv reports for the onroad or onroad_ca_adj sectors. The following types of reports are generated. Note that not all types of reports are generated for all sectors: *state.txt: State totals. *county.txt: County totals. *state_scc.txt: State/SCC totals. *county_scc.txt: County/SCC totals. *state_naics.txt: State/NAICS totals. *cell_${GRID}.txt: Totals by grid cell. *cell_county_${GRID}.txt: Totals by grid cell and county. *state_grid_${GRID}.txt: State totals after gridding. *srgid_${GRID}.txt: Emissions totals at various resolutions after gridding, and also including the spatial surrogate assignment. *pm25prof.txt: Totals of PM2.5 at various resolutions, and also including the PM2.5 speciation profile assignment. *vocprof.txt: Totals of VOC at various resolutions, and also including the VOC speciation profile assignment. For sectors which are integrated and have both NONHAPVOC and VOC, or have multiple modes of VOC, there may be multiple VOC profile reports for NONHAPVOC and (no-integrate) VOC and/or for each mode. For all sectors except those processed with SMOKE-MOVES, Smkmerge generates daily county total reports in the $CASE/reports/smkmerge directory. These are then summed to annual by state and county, output to the $CASE/reports/annual_report directory. Emissions totals in the reports/annual_report directory (post-temporalization) should be within 1-2% of the totals in the reports/inv directory (pre-temporalization). For SMOKE-MOVES sectors, the $CASE/reports/smkmerge directory includes daily (or weekly if DAYS_PER_RUN=7) totals by county/SCC. Scripts to aggregate these totals to monthly or annual by state, county, state/SCC, and county/SCC are provided in the SMOKE utilities zip, movesmrg_report_postproc/ directory. == 11. Spinup == A parameter in the run scripts called SPINUP_DURATION supports the processing of emissions for December of the previous year for model spinup purposes. If SPINUP_DURATION > 0, then: - For sectors where emissions are processed for representative dates only, the entire month of December of the previous year will be processed - For sectors where emissions are processed daily, the last X days of December of the previous year will be processed, where X = SPINUP_DURATION However, in EPA emissions modeling platform applications, SPINUP_DURATION is set to a nonzero value (typically 10) ONLY for biogenics processing and for the final sector merge. For all other sectors, SPINUP_DURATION is set to 0 when running SMOKE. The scripts in this package are configured accordingly as the default. When the final sector merge is run, spinup period emissions for all sectors except biogenics are taken from December of the base year. For representative day sectors, dates are mapped based on day-of-week, and also Christmas holidays; for daily sectors, calendar dates are mapped. Using 2016 as an example, the 12/28/2015 merged emissions include sectors from the following dates: - beis: 12/28/2015 (actual spinup emissions) - afdust_adj, fertilizer, livestock, onroad, onroad_ca_adj, othafdust_adj, othptdust_adj, rwc: 12/28/2016 (daily sectors use December 2016 matched up by calendar day) - airports, canada_ag, canada_og2D, nonpt, nonroad, onroad_can, onroad_mex, othar, pt_oilgas, ptnonipm: 12/05/2016 (representative dates based on day-of-week) - np_oilgas, np_solvents, rail: 12/06/2016 (also representative dates, but these sectors have a single day per month) An illustration of how the dates are mapped in the spinup period is available in the smk_merge_dates files, for example: $INSTALL_DIR/smoke4.8.1/scripts/smk_dates/2019/smk_merge_dates_201812.txt Similarly for point sectors, when CMAQ is run, inline emissions from the base year are used to cover the spinup period as follows: - othpt, ptnonipm, pt_oilgas use representative date / day-of-week based mappings - cmv_c1c2, cmv_c3, ptegu use calendar date-based mappings - ptfire-rx, ptfire-wild, ptagfire, ptfire_othna (all fires) use January 1 emissions for all spinup days Sometimes, base year inline point emissions files are physically copied to December spinup dates to simplify the CMAQ setup, but CMAQ scripts can also be configured to point to base year dates while running spinup.