*** PLEASE READ THIS BEFORE RUNNING SMOKE. *** == 1. Introduction == The packages labeled 2018ed contain scripts, inventories, and ancillary files related to the 2018ed_v6_11f case, which is the basis for EPA's 2018 ozone transport rule emissions. This platform will be used for assessing ozone transport within the continential U.S. A federal register notice has been developed to announce the availability of this data, and a docket (#EPA-HQ-OAR-2013-0809) has been established to accept comments on the data. Please plan to submit any comments on the data as described in the notice so they can be properly tracked and so that proposed adjustments can be evaluated for inclusion in future versions of the data files. CMAQ and CAMx model-ready emissions generated using these packages with SMOKE v3.5.1 should be identical to those used in EPA's 2018ed case under the 2011v6 platform. However, there may be emissions differences resulting from differences in the Linux operating system, hardware platform, and other system-specific differences. Additional information on the 2011v6 platform is available here: http://www.epa.gov/ttn/chief/emch/index.html#2011 The data files are divided into several directories: - 2018emissions contains emission inventories for the year 2018, including national CEMS emissions, onroad and nonroad inventories in separate files and some other data that is specific to 2018; most point and nonpoint inventories are in 2018ed_v6_11f_inputs.zip - 2018emissions/onroad_by_state has state-specific onroad emissions output from SMOKE - 2018emissions/nonroad_by_state has state-specific nonroad emission inventories input to SMOKE - 2018emissions/point_by_state has state-specific point source emission inventories input to SMOKE and some inventory-level data output from SMOKE - 2018emissions/nonpoint_by_state has state-specific nonpoint source emission inventories input to SMOKE - 2018emissions/moves_eftables contains MOVES emission factor tables input to SMOKE. Note that these are quite large, so only need to be downloaded if you plan to do SMOKE-MOVES modeling - data_common_to_all_years contains general ancillary files (ge_dat) and ocean_chlorine files that don't change across years - reports contains reports and summaries for 2018 - smoke_scripts_utils contains scripts to run SMOKE and utility programs - spatial_surrogates contains 12km and 36km spatial surrogates Further information about the provided data and reports be found in in the document DetailsAbout2018DataFiles01032014.pdf. Section 4 of this README includes information about the modeling sectors used in the platform. Section 5 of this README includes information about the inventories provided. Section 6 of this README includes information about the ancillary (non-inventory) files included. == 2. Requirements for processing emissions for air quality modeling == If you are only reviewing inventories and not developing emissions for air quality modeling, you do not need to install SMOKE or to follow the instructions below. Instead, unzip the files with the data of interest and examine those and the corresponding reports that are provided. If you plan to develop emissions for air quality modeling, SMOKE v3.5.1 is recommended to process this case. Some bug fixes and new features were implemented between SMOKE v3.1 and v3.5.1 that are needed by this case, so we recommend that this case be processed with SMOKE v3.5.1. Python: We also recommend (if not require) python version 2.6 or later, along with select python libraries. Many of the helper scripts included in this package use python. The python scripts within this package reference '#!/usr/bin/env python' or '#!/usr/bin/env python2.6'; you may need to change this on your computing platform. == 3. Installation of data files and scripts == This readme covers the installation of the SMOKE inventories, scripts, and ancillary files used for the 2018ed_v6_11f case in the 2011v6 platform. Some ancillary files for the 2018ed_v6_11f case may come from 2011ec_v6_11f and 2011ed_v6_11f packages within the 2011v6 platform. Therefore, if you are planning to process emissions for 2018, we recommend that you also install the 2011 platform. Choose an install directory on your system; we will refer to this directory as "INSTALL_DIR". To review/reproduce emissions for all sectors, unzip/untar all the .zip and .tar.gz files into INSTALL_DIR. The packages have subdirectories embedded within them, so it is important that all files be unpacked in the same place in order for the scripts to run. If you are only interested in reproducing or examining emissions for specific sectors, you may download and unzip only the data for those sectors from the 2018emissions directory, but you should also include the files in the directories: data_common_to_all_years, smoke_scripts_utils, and the spatial_surrogates for the grid(s) you plan to run. All SMOKE inventories, scripts, and ancillary files used for the 2018ed_v6_11f case are provided. The following are NOT included in the packages: - SMOKE executables: These can be downloaded from http://www.smoke-model.org/. Again, we recommend SMOKE v3.5.1. The SMOKE executables should be installed in INSTALL_DIR/smoke3.5/[platform], where [platform] varies depending on your computing platform. On EPA systems, [platform] = "Linux2_x86_64ifort". If you are not sure what [platform] should be, running a script once should produce an error message that will tell you in which directory the scripts expect to find the SMOKE executables. - I/O API utilities, such as m3xtract and juldate. For the scripts to run, these should be copied or linked to INSTALL_DIR/ioapi/[platform] (see above for a description of [platform]). - MCIP meteorology data, which is used for the afdust, onroad, onroad_catx_adj, onroad_rfl, biogenics (beis), and ptfire3D sectors. See MET_ROOT instructions below. The sea salt portion of the CAMx conversion (see section 9) requires CAMx-formatted WRF output, which is also not included in the package. Prior to running SMOKE, in each run script, you will need to edit the INSTALL_DIR (your install directory) and MET_ROOT (location of MCIP meteorology data) environment variable definitions at the top of the script. The MCIP data for each grid (12US2, 36US1) should be located in MET_ROOT/[grid]/mcip_out/. SMOKE only uses the GRIDCRO2D, METCRO2D, and METCRO3D files. In the ptegu and ptegu_pk run scripts, you will also need to edit NAMEBREAK_HOURLY so that it equals the number of underscores in your INSTALL_DIR (the full path), plus 4. Example: if INSTALL_DIR = '/disk/your/home_directory/foo_foo/install_dir', then set NAMEBREAK_HOURLY = 7 (3+4). == 4. Case description and instructions for each sector == 2018ed_v6_11f emissions were processed for one grid: a 12 km national grid (12US2). Emissions were processed for two air quality models: CMAQ and CAMx. SMOKE outputs are "CMAQ-ready" as-is. After all CMAQ-ready emissions are processed, they are converted to CAMx by a standalone converter. This converter is included in the package and is described in section 9. Emissions processing is split into "sectors". Each sector has its own run scripts for processing, with one (or more) run scripts per modeling grid. (See section 7 for information about the run script zips.) All sectors are US-only unless otherwise noted. The sectors are: AFDUST: Particulate emissions from fugitive dust sources. This sector is processed in two steps. The first (Annual_afdust_12US2_2018ed_v6_11f.csh) processes the annual inventory, and the second (Annual_afdust_adjustment_12US2_2018ed_v6_11f.csh) applies adjustments - transportable fraction and meteorologically-based - and outputs the adjusted emissions under the sector name "afdust_adj". The 12US2 afdust scripts must be run in that order. AG: Agricultural ammonia emissions. BEIS: Biogenic emissions generated using the beis3 model, version 3.14. C1C2RAIL: Emissions from C1 and C2 commercial marine sources, including ports and navigable waterways. Includes offshore c1/c2 marine emissions in the Atlantic and Pacific Oceans, and the Gulf of Mexico. Railway emissions are also part of this sector. C3MARINE: Ocean going class 3 marine port and underway emissions within, or close to, the US mainland. (Offshore c3 emissions are in the othpt sector.) This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). NONPT: Area source emissions not included in other sectors. NONROAD: Off highway mobile source emissions. NP_OILGAS: Area source oil and gas emissions. ONROAD: On highway mobile source emissions, excluding refueling, and excluding California and Texas. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4A. ONROAD_CATX_ADJ: On highway mobile source emissions, California and Texas only, excluding refueling. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4A. ONROAD_RFL: On highway mobile source emissions, refueling only, entire country. This sector is processed using SMOKE-MOVES with multiple scripts as described in section 4A. OTHAR: Area source emissions from Canada and Mexico. OTHON: Mobile source emissions from Canada and Mexico. OTHPT: Point source emissions from Canada, Mexico, and offshore areas. Includes all offshore c3 commercial marine emissions, which were part of the c3marine sector in prior platforms (e.g. 2007v5). This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. All emissions in this sector are elevated (no low-level contribution). PTEGU: Electric generating unit emissions, excluding "peaking" units (see ptegu_pk sector). This sector incorporates CEM (Continuous Emissions Monitoring) hourly emissions for a majority of sources. This sector, combined with ptegu_pk, was called "ptipm" in prior platforms, such as 2007v5. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. You may need to change the NAMEBREAK_HOURLY setting in the run scripts, as described in the run script comments, and in section 3 of this readme. PTEGU_PK: Electric generating unit emissions, "peaking" units only. A "peaking" unit is one that only operates during times of peak demand, rather than for most or all of the year. Peaking units were defined by EPA's Clean Air Markets Division (CAMD). Peaking units are kept in a separate sector for the purposes of source apportionment in future modeling applications. This sector incorporates CEM (Continuous Emissions Monitoring) hourly emissions for all sources. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. You may need to change the NAMEBREAK_HOURLY setting in the run scripts, as described in the run script comments, and in section 3 of this readme. PTNONIPM: Point source emissions from industrial activities. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PTFIRE: Point source emissions from year specific controlled burning and wild fires. For this sector (as opposed to ptfire3D), fires are processed in the 'inline' format for CMAQ, and are all elevated (no low-level contribution). These inline fire emission files are only needed by CMAQ, not CAMx, so if you are not running CMAQ, you can omit this sector and only run ptfire3D. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PTFIRE3D: Same as ptfire, except instead of processing emissions as 'inline', 3-D layered emissions files are produced, using meteorology data and the SMOKE Laypoint program to calculate plume rise. Fire emissions are output into one of Z vertical layers, where Z = the number of layers in your MCIP data. The number of layers is defined in the run script by the parameter SMK_EMLAYS; for us, this was equal to 25. These 3-D ptfire emissions files are only needed by CAMx, not CMAQ, so if you are not running CAMx, you can omit this sector and only run ptfire. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. PT_OILGAS: Point source oil and gas emissions. This is a 'point' sector, and like all 'point' sectors, is processed via two scripts: the 'onetime' script, and the 'daily' script. The 'onetime' script must be run first. RWC: Area source residential wood combustion emissions. == 4A. Notes regarding onroad == Onroad emissions are processed using SMOKE-MOVES. The processing is split into multiple run scripts. As described in the SMOKE online documentation, SMOKE-MOVES handles onroad emissions separately for three types of processes: - On-network emissions (RatePerDistance, or RPD) - Off-network emissions, fuel vapor venting (RatePerProfile, or RPP) - Off-network emissions, non-venting (RatePerVehicle, or RPV) For each of the three onroad sectors (onroad, onroad_catx_adj, onroad_rfl), there are separate run scripts for RPD, RPP, and RPV, plus a merge script that combines emissions from RPD, RPP, and RPV into a single emissions file per day. Onroad_rfl does not have an RPP component, so the onroad_rfl merge combines RPD and RPV only. These scripts may take a particularly long time to run, especially RPD. Therefore, consideration should be given to running multiple RPD jobs in parallel, such as one job per quarter. The reasons onroad has been split into three sectors - onroad, onroad_rfl, and onroad_catx_adj - are: - For refueling emissions, we use a different gridding surrogate - 600, for gas stations - than for the non-refueling emissions. The only way to apply this in SMOKE is to split refueling emissions into their own sector, onroad_rfl, and give that sector a different gridding cross-reference which assigns all emissions to gridding surrogate 600. - As part of the 2011v6 platform, we wished to match SMOKE-MOVES annual emission totals with those provided by California and Texas. To do this, we split CA and TX into a separate sector, and ran SMOKE-MOVES with a control factor file (CFPRO) which nudges the emissions so that the annual totals post-SMOKE-MOVES equal those provided by the states, at the county/SCC3 level in California, and at the county/SCC7 level in Texas. Essentially, this method uses overall emissions totals provided by the state agencies, combined with SMOKE-MOVES temporalization. Onroad_catx_adj does not contain any refueling emissions, because refueling emission estimates are typically not provided in state submittals of onroad emission inventories. The onroad_rfl sector covers all states and is not "nudged". == 4B. Sector merge == After all sectors have been processed, the Sector_merge script merges the low-level emissions from all sectors into a single CMAQ-ready emissions file per day. Merged model-ready emissions will be output to: INSTALL_DIR/2018ed_v6_11f/smoke_out/2018ed_v6_11f/[grid]/cmaq_cb05_soa/ Inline emissions files will be output to the same directory, except in subdirectories by sector name (e.g. .../cmaq_cb05_soa/ptnonipm/). For the CAMx conversion, see section 9. == 5. Description of inventory packages == Inventories for 2018ed_v6_11f are included in the following files, all of which should be unpackaged in INSTALL_DIR. Files for processing onroad emissions only (these files are many GB and only need to be downloaded if you are planning to run SMOKE-MOVES): 2018emissions/moves_eftables/2018ed_RPD_eftables_1_1.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_1_2.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_1_3.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_1_4.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_1_5.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_7_1.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_7_2.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_7_3.tar.gz 2018emissions/moves_eftables/2018ed_RPD_eftables_7_4.tar.gz 2018emissions/moves_eftables/2018ed_RPD_rfl_eftables_1.tar.gz 2018emissions/moves_eftables/2018ed_RPD_rfl_eftables_7.tar.gz 2018emissions/moves_eftables/2018ed_RPP_eftables_1.tar.gz 2018emissions/moves_eftables/2018ed_RPP_eftables_7.tar.gz 2018emissions/moves_eftables/2018ed_RPV_eftables_1.tar.gz 2018emissions/moves_eftables/2018ed_RPV_eftables_7.tar.gz 2018emissions/moves_eftables/2018ed_RPV_rfl_eftables_1.tar.gz 2018emissions/moves_eftables/2018ed_RPV_rfl_eftables_7.tar.gz 2018emissions/2018ed_v6_11f_inputs_onroad_ff10.tar.gz* (* - not required for SMOKE-MOVES processing, as described below) Inventory files for sectors other than onroad: 2018emissions/2018ed_v6_11f_cem.zip 2018emissions/2018ed_v6_11f_inputs_nonroad.zip 2018emissions/2018ed_v6_11f_inputs_oth.zip 2018emissions/2018ed_v6_11f_inputs.zip The RPD, RPP, and RPV packages include the emission factor tables used by SMOKE-MOVES for the onroad sector. These files are large, so they were split up into multiple packages. The "inputs_onroad_ff10" package contains an FF10-formatted inventory summarizing the post-SMOKE-MOVES emissions (including California and Texas adjustments) at the county/SCC/month level. It is not needed in order to process SMOKE-MOVES, and is provided mainly for the purposes of generating reports. Alternatively, one could process this FF10 through "traditional" SMOKE in place of running SMOKE-MOVES. While running traditional SMOKE using this FF10 would produce the same monthly and annual emissions totals as SMOKE-MOVES, it would not replicate the gridding or temporal distributions that come out of SMOKE-MOVES. The "cem" package includes the hourly CEM (Continuous Emissions Monitoring) emissions used by the ptegu and ptegu_pk sectors. These data are based on the hourly temporal patterns derived from the 2011 hourly CEM data available on EPA's Clean Air Markets Program Data website (ampd.epa.gov). Seasonal (winter/summer) to hour temporal factors based on 2011 CEM sources were applied to the seasonal 2018 IPM source emissions, where ORIS Facility and Boiler IDs matched. The "inputs_nonroad" package includes the inventory for the nonroad sector. It was given its own package due to the large size of the inventory. The "inputs_oth" package includes the inventories for Canada, Mexico, and offshore emissions. These are used by the othar, othon, and othpt sectors. All other emissions inventories are included in the "inputs.zip" package. The three "inputs" packages also include sample list (.lst) files, which list the inventories used by each sector. The run scripts automatically create new .lst files with the correct paths, so there is no need to edit these unless you are running SMOKE with your own scripts. See section 4 for a description of each modeling sector. == 6. Description of ancillary file packages == The following packages should be unpacked in INSTALL_DIR: 12US1_CAN2006_MEX2010v3_surrogates.zip CONUS12_2010_v3_20130808_surrogates.zip ge_dat_for_2011ec.zip ge_dat_for_2018ed.zip The two "surrogates" packages contain the gridding surrogates for 36US1 and Canada/Mexico. The "ge_dat" packages contains all other ancillary files used by SMOKE and/or the run scripts. The 2011ec ge_dat packages provides the base 2011v6 platform ancillary files, while the 2018ed ge_dat package provides supplemental ancillary files specific to the 2018 emissions. Both packages should be extracted. These packages include the following: - Speciation profiles (GSPRO), cross-reference files (GSREF, GSPRO_COMBO), and VOC-to-TOG conversion factors (GSCNV) - Gridding cross-reference (GREF) files - Temporal profile (TPRO) and cross-reference (TREF) files - Lists of sources to exclude from CAP/HAP integration (NHAPEXCLUDE) - List of sectors to include in the sector merge (SECTORLIST) - Gridding surrogate descriptions (SRGDESC) - Onroad control factor files (CFPRO) for extended idle, and for California and Texas - Onroad reference county (MCXREF) and fuel month (MFMREF) cross-references - Onroad pollutant (MEPROC) and emission factor table (MRCLIST) lists - Hourly average speed data for onroad (SPDPRO) - Gridded transportable fractions for afdust (XPORTFRAC) - Daily temporal profiles for rwc, produced by Gentpro (TPRO_DAY) - Hourly temporal profiles for ag, produced by Gentpro (TPRO_HOUR) - Biogenic emission factors (B3FAC) - Gridding surrogates for biogenic reports (BGPRO) - Other miscellaneous SMOKE inputs, such as the ARTOPNT, COSTCY, GRIDDESC, HOLIDAYS, INVTABLE, MACTDESC, MCODES, NAICSDESC, ORISDESC, PELVCONFIG, PSTK, SCCDESC - Smkreport configuration files (REPCONFIG, all in ge_dat/repconfig/default) The run scripts (see section 7) are already set up to use the proper ancillary files and inventories for each sector and grid. == 7. Description of script packages == These two run script packages should be unpacked in INSTALL_DIR: smoke_scripts_utils/smoke_helper_scripts_2011platform.zip smoke_scripts_utils/2018ed_v6_11f_scripts.zip The helper_scripts package includes scripts which are called by the primary run scripts. You should not have to run any of these scripts directly. The 2018ed scripts package includes the scripts that you run directly in order to replicate our emissions. Separate script(s) are provided for each grid and sector. The scripts are unpacked to INSTALL_DIR/2018ed_v6_11f/scripts/. See section 4 for information pertinent to each sector. In general, you edit the INSTALL_DIR and MET_ROOT, and then run. For point sectors, run "onetime" first, and then "daily". For onroad sectors, run RPV/RPD/RPP first, then the merge. For CAMx conversion scripts, see section 9. == 8. Description of other packages == ocean_chlorine.zip: This package contains gridded ocean chlorine emissions, which are included in the sector merge. Unzip in INSTALL_DIR. == 9. CAMx conversion == The package smoke_scripts_utils/2018ed_v6_11f_camx_conversion.zip contains all scripts and programs needed in order to convert the CMAQ-ready emissions (SMOKE outputs) into emissions inputs for the CAMx air quality model. The primary run scripts are: - 2018ed_v6_11f/scripts/camx/camx_convert_2018ed_[grid].csh: Converts SMOKE outputs to CAMx format. The relevant SMOKE outputs are the merged low-level emissions, the inline point emissions, and the layered ptfire3D sector emissions. This program creates two outputs per day: an emis2d file (low-level emissions), and a mrgpt file (elevated emissions, including ptfire3D). - 2018ed_v6_11f/scripts/camx/seasalt_2018ed_[grid].csh: Adds sea salt emissions to the emis2d files (CAMx-format low level emissions). All CAMx-ready files will go to: INSTALL_DIR/2018ed_v6_11f/smoke_out/2018ed_v6_11f/[grid]/camx/ == 9A. FORTRAN programs used by the CAMx converter == The CAMx converter uses several FORTRAN programs. The camx_conversion.zip contains both our compiled FORTRAN executables (which may or may not work on your system), and the original FORTRAN source code and Makefiles, so that you may recompile these helper programs yourself. The source code is in 2018ed_v6_11f/scripts/camx/src/. Also note the subdirectories. Our executables are in 2018ed_v6_11f/scripts/camx/bin/. If you have to recompile the code, this is where the recompiled executables should ultimately go. For the sea salt script, the executables and source code are both in 2018ed_v6_11f/scripts/camx/seasalt/. The relevant programs are seasalt.lnx and mrgspc.lnx. == 9B. Other files used by the CAMx converter == The subdirectories 'dates' and 'Species_Mapping_Tables' include other files used by the converter. The sea salt script requires CAMx-formatted meteorological data from WRF. Specifically, the script uses a 'met3d' file, 'met2d' file, and 'landuse' file. Like the MCIP meteorology, these files are not included in the package.