Last update: 12 Jan 2021 *** PLEASE READ THIS README BEFORE RUNNING SMOKE. *** This package contains files relevant to processing 2017 onroad emissions based on MOVES3 through SMOKE-MOVES, using data developed for the 2017 EQUATES time series project. SMOKE-MOVES emissions processing using MOVES3 data is different than in past platforms based on older versions of MOVES, and those differences are described here. For onroad and onroad_ca_adj, ancillary files for onroad are included in this package as well as activity datasets and scripts. Emission factor tables for onroad are provided with separate zips for each state. These tables do not include a NOx humidity adjustment, and instead that adjustment is applied in SMOKE-MOVES using hourly meteorology. The NOx humidity adjustment impacts RPD, RPH, and RPHO. This package includes precompiled SMOKE 4.8. SMOKE 4.8 or later is required for MOVES3-based SMOKE-MOVES processing. == 1. Introduction == This SMOKE-MOVES-focused package consists of the following zip files: - smoke_EQUATES_2017_onroad_scripts_execs_11jan2021.zip: Scripts, pre-compiled SMOKE executables, and I/O API utilities. - EQUATES_2017_inventory_onroad_activity_11jan2021.zip" Onroad activity data for running SMOKE-MOVES. Includes vehicle miles traveled (VMT), vehicle population (VPOP), average speed (SPEED), and hoteling hours. Also includes two activity datasets which are new for MOVES3-based processing: off-network idling hours, also known as ONI; and STARTS (counts of total vehicle starts). - EQUATES_2017_eftables_cb6_st*.zip: Emission factor tables from MOVES3, organized into zip files by state. - EQUATES_spatial_surrogates_CONUS12_2017NEI_18mar2020_with_2017_oilgas.zip: 12km spatial surrogates for processing the EQUATES 2017 case. - ge_dat_for_EQUATES_2017_onroad_all_11jan2021.zip: All ancillary files other than spatial surrogates needed to process MOVES3-based SMOKE-MOVES for 2017. - EQUATES_2017_onroad_ff10_SMOKE-MOVES-emissions_15dec2020.zip: FF10-formatted emissions inventory representing the SMOKE-MOVES results for 2017. == 2. Installation of data files and scripts == Choose an install directory on your system; we will refer to this directory as "INSTALL_DIR". To review/reproduce emissions for all sectors, unzip all the .zip files into INSTALL_DIR. The packages have subdirectories embedded within them, so it is important that all files be unpacked in the same place in order for the scripts know where to find the inputs. Prior to running SMOKE, you will need to edit the INSTALL_DIR (your install directory) and MET_ROOT (location of MCIP meteorology data) environment variable definitions in the "directory_definitions.csh" script located in each CASE/scripts directory. This script is sourced by each of the individual run scripts for each sector. Regarding the MCIP data, SMOKE only uses the GRIDCRO2D, METCRO2D, and METCRO3D files. MCIP meteorology data is not included in the package but is required for SMOKE-MOVES processing. == 3. Notes regarding onroad == Onroad emissions are processed using SMOKE-MOVES. The processing is split into multiple run scripts. For the EQUATES project, SMOKE-MOVES inputs were prepared using MOVES3. The main differences between MOVES2014b-based processing and MOVES3-based processing are: - The number of processes / rates has increased from 4 to 6 - More fuel-vehicle combinations are supported, including gas intercity buses, several new CNG vehicle types, and brake/tire emissions from electric vehicles Speciation of VOC emissions is handled within the MOVES model. MOVES3 supports speciation for the CMAQ 5.2 version of CB6. MOVES3 also supports the latest version of CB6, known as CB6R3AE7. However, MOVES3 was not run for the CB6R3AE7 mechanism for the EQUATES project; instead it was run for the prior version of CB6. The new CB6R3AE7 speciation would have only a minor impact on onroad emissions. -- 3A. Rates / processes for MOVES3 -- SMOKE-MOVES processing is split into multiple sectors, which themselves are split into multiple "rates", or processes. SMOKE-MOVES handles onroad emissions separately for six types of processes for MOVES3-based processing. This is an increase from MOVES2014b-based processing, which had four types of processes. The MOVES3 rates/processes are: - On-network emissions (RatePerDistance, or RPD) - Off-network emissions, fuel vapor venting (RatePerProfile, or RPP) - Off-network emissions, extended idling (RatePerHour, or RPH) - Off-network emissions, evaporative emissions other than fuel vapor venting (RatePerVehicle, or RPV) - Off-network emissions, exhaust other than extended idling (RatePerStart, or RPS) - Off-network idling exhaust emissions (RatePerHour - Off-network idling, or RPHO) The RPS and RPHO processes are new for MOVES3-based processing. RPS exhaust emissions were formerly part of RPV. MOVES2014b-based RPV is equivalent to the combination of MOVES3-based RPS and RPV. RPS includes the exhaust emissions which were formerly part of RPV, and RPV now only includes evaporative emissions. The off-network idling emissions represented by the RPHO process are completely new in MOVES3 and have no equivalent in MOVES2014b. -- 3B. Sector splits for MOVES3 -- For the EQUATES project, onroad emissions have been separated into four sectors. Each of the sectors includes either 4 or 5 of the 6 rates/processes listed above. The four sectors are: - onroad_ca_adj_gas: Gas, E-85, CNG, and electric emissions (all but diesel) in California. - onroad_ca_adj_diesel: Diesel emissions in California. - onroad_gas: Gas, E-85, CNG, and electric emissions (all but diesel) outside of California. - onroad_diesel: Diesel emissions outside of California. The 'gas' sectors have five of the six processes: RPD, RPP, RPV, RPS, and RPHO (no RPH). The 'diesel' sectors have four of the six processes: RPD, RPH, RPS, and RPHO (no RPP or RPV). The gas sectors (which also include E-85, CNG, and electric) were separated from diesel to support downstream projects involving source apportionment, or sensitivities involving targeted emissions adjustments. California is separated because the procedure for running California involves adjusting emissions in SMOKE-MOVES to match datasets provided by the California Air Resources Board by way of a control factor file (CFPRO). There are separate run scripts for each process within each sector. Then, there is a merge script for each sector which merges the emissions from each process together. So there are 22 total scripts: 5 for each of the two gas sectors, 4 for each of the two diesel sectors, and one merge script for each of the four sectors. The RPD, RPP, RPV, RPS, RPHO, and RPH scripts are run first and can be run in parallel. The merge sectors are run last, after all processes for each sector have been completed. The end result is four emissions files per day, one for each of the four onroad sectors. To summarize all of the sector processes and jobs: - onroad_ca_adj_gas: RPD, RPP, RPV, RPHO, RPS, merge - onroad_ca_adj_diesel: RPD, RPH, RPHO, RPS, merge - onroad_gas: RPD, RPP, RPV, RPHO, RPS, merge - onroad_diesel: RPD, RPH, RPHO, RPS, merge -- 3C. Combining sectors -- The RPD, RPP, and other processes must be run separately. However, the four different sectors - onroad_gas, onroad_diesel, onroad_ca_adj_gas, onroad_ca_adj_diesel - can be run together, all at once. To run gas and diesel together rather than as separate sectors: - For RPD, RPS, and RPHO, create a new run script by copying the gas script. Edit the sector name at the top of the script to remove "_gas" from the name. Then copy the EMISINV_A (and EMISINV_B for RPD) inputs from the _diesel script and add them to your new script with different letters, e.g. EMISINV_C or EMISINV_D, so that there are no duplicate names. For example, the new RPD script should have EMISINV_A (gas VMT), EMISINV_B (gas SPEED), EMISINV_C (diesel VMT), and EMISINV_D (diesel SPEED). RPS and RPHO would just have an EMISINV_A (gas ONI/STARTS) and EMISINV_B (diesel ONI/STARTS). You do not need to change any other inputs in the sector, as the EF tables, MRCLISTs, and other inputs already handle all SCCs. - RPH only has a diesel component, and RPP+RPV only have a gas component. So for those processes, you only need to change the sector name at the top of the script. - For the merge script, copy the gas merge script, change the sector name as in other scripts, and then delete the "NO_RPH" parameter so that the merge job knows to look for all six processes. To run California and the rest of the country together rather than as separate sectors: - For each process, start with the California (onroad_ca_adj) script. Edit the SECTOR name at the top, removing the _ca_adj portion of the sector name (or renaming to something else, user's choice). Then open the non-California script and copy all EMISINV inputs from that script (except those which are commented out) into your new script, with different letters; e.g. EMISINV_C and EMISINV_D so that there are no duplicates. - Keep all other inputs and parameters the same, including the California-specific CFPRO and USE_CONTROL_FACTORS parameter. To run all four sectors together as one, follow all of the above instructions in succession, starting with the gas/diesel combination instructions, followed by the California+rest of US instructions. -- 3D. DAYS_PER_RUN -- SMOKE-MOVES processing can take many hours and days to complete for an entire year. To help speed up processing, SMOKE supports running multiple days at once. This is controlled by a parameter called DAYS_PER_RUN. SMOKE-MOVES can be run more efficiently if running more than one day at a time. For example, Movesmrg can create one 7-day emissions file more quickly than it can create seven individual 1-day emissions files. To turn on this feature, use the DAYS_PER_RUN variable, set to the number of days you wish to run in a single Movesmrg instance. The recommended value for DAYS_PER_RUN is 7. The onroad scripts include a setting called "DAYS_PER_RUN", set to 1 as the default. If DAYS_PER_RUN > 1, after Movesmrg is run, the run scripts will use the I/O API utility m3xtract to split up the multi-day emissions file into single day (25-hour) emissions files. Multi-day Movesmrg runs will never cross months. For example, if DAYS_PER_RUN = 7, then the last Movesmrg run of January will start on January 29th and end on January 31st (3 days), and the first Movesmrg run of February will start on February 1st and end on February 7th. Using the multi-day Movesmrg functionality requires multi-day MCIP files. For example, if DAYS_PER_RUN = 7, your METCRO2D files must also be 7 days (169 hours) long. These multi-day MCIP files should be stored in MET_ROOT_${Xday}/, where X = DAYS_PER_RUN (i.e. /7day for DAYS_PER_RUN = 7). For example, if the single day MCIP files are in /foo/foo/mcip_dir/, then 7-day MCIP files should be in /foo/foo/mcip_dir_7day/. The primary drawback to using this multi-day Movesmrg functionality is an increase in the memory usage. -- 3E. Runtime information -- When running for an entire year, it is normal for SMOKE-MOVES processing to take several days to complete for an entire year. This table show average per-day runtimes in minutes for each sector and process on EPA systems: RUNTIMES (minutes per day) when DAYS_PER_RUN = 1 on EPA systems RPD RPH RPHO RPP RPS RPV US gas 100 n/a 5 12 23 11 US diesel 70 2 4 n/a 20 n/a CA gas 25 n/a < 1 < 1 1 < 1 CA diesel 10 < 1 < 1 n/a < 1 n/a This table show average per-week runtimes in minutes for each sector and process when using DAYS_PER_RUN = 7: RUNTIMES (minutes per 7 days) when DAYS_PER_RUN = 7 on EPA systems RPD RPH RPHO RPP RPS RPV US gas 350 n/a 55 17 80 45 US diesel 205 45 45 n/a 70 n/a CA gas 41 n/a 5 2 5 5 CA diesel 19 4 5 n/a 4 n/a For example, the US gas sector (onroad_gas) can complete one day in 100 minutes when running one day at a time, but can complete seven days in 350 minutes (50 minutes per day) when DAYS_PER_RUN = 7. Gas takes longer to run than diesel because there are more different classifications of gas emissions (SCCs) than there are diesel. If you are combining gas+diesel in a single sector, you should expect a longer runtime than when running gas by itself. But, total runtime will be less than the separate gas runtime plus the separate diesel runtime. In other words, running gas+diesel together is faster than running gas and diesel one after the other, but slower than running gas and diesel in parallel. The same goes for combining California with the rest of the US. Because of the long runtimes, consideration should also be given to running multiple RPD jobs in parallel, such as one job per quarter. On EPA systems, we typically run 16 RPD jobs - one per quarter, for each of the four sectors (onroad_gas/diesel, onroad_ca_adj_gas/diesel) - all at once, in addition to the jobs for other processes, all with DAYS_PER_RUN = 7. With this setup, an annual run takes 3-4 days to complete on EPA systems. Each job is allocated 20GB of memory on EPA systems as our default, although the jobs other than RPD can be run with far less memory.