######################################################################
 CEM Correct
 version 2.1
 Copyright 2012-2014 UNC-Institute for the Environment                                                       
######################################################################

Updates
-------------
* Added an anomaly summary report that flags all records in the raw CEM files
    that are > 2x the mean and lists the action taken on the record (corrected or kept)
* Fixed a bug in the hourly mean record calculation
* Changed the anomaly replacement algorithm:
     - Only keep data anomalies (>2x mean) if the data flags for NOXMASS, SO2MASS, and HTINPUT
         are all either 1 (measured) or 2 (calculated). Logic will results in anomalies
         being either all kept or all removed; avoids cases where one pollutant is kept and 
         another one is removed for the same hour at the same facility
     - Do not touch any data records that are <= 2x the mean

Description
-------------
Under Part 75 of Volume 40 in the Code of Federal Regulations, continuous emissions
monitoring (CEM) and reporting is required for large EGUs and industrial facilities. The
U.S. EPA Clean Air Markets Division (CAMD) collects and distributes hourly CEM data
for NOx and SO2 emissions (lbs/hr), heat input (mmBTU), gross load (MW), and steam
load (1000 lbs/hr) for thousands of U.S. sources from the year 1995 to the present. Some
units are required to report hourly emissions year-round (annual reporters), while other
units are only required to report hourly emissions for part of the year (partial year
reporters). To satisfy the Part 75 requirement that CEM data are reported for every
operating hour that is required to report emissions, a complex process for reporting and
filling in missing data has been defined. Many times, missing emissions are substituted
with values that are much larger than the actual emissions that were emitted.

The program CEM Correct identifies anomalous values in the CEM database, determines 
if they are substituted values, and replaces the anomalies with mean data values.
Details of the algorithms implemented in CEM Correct are available here:

http://www.ie.unc.edu/cempd/projects/SEMAP/secure/documents/SEMAP_EGU_Modeling_Approach_11_17_2011.pdf

Input
-------------
SMOKE formatted CEM emissions files with the headers removed

Usage
-------------
1) You must have Java installed.
2) Under cemcorrect/cemcorrect_1.3 run the script cemcorrect.sh

Configuration 
-------------
cemcorrect/cemcorrect_1.3/config/config.properties

input.dir - full directory path to the input CEM data files
output.dir - full directory path to the output corrected CEM data files
report.prefix - output report filename prefix
report.suffix - output report filename suffix
states.list - list of states to apply corrections; program designed to work on CEM files that are broken up by state;
   you can run on data files that contain more or less than one state; the two-character state code(s) set here 
   reference the file list by state below (i.e. states.list = AL references the file list under alabama.files below)
alabama.files - CEM files for Alabama, alternatively CEM files associated with the state code AL provided in state.list above
events - ORIS, Boiler, start date (YYMMDD), end date (YYMMDD) for the application/removal of emissions control devices
