UNITED STATES ENVIRONMENTAL PROTECTION AGENCY Office of Air Quality Planning and Standards Research Triangle Park, North Carolina 27711 March 23, 1989 MEMORANDUM SUBJECT: Model Clearinghouse Review of Outline for PM10 SIP Modeling Protocol FROM: Joseph A. Tikvart, Chief /s/ Source Receptor Analysis Branch, TSD (MD-14) TO: Robert B. Wilson, Regional Meteorologist Region X In response to your request, the Model Clearinghouse has reviewed the "Outline for PM10 SIP Modeling Protocol." Our understanding is that your States are asked to follow that outline in developing a specific model protocol for each PM10 SIP area. We also understand from your cover memorandum, that the stagnation model evaluations that have occurred in your States, pursuant to Section III of the outline, have not generally involved a rigorous application of the "Interim Procedures for Evaluating Air Quality Models (Revised)." Rather they have utilized a more limited data base, more in line with the 3a criteria of the Guideline. We also note that you have found that it would be desirable to have somewhat more model evaluation data than that available from "a few monitoring stations" you have had to work with. We agree that your outline is appropriate for modeling PM10 areas where valley stagnation or other unique meteorological phenomena may govern the design concentration. We also agree that your requirement for a limited performance evaluation based on the "Protocol for Determining the Best Performing Model" is an acceptable way of implementing the principles in the outline. By copy of this memorandum, we are urging other Regions to follow the principles contained in your outline, to promote national consistency. cc: S. Reinders, SRAB/TSD (MD-14) M. Smith, SDPMPB/AQMD (MD-14) D. Stonefield, SDPPB/AQMD (MD-14) Regional Modeling Contact, Regions I-X (w/Copy of incoming memorandum and list of FY-89 Clearinghouse memoranda) (Attachments Following) UNITED STATES ENVIRONMENTAL PROTECTION AGENCY REGION 10 1200 Sixth Avenue Seattle, Washington 98101 March 15, 1989 MEMORANDUM SUBJECT: Model Clearinghouse Review of Outline for PM10 SIP Modeling Protocol FROM: Robert B. Wilson Regional Meteorologist TO: Joseph A. Tikvart, Chief Source Receptor Analysis Branch, OAQPS (MD-14) Last year I developed the attached "Outline for PM10 SIP Modeling Protocol" to assist the State and Local agencies in our Region in preparing modeling protocols as part of the PM10 SIP development process. In developing the outline we tried to follow the principles contained in the Guideline on Air Quality Models (Revised). That is, one should first consider the use of an Appendix A Guideline model in the situation. Then, if there are concerns that the Appendix A model may not be applicable to the meteorology, topography, or source characteristics of the problem, one should consider the use of an appropriate non-Guideline technique. If a non-Guideline model is to be selected, Section 3.2.2 indicates that if the alternative model is theoretically sound, and the data bases to run the model are available, then its applicability needs to be demonstrated through some kind of performance evaluation. If the Appendix A model is clearly not applicable, then the "3a" criteria of Section 3.2.2 may be used, whereby it is shown that the non-Guideline model is not biased toward underestimation. If it is not clear that the Appendix A model is inappropriate, then the "3b" criteria applies, and a rigorous application of the Interim Procedures for Evaluating Air Quality Models (Revised) is necessary. As we have applied that general guidance to the Group I areas in our Region, a consistent theme has arisen. Most of the PM10 problems occur during stagnation conditions, suggesting that no Appendix A model may be applicable. Thus, the use of a non-Guideline model has become necessary. We have generally required that agencies perform modeling with both a stagnation model, WYNDvalley, and a Guideline model, such as, RAM or ISC. We believe that application of WYNDvalley as a non-Guideline model really fits the 3a criteria of the Guideline. However, because very little data exist to demonstrate that the model does not underpredict, we believe that for each application, some comparison is necessary of the non-Guideline model estimates with both measured air quality concentrations, and, as a point of reference, with estimates from a Guideline model. As guidance for selecting the appropriate Guideline model, RAM has generally been recommended where area sources in the urban area are more important, relative to point sources. On the other hand, ISC is recommended where industrial point sources are relatively important, and the potential exists for downwash. In these non-Guideline model situations, we have required a limited performance evaluation using Bill Cox's "Protocol for Determining the Best Performing Model" to select the model to use for determining the design concentration and for evaluating control strategies. One aspect in which we deviate from Cox's protocol is that we pair the predictions and observations in time and space. This is because: 1) as a practical matter, WYNDvalley is applied on an episode day basis, and 2) evaluating individual source contributions to exceedance concentrations at a particular location during a particular time is the most important aspect of this type of modeling exercise. The performance evaluations are limited in the sense that usually only a few monitoring stations are available to provide data for comparison with model predictions, and because of significant uncertainties in the input data bases of emissions and meteorology. Thus, to date, confidence is not high that these evaluations are able to provide conclusive information concerning the relative skills of the models. Yet, we are nonetheless left in the position of having to apply the model(s) exhibiting the best relative performance in order to fulfill the exigencies of the PM10 SIP process. We are somewhat concerned that there may be a lack of consistency in how non-Guideline models are selected, evaluated, and applied to PM10 stagnation situations. Our ability to effect a credible modeling program for stagnation in our States depends to some extent on what other Regions/States are doing for such situations. We would appreciate the Model Clearinghouse review of the attached outline. To the extent that you think it is appropriate for valley stagnation situations, we suggest you share it with other Regions to promote consistency. Please provide comments to me at your earliest convenience. Attachment Outline for PM10 SIP Modeling Protocol I. Analysis of High Concentration Days This analysis should be based on at least two years of the most recent PM10 monitoring data from the areas of concern. The data should be from everyday monitoring. Justification should be provided that the monitoring location represents maximum concentration locations. The highest fifteen concentration days, or all exceedance days, whichever is greater should be analyzed. Representative meteorological data associated with the high concentration days should be evaluated to assess worst-case conditions. For example, are average wind speeds low, what are prevalent wind directions, were there meteorological conditions typical throughout a regional area, etc. Important sources contributing to the air quality problem should be identified (area sources, point sources, downwash, terrain impact, etc.). The identification of important sources may be based on other prior modeling, emission inventories, or other knowledge of the area. The purpose of this analysis it to provide a basis to determine what dispersion models might be applicable to the problem. II. Determination of Applicable Models Considering the sources, topography, and meteorology associated with the air quality problems, appropriate modeling techniques should be identified. First, consideration should be given to EPA modeling guidance ("Guideline on Air Quality Models," EPA-450/2-78-027R, July 1986, including Supplement A, January 1988). Guideline techniques should be used as applicable and appropriate. Where Guideline models are not applicable, consideration should be given to the use of non-Guideline models. For example, in the case of a pulp mill with elevated terrain in the area, the use of two Guideline models, ISC and Complex I, is appropriate. In another example, there are no Guideline models which are applicable to stagnation problems. In this case, the use of the non-Guideline model WYNDvalley should be considered. In some cases, the data bases necessary to perform the modeling are not available. A data collection program will be required to develop the necessary data. For example, some models require at least one year of on-site meteorological data to perform an adequate analysis. III. Model Evaluation Procedure (for use of a non-Guideline model) Where a non-Guideline model is used, justification must be provided in a comparison with a Guideline model (see "Interim Procedures for Evaluating Air Quality Models, EPA-450/4-84-023, September 1984). This justification should include a comparison of the two models on a theoretical basis and on a performance basis. The model performance evaluation is the most important part. A protocol for the performance evaluation should be developed which establishes the basis for the selection of the best model to use in the SIP demonstration modeling. The protocol will identify the measured concentrations, the meteorological data, the model options, the statistics to be generated based on comparing predicted and observed values, the selection criteria, etc. IV. SIP Demonstration Modeling The selected model(s) is then used to model the high concentration days. Various control strategies can be included to evaluate effectiveness. If the model is a Guideline technique, it must be used according to recommendations in the Guideline. If it is a non-Guideline technique, the model must be used in a manner consistent with the methodology used in the performance evaluation which was the basis for its selection. Compliance with standards should be demonstrated in all areas at all times.