The Savannah River Site (SRS) is a Department of Energy (DOE) complex covering roughly 800 km² of southwest South Carolina. Traditionally, hazards posed by SRS operations have been associated with nuclear reactors, waste-vitrification facilities, chemical reprocessing plants, or fuel fabrication. Therefore, facilities such as the SRS rely on the predictive capabilities of dispersion models to assess possible emergency response actions. Accurate and timely wind field input to these models is crucial. For the initial and lateral boundary conditions, steady-state (diagnostic) winds are commonly used because they can be obtained with minimal computational effort; however, their use can cause serious prediction errors during changing weather conditions. Recent advances in computing speed enables mesoscale numerical models to generate three-dimensional prognostic fields in an attempt to describe these variations. It is important to assess the accuracy of the model simulations. There are numerous difficulties in making direct comparisons of model output with observations (Nappo et al., 1982), including the comparison of spatially averaged model results with observations representing point values, inadequate model formulation, inaccurate boundary conditions, and observational errors. Systematic differences (e.g., model physics errors) should affect predicted model output in a consistent way, while data errors are generally random.

This paper describes the statistical measures used to compare several observed and predicted meteorological variables for the southeastern United States spanning two full years (April 1998 to March 2000). Six different locations within the model domain were selected, with the majority of comparisons being made at the surface, since application to operational work performed at the SRS is at or near the ground level.

RAMS simulations are initialized using large-scale data available from NOAA and lateral boundary conditions are generated at 6-hr intervals thereafter, with simulations being updated twice daily (based on 00 and 12 GMT data). For this study, the actual simulation spans 30 hours, but the first 6 hours are ignored while the model is ‘spinning up’ a realistic boundary layer. Because the large-scale data typically take 3 to 4 hours to be processed, the 6-hr forecast from the NOAA data are used as initial conditions for the RAMS simulations (i.e., starting at 06 or 18 GMT). Therefore, a RAMS simulation using large-scale data analysis valid at 00 GMT provide a 24-hr forecast beginning at 12 GMT. Modifications to RAMS were also made to generate standard deviations of azimuth and elevation angle considering the sum of a resolved and subgrid-scale component. For the two-year period, a total of 1462 simulation periods could have been generated. However, due to a variety of problems including missing large-scale data, script errors, or computer outages, only 1337 simulations were created.

Observations were taken from NWS stations within the domain (Fig. 1), as well as at a tower centrally located within the SRS. Wind and temperature observations for the NWS sites are at 10 and 4 m AGL, respectively, while the SRS tower measurements are at 18 and 36 m AGL. The lowest model level in RAMS was 26 m AGL, so linear interpolation between the 18 and 36-m levels for the SRS values was used. (Note that for the SRS location, interpolated values are only available beginning in February 1999). The values at the NWS stations are compared at the differing heights, with the understanding that model biases will arise in the results due to varying heights.

Define s_i and o_i as the simulated and observed value of the i^th point, respectively, of which there are a total of N points. The mean difference between simulation and observation is given by

. (1)

Similarly, the mean absolute bias is determined by taking the absolute difference between all simulated and observed values and determining an average

, (2)

while the standard deviation of the difference is estimated by:

. (3)

The percentage of data points falling within the desired criteria were also determined. Sampling distributions for the means are typically assumed to be normal if N is large.

The skill of the model may be determined using the ratio of the root-mean-square error to the standard deviation of the observations (Keyser and Anthes 1977). This may be written

(4)

where the scalar weighting term, w, is set to unity, and vector quantities denote the possible use of wind vectors in the calculations. This ratio should be less than unity to show skill in making a prediction.

Although results are available for all 6 locations, for brevity, surface comparisons are shown in Table 1 for the SRS location only, since turbulence intensity comparisons are available. Statistical measures as defined by Eqs. (1)-(3) are given for forecast times at 6-hr increments, along with the objective criteria. It is evident that the absolute mean bias for wind direction increases with forecast time with biases on the order of 30 to 40°. Absolute wind direction errors less than 30° are registered 55-65% of the time which compares favorably with values obtained in Cox et al. (1998) when using RAMS. On the other hand, poorly forecasted wind directions (>75°, not shown) occur between 10 to 15%. For emergency response use, it is important that directional errors of this kind be reduced.

Horizontal wind speed values indicate a small positive mean difference. The absolute mean bias and standard deviation difference tends to increase with forecast times, but is worse at the 6 and 18-hr forecast times. This can be traced to the average observed wind speed during this period being higher (i.e. mid-afternoon). However, the objective criterion is met roughly 75% of the time or better.

Table 2 shows results for Charleston at both the surface and aloft (at 4 different pressure levels) for winds and temperature at 0, 12, and 24 hr forecast periods. Absolute bias in wind direction tends to increase with time at all levels, while directional errors are reduced with increasing height. Cox et al. (1998) obtained upper-air percentages between 80 and 95% for a range of 30° for all vertical levels (10 levels between 1000 and 100 mbar) considered. At the 30° interval for the 4 levels and cities chosen in this study, the range is from 70 to 90%. Improvement at the upper levels in direction is the result of less variability with decreasing pressure.

A plot of average bias (Eqs. 1 and 2) as a function of forecast time for Augusta is depicted in Fig. 2 for winds and temperature. This figure contains differences between observations and simulations at 2-hr intervals (from the 0-hr to the 24-hr time period) as averaged over an entire month for simulations using 00 GMT large-scale analysis times (with forecast times beginning at 12 GMT). Each line within the plot represents a different month (covering the 2-yr period of interest). Once again, the wind direction bias is forced to be between 0 and 180°. These figures are useful because conclusions may be determined regarding forecast accuracy as a function of the time of year, as well as the time of day.

The wind direction error as a function of time of day tends to be highest after the transition period from sunlight to darkness (~00 GMT to 06 GMT). However, this does not seem to be the case for wind speed. The temperature bias clearly shows underestimates in the late morning and overestimates overnight. Seasonally, the worst positive bias typically occurs during the warmer months, while negative bias tends to be higher in the cooler months. The large shift in bias from under- to over-prediction at ~00 GMT and from over- to under-prediction at ~12 GMT is likely related to radiative transfer calculations in RAMS, or to differences in measurement heights between model and observation. Although not shown, there is also a clear relationship with simulated moisture content during this transition (00 GMT). On those occasions when temperature is severely under-predicted, dew-point temperature is severely over-predicted. In this case, the soil moisture content may be too large initially, leading to wetter, cooler conditions than observed. Alternatively, when the temperatures are too high, dew-point temperature is often too low. The soil moisture content may be too low initially, resulting in dryer, hotter conditions than measured. In summary, the prognostic model has difficulty with the transition from daytime to nocturnal conditions, particularly in the prediction of temperature (and moisture) fields.

The RMS ratio (Eq. (4)) is illustrated in Fig. 3 for the 6 locations. In the top half of the figure, average values over all months at a given forecast time are shown for vector winds and temperature for the 00 GMT analysis file simulations. The bottom half of Fig. 3 shows the ratios for each observing location as a function of month, averaged over all forecast times. The dropouts for Alma correspond to the nocturnal period in which measurements are not taken.

In most cases, R_RMS<1, indicating that the RAMS model shows some skill in forecasting averaged trends in winds and temperature. For wind comparisons, all locations show R_RMS~0.5, except for Alma. Although values for R_RMS are only slightly higher for temperature than winds, it has been found (not shown) that a much more significant bias exists for the temperature, especially for 12 GMT simulations. A rise in R_RMS for temperature (Fig. 3b) after 12 GMT is also seen in Fig. 2c. For both sets of simulations, ratios at Charleston appear to be lowest.

On a monthly basis (Fig. 3c), wind ratios do not generally indicate a trend as a function of the time of year, with the exception of Alma, which worsens during the warmer months. The maximum ratios for temperature are seen during the summer of 1998, and somewhat less during the warmer season of 1999.

There are numerous reasons for the model biases. In RAMS, temperature differences between the ground and the lowest atmospheric levels affect surface fluxes of sensible heat, momentum and moisture, leading to changes in surface speeds. Overestimates in temperature lead to higher heat and momentum fluxes, hence higher wind speeds. Overall temperature errors seem to be higher during the cooler months for the inland locations (Augusta and Atlanta), indicating possible problems with surface energy budgets in the model. Deficiencies within the radiation and soil parameterizations, as well as the initial soil moisture selected for the model simulations all contribute to this error. There are numerous studies that address this very problem (e.g. Wilson et al. 1987). Of special concern is the large error that occurs during the transition from day to night (and vice-versa). Another issue deals with precipitation. Although RAMS can be configured to generate precipitation, it is more time-consuming, which limits its use in an operational capacity. The operational version used at SRS generates precipitation through convective parameterization, but does not incorporate explicit microphysics, possibly leading to poorer results for those times in which rainfall occurs.

The RAMS simulations are valuable because they provide meteorological information in data-sparse regions and as a forecast. The statistical comparisons discussed here have shown that the model can generate realistic results with skill. A major weakness is seen to be reliance on accurate input conditions (i.e. large-scale synoptic models) which drive the mesoscale model initial and lateral boundary conditions. One way to alleviate biases introduced as a result of large-scale model errors, is to use four-dimensional data assimilation (e.g. Stauffer and Seaman 1990). This will typically provide improved accuracy several hours beyond the initial simulation time period with the bulk of the measurements being restricted to the surface. The initial state of the atmosphere for the current RAMS simulations could be improved by using the local SRS tower data, which are available in 15-minute averages. This has already been done on an experimental basis (Fast et al. 1995) and would be particularly important for local transport applications.