Semisequicentennial Transportation Conference Proceedings
May 1996, Iowa State University, Ames, Iowa

Use of Expert Systems for Roadway Weather Maintenance Decisions

Eugene S. Takle and Paul C. Thomson

E.S. Takle, 3010 Agronomy Hall,
Iowa State University,
Ames, Iowa 50011.

P.C. Thomson,
Black and Veatch,
8400 Ward Parkway,
P.O. Box 8405,
Kansas City, Missouri 64114.

We have developed and deployed automated systems for forecasting frost and fog on roadways and bridges at specific locations in Iowa. These systems ingest current observations and forecasted values of specific weather variables and produce forecasts of the indicated roadway condition. Forecasts made on the basis of uncertain (weather) input information will invariably lead to less-than-maximum hit rates and greater-than-zero false-alarm rates. A procedure, based on signal detection theory, has been developed to separately analyze the accuracy and bias of the systems. By using this procedure, the roadway maintenance manager can tune the system to achieve the optimum balance of hit-rate-versus-false-alarm rate for a given application. Comparison of the estimated levels of accuracy of these forecast systems with other reports in the meteorological literature reveals that our systems have skill levels sufficient to have practical value. Key words: frost, fog, expert system, roadway weather, decision making.


An expert system is a computer-based tool that stores a model of human expert reasoning with an associated knowledge base and combines these to reach the same conclusion as a human expert to a complex problem. We have developed expert systems for two specific tasks relating to roadway weather decision making. The first is a system for forecasting frost formation on bridges and roadways in central Iowa, and the second is a system for forecasting fog on US Highway 30 in Cedar Rapids, Iowa, due to plumes emitted by cooling towers at a corn-sweeteners production plant adjacent to the roadway. From these experiences we have concluded that expert systems can be useful in roadway weather maintenance decisions. These experiences also have allowed us to consider the more general issue of decision making with regard to the use of weather information.

FROST FORMATION ON BRIDGES AND ROADWAYS

Frost formation on bridges and roadways in Iowa poses a potential safety problem for motorists, in large measure due to its patchy nature. Frost suppression measures, such as sanding and salting affected areas, must be implemented in a timely manner. An accurate forecast of frost is needed so that the Iowa Department of Transportation (Iowa DOT) can have personnel, equipment, and material available at the locations needing attention. Under sponsorship of the Iowa DOT, we developed an expert system to forecast frost 18 hours in advance (1,2).

The expert system uses a backward-chaining system and consists of 32 parameters and variables and 33 rules. Roadway and bridge frost data from December, January, and February of four frost seasons (1985–89) were used to develop the rules for the system. The rules are used in combination to forecast separate values of temperature for the bridge and roadway, which are compared with the forecast of the dew-point temperature to determine the likelihood of frost.

Input to the system consists of the three data items and seven forecast variables listed in Table 1. The system was run at about 11:00 a.m. LST to forecast conditions at approximately 5:00 a.m. the following morning. In the operational setting, the input variables were supplied by the forecast meteorologists who also ran the expert system.

The verification matrix of Table 2 summarizes the performance of the system as measured against actual outcomes. The conditions for frost to occur are that (1) the surface temperature must be below freezing, (2) the surface temperature must be below the dew-point temperature, and (3) the dew-point temperature must be near (even above) freezing or else well above the surface temperature for a significant period of time. We first set the decision criterion to be that frost would form if the estimated surface temperature was less than or equal to the dew-point temperature. However, this criterion can be changed to examine the influence on the hit rate and false-alarm rate. If we increase the temperature threshold by 1oC we are saying that frost will form somewhere in the region even if the surface temperature at some reference location is 1oC higher than the dew-point temperature. The plot in Figure 1 shows how changing the threshold changes the hit and false-alarm rate.

The system was designed, tested, and deployed operationally. In practice, the forecasters typically would run the system several times with different combinations of the parameters in Table 1 to examine the sensitivity of the present situation to small changes in the forecast variables. The system was found to have accuracy comparable with human forecasters. Details of the comparison of forecast accuracy of the system are given in (1).

ROADWAY FOG PRODUCED BY AN ADJACENT PLANT

Heavy fog with accompanying low visibility form in the vicinity of US Highway 30 in Cedar Rapids, Iowa, due to copious amounts of water vapor released from linear mechanical-draft cooling towers at a corn-sweetener plant adjacent to the roadway. Ambient atmospheric conditions of wind speed, wind direction, temperature, dew-point temperature, and surface moisture are key conditions that determine whether the resulting water-vapor plume will lead to low visibility for motorists on Highway 30. Safety precautions by Iowa DOT in the event of fog include rerouting traffic to a city street during the episode.

Accurate forecasts of onset and termination of low visibility conditions are needed to assist Iowa DOT personnel in their monitoring efforts and in scheduling closure events. Iowa DOT has supported a research project to evaluate the conditions under which low visibility occurs and to develop automated systems to forecast these events (3).

An expert system was developed using the same shell as was used in the frost problem previously described. This system determines the probability of fog individually for forecast values of temperature, dew-point temperature, wind speed, and wind direction. Each value is then multiplied by a probability factor and combined with the others to determine the aggregate probability that the cooling-tower plumes will trigger a road closure. The rules and probabilities were developed by use of discriminate analysis techniques and trial-and-error methods to achieve the best combination of accuracy (false-alarm rate vsersus missed closure events). The system was developed by use of meteorological data and Iowa DOT road-monitoring data from October 1989 through March 1994, which contained 2,153 hours of observations including 27 road closures. The occurrence of fog on the roadway is highly dependent on the plant operating procedure. In 1994, the plant reduced use of one of the cooling towers nearest the roadway, leading to an overprediction by the expert system. The system remains in use by local Iowa DOT personnel, however, to provide a worst-case scenario.

Citing the need to reduce the number of hours for monitoring the roadway for fog occurrences, the Iowa DOT requested an investigation of procedures to forecast fog occurrences regardless of whether they would lead to road closure. Improvements in data availability and communication during this period allowed development of a more advanced method for creating and delivering fog forecasts. The new system acquires forecast values of the key meteorological variables previously listed directly from the Nested Grid Model (NGM) Model Output Statistics (MOS) of the National Meteorological Center every 12 hours. These data were interpolated to the Cedar Rapids site and used as input to a Fortran algorithm having the same logic as the expert system. The output of the system generated a combined forecast fog probability category of high, medium, low, or zero twice daily at 11:00 a.m. and 11:00 p.m. for 11 three-hour hour intervals beginning, respectively, at noon and midnight. For evaluation purposes, the first four intervals were considered as the forecast and the remaining seven intervals as an outlook. The results of the calculation were immediately sent electronically to Iowa DOT personnel in Cedar Rapids and to the Iowa DOT forecasters without requiring human intervention.

A period from 1 January through 28 February 1995 was used to verify the automated procedure. During this period 85 separate forecasts were issued. By its previous criterion for roadway surveillance, the Iowa DOT monitored the roadway 114 hours during this period. Of these 114 hours, only 24 hours corresponded to periods where the model was issuing high or medium probability of fog, leaving 90 hours of monitoring when the model forecast no fog problem. Of these 90 hours, 6 hours corresponded to events missed by the Fortran program (e.g., periods where a low or zero category was forecast but for which a plume was observed above the roadway at any elevation). The model tended to over-predict fog during the early forecast periods (from three to 12 hours). In spite of this conservative bias, the model would have reduced the roadway monitoring time from 114 hours to 60 hours, a reduction of 47 percent.

The verification matrix of Table 2 was used to evaluate the model performance. The original decision criterion was that a closure would occur if the probability of fog was greater than 70 percent (forecast category medium and high). By changing this threshold we can examine the influence on hit rate and false-alarm rate, which are plotted in Figure 2 as a function of closure probability. The general trend is for both hit and false-alarm rates to increase as probability threshold decreases.

USE OF UNCERTAIN INFORMATION IN MAINTENANCE DECISIONS

The delivery of a tailored weather forecast and the development of a resulting policy decision based on this forecast raise the issue of division of responsibility. For example, a meteorologist is accustomed to issuing a forecast that there is a 30 percent chance that frost will form on a bridge. The maintenance supervisor must decide if this is sufficient justification to deploy a sanding truck. Factors entering this decision include the actual costs of manpower, equipment, and materials but also the potential for an accident and possible litigation resulting from not taking action. Actual cost of the first (low for individual events but large in aggregate) must be weighed against the potential cost of the second (possibly very large if it occurs). The maintenance supervisor cannot ask the forecaster to give a "yes" or "no" on frost, because this would force the meteorologist to make a policy decision based on some level of risk, which is the maintenance supervisor's responsibility. Rather, the forecaster should issue a percentage chance, and the supervisor must establish a threshold, or decision criterion, beyond which frost-suppression action is taken.

The method of signal detection theory (SDT) (4,5,6,7) allows us to evaluate the probabilities of a "hit," "miss," "false alarm," or "correct nonoccurrence" and their relationship, separately, to forecast accuracy (the responsibility of the forecaster) and decision criterion (the responsibility of the maintenance supervisor). Increasing the hit rate also increases the false-alarm rate. By use of signal detection theory, the maintenance supervisor can balance hit rate against false-alarm rate, independently of forecast accuracy.

From SDT, an index of accuracy, d', is the number of standard deviations separating the means of the (assumed normal) distributions of decision variables preceding occurrence and preceding nonoccurrence. Thus, if d' = 0, there is no skill because the probability of hit and false alarm are equal. A second index, B, is the likelihood ratio that the given data suggest occurrence over nonoccurrence. The criterion placement is considered unbiased if B = 0, biased toward maintaining a low false-alarm rate (at the expense of a lower hit rate) if B > 1, and biased toward maintaining a high hit rate (at the expense of a higher false-alarm rate) if B > 1. A third parameter, A, is the area under the curve of the SDT relative operating characteristics curve (1) and can be interpreted as the percentage of time that the system can distinguish between conditions leading to occurrence from conditions leading to nonoccurrence of fog or frost. Swets (7) considers a system with A values below 70 to have insufficient accuracy for much practical value and systems with values between 70 and 90 to be useful for some purposes. Table 3 gives the estimated values of d', B, and A for the expert systems for bridge and roadway frost and for the Fortran program used for fog forecasting. All systems show skill at discriminating occurrences from nonoccurrences (d' > 0), and, for the decision criteria used, all systems are biased toward maintaining a low false-alarm rate at the expense of a lower hit rate. The values of A suggest that all systems exhibit skill in predicting their respective roadway conditions. For the frost project, we obtained data and computed analogous statistics for human forecasts as shown in Table 3. These results show that the human forecasters were less biased toward maintaining a low false-alarm rate, and that the skill was comparable or slightly lower than the expert system. We emphasize that this comparison is not strictly valid because the expert system is evaluated at its potential best because we have assumed “perfect forecasts” for the input variables (Table 1) to the system. For the fog problem, the human forecasters had a hit rate of 70 percent, comparable to the Fortran program, but they had approximately twice the false-alarm rate. TABLE 3 Measures of Accuracy and Bias for Expert Systems and Human Forecasters for Forecasts of Frost and Fortran Model for Forecasts of Fog

d' B Area
Frost, Expert System (00C criterion)
Bridge 1.7 2.32 85
Roadway 1.4 1.90 82
Frost, Human Forecaster
Bridge 1.2 .093 78
Roadway 1.4 1.60 82
Fog, Expert System(70% probability criterion)
Forecast 1.5 3.94 83
Outlock 1.6 3.48 84

RECOMMENDATIONS FOR USE OF EXPERT SYSTEMS

Our experience in developing these systems and our observations of other expert systems that have been developed to provide more general meteorological forecasts has taught us that such expert systems are more likely to be successful if they are designed to forecast a specific event (e.g., frost on a bridge) rather than more general conditions (e.g., occurrence of severe weather). This is because the number of rules needed to discriminate occurrence from nonoccurrence is fairly limited (about 50 for our systems). Verification of simpler systems is a much more manageable task.

REFERENCES

  1. E.S. Takle. Bridge and Roadway Frost: Occurrence and Prediction by Use of an Expert System. J. Appl. Meteorol., Vol. 29, 1990, pp. 727–734.
  2. E.S. Takle and S. T. Hansen, Jr. Development and Validation of an Expert System for Forecasting Frost on Bridges and Roadways. Preprints, Sixth Int’l. Conf. on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology, 1990, pp.197–200.
  3. P.C. Thomson. Development of Software to Forecast Periods of Cooling-Tower Fog along US Highway 30 in Cedar Rapids, Iowa. MS thesis, Iowa State University. 1995. 112 pp.
  4. I. Mason. On Scores for Yes/No Forecasts. Proceedings of Ninth Conference on Weather Forecasting and Analysis. Amer. Meteorol. Soc., 1982, pp. 169–174.
  5. J.A. Swets. The Relative Operating Characteristic in Psychology. Science, Vol. 182, 1973, pp. 990–999.
  6. J.A. Swets. Indices of Discrimination or Diagnostic Accuracy: Their ROCs and Implied Models. Psychol. Bull, Vol. 99, 1986, pp.100–117.
  7. J.A. Swets. Measuring the Accuracy of Diagnostic Systems. Science, Vol. 240, 1988, pp.1285–1293.

This research was funded by the Iowa Department of Transportation through the Iowa Highway Research Board research projects HR-305 and HR-357. The opinions, finding, and conclusions expressed in this publication are those of the author and not necessarily those of the Highway Division of the Iowa Department of Transportation.

CTRE is an Iowa State University center, administered by the Institute for Transportation.

Address: 2711 S. Loop Drive, Suite 4700, Ames, IA 50010-8664

Phone: 515-294-8103
FAX: 515-294-0467

Website: www.ctre.iastate.edu/

Iowa State University--Becoming the Best