Possible hydrologic forecasting improvements resulting from advancements in precipitation estimation and forecasting for a real-time flood forecast system in the Ohio River Valley, USA

Journal of Hydrology 579 (2019) 124138

Contents lists available at ScienceDirect

Journal of Hydrology journal homepage: www.elsevier.com/locate/jhydrol

Research papers

Possible hydrologic forecasting improvements resulting from advancements in precipitation estimation and forecasting for a real-time flood forecast system in the Ohio River Valley, USA Thomas E. Adams, IIIa, a b

⁎,1

T

, Randel L. Dymondb

TerraPredictions, 1724 Sage LN, Blacksburg, Virginia 24060, United States Charles Edward Via Department of Civil and Environmental Engineering, Virginia Tech, 200 Patton Hall, Blacksburg, Virginia 24060, United States

A R T I C LE I N FO

A B S T R A C T

This manuscript was handled by Emmanouil Anagnostou, Editor-in-Chief, with the assistance of Pierre-Emmanuel Kirstetter, Associate Editor

Errors in hydrometeorological forcings for hydrologic modeling lead to considerable prediction uncertainty of hydrologic variables. Analyses of Quantitative Precipitation Estimate (QPE) and Quantitative Precipitation Forecast (QPF) errors over the Ohio River Valley were made to quantify QPE and QPF errors and identify hydrologic impacts of forcing errors and possible improvements resulting from advancements in precipitation estimation and forecasting. Monthly, seasonal, and annual bias analyses of Ohio River Forecast Center (OHRFC) NEXtgeneration RADar (NEXRAD) based Stage III and Multisensor Precipitation Estimator (MPE) precipitation estimates, for the period 1997-2016, were made with respect to Parameter-elevation Regressions on Independent Slopes Model (PRISM) precipitation estimates. Verification of QPF from NWS River Forecast Centers from the NOAA/NWS National Precipitation Verification Unit (NPVU) was compared to QPF verification measures from several numerical weather prediction models and the NOAA/NWS Weather Prediction Center (WPC). Improvements in NEXRAD based QPE over the OHRFC area have been dramatic from 1997 to present. However, from the perspective of meeting hydrologic forecasting needs, QPF shows marginal improvement. A hydrologic simulation experiment illustrates the sensitivity of hydrologic forecasts to QPF errors indicated by QPF Threat Score (TS) values. A monte carlo experiment shows there can be considerable hydrologic forecast error associated with QPF at expected WPC TS levels and, importantly, that higher TS values do not necessarily translate into improved hydrologic forecasts. These results have significant implications for real-time hydrologic forecasting. First, experimental results demonstrate the value gained in terms of improvements in hydrologic modeling accuracy from long-term radar-based precipitation bias reductions. Second, experimental results show that improvements made in terms of QPF have marginal effect on hydrologic prediction and that, even with significant improvement in QPF accuracy, we should expect large hydrologic prediction uncertainty. These issues are discussed more fully as they relate to the future of operational hydrology and ensemble hydrologic forecasting.

Keywords: Hydrologic Modeling QPE QPF Forecast Error Bias

1. Introduction Hydrologic forecast accuracy is largely dependent on the magnitude of measurement and prediction errors of hydrometeorological forcings used as model inputs, where both the spatial placement of hydrometeorological quantities over watersheds and temporal distribution can affect watershed response significantly (Maurer and Lettenmaier, 2003; Tetzlaff and Uhlenbrook, 2005; Gourley and Vieux, 2005; Benke et al., 2008; Wood and Lettenmaier, 2008; Mascaro et al., 2010; Schröter et al., 2011; Newman et al., 2015). The need for improvements in forecast accuracy and the quantification of forecast uncertainty are

central recommendations by the National Research Council (NRC) to the National Oceanic and Atmospheric Administration (NOAA), National Weather Service (NWS) found in National Research Council (2006). Key to meeting these recommendations by the NRC was the development and implementation of the Advanced Hydrologic Prediction Services (AHPS), where NRC (2006) identifies the need to strengthen “quantitative precipitation estimation (QPE) and quantitative precipitation forecasts (QPF) for hydrologic prediction through an end-to-end evaluation that assesses QPE/QPF quality and impacts on flood and streamflow products for basins”. Thus, improvements in the measurement of observed and prediction of future hydrometeorological forcings

⁎

Corresponding author. E-mail addresses: [email protected] (T.E. Adams, III), [email protected] (R.L. Dymond). 1 Previously with NOAA/NWS Ohio River Forecast Center, Wilmington, Ohio. https://doi.org/10.1016/j.jhydrol.2019.124138 Received 13 October 2018; Received in revised form 14 July 2019; Accepted 10 September 2019 Available online 16 September 2019 0022-1694/ © 2019 Elsevier B.V. All rights reserved.

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

States, as observed precipitation forcings for hydrologic modeling (Fread et al., 1995; National Research Council, 2012; Adams, 2016). National Centers for Environmental Prediction (NCEP) Stage IV (Lin and Mitchell, 2005) products are a nationwide mosaick of Stage III or MPE products generated by RFCs. A recent study by Nelson et al. (2016) made an assessment of NCEP Stage IV QPE. Many RFCs have utilized hourly, gridded Multi-Radar/Multi-Sensor (MRMS) (Zhang et al., 2016) radar-rainfall field estimates within their QPE workflows for operational hydrologic forecasting since the data became available in 2011 with the availability of Q2 (Vasiloff et al., 2007), which was a precursor to MRMS, where the MRMS QPE system is refered to as Q3 (Martinaitis et al., 2014). The MRMS, originally called the National Mosaic and QPE (NMQ) algorithm package, was developed at the NOAA National Severe Storms Laboratory and subsequently moved to the NOAA/NCEP for operational support of NWS RFCs and WFOs. In western regions of the U.S., where radar beam blockage is problematic in mountainous areas, NWS estimation methods rely on data from raingauge and Natural Resources Conservation Service, Snow Telemetry (SNOTEL) networks for precipitation estimation. Gauge data are processed at RFCs, using spatial interpolation algorithms and historical data, such as Parameter-elevation Relationships on Independent Slopes Model (PRISM) (Taylor et al., 1993; Taylor et al., 1995; Daly et al., 2008), within the Advanced Weather Interactive Processing System (AWIPS), to generate gridded estimates of precipitation utilizing Mountain Mapper and Data QC (Schaake et al., 2004).

are critical to improving hydrological forecast accuracy. In operational settings, hydrologic model forcings usually take the form of quantitative estimates of observed and forecasted precipitation and temperature (Adams and Pagano, 2016). The quantification of precipitation errors in watershed runoff prediction errors has a significant research history. As early as 1969, research by Fogel (1969) quantified differences in watershed runoff due to rainfall variability, using a dense raingauge network. More recently, using distributed precipitation inputs, Wilson et al. (1979) and Faurès et al. (Dec. 1995) demonstrated that large variations in modeled watershed runoff can result from spatially variable rainfall. Utilizing dense raingauge networks, Jones and Wendland (1984), Goodrich et al. (1995), and Zhang et al. (2007) report the occurrence of significant rainfall variability over short distances (100–1000 m) which, with gridded precipitation fields, would be considered the subgrid scale. 1.1. QPE errors Past studies have shown that accurate quantification of observed precipitation, known as quantitative precipitation estimate (QPE), is problematic for both raingauge-based and radar-derived estimates. Raingauge based estimation errors arise, largely, due to insufficient gauge density of raingauge networks (Huff, 1970; Sungmin et al., 2016; Cecinati et al., 2017) and low-catch biases of individual raingauges (Humphrey et al., 1997; Ciach and Krajewski, 1999; Ciach, 2003). It has been shown that the highest quality radar based estimates of precipitation depend on raingauge based bias adjustments and other corrections within precipitation processing algorithms (Anagnostou et al., 1998; Young et al., 2000). Such corrections are needed to account for a range of systematic detection and measurement errors associated with beam attenuation, full/partial beam blockage, ground clutter, beam overshooting, curvature of the Earth, anomalous propagation, brightband contamination, conversion from reflectivity to rainfall rates, i.e., non-unique R-Z relationships, beam attenuation, and range effects, including sampling and averaging errors. Significant research by the U.S. NOAA/NWS began in the 1980s related to the development of radar precipitation processing system algorithms, principally aimed at improving precipitation estimation for hydrologic forecasting. This research lead to the development of the 3stage precipitation processing system, comprised of Stage I, II, and III, for NWS S-band Weather Surveillance Radar-1988 Doppler (WSR-88D) radars (Fulton et al., 1998; Young et al., 2000). Stage I processing occurs at the radar location itself, addressing the fundamental conversion of the radar reflectivity signal to precipitation estimates, including corrections for partial beam blockage, ground clutter, anomalous propagation, hail, range degradation, etc. (Fulton, 1998). An output from Stage I is the hourly digital precipitation array product, which is transmitted to NWS River Forecast Center (RFC) and Weather Forecast Office (WFO) field offices. Stage II processing at NWS RFCs and WFOs generates an optimal, hourly rainfall field estimate for each radar (independently) using a multivariate objective analysis scheme that incorporates radar and raingauge observations. This multisensor rainfall field estimate, utilizing all available rain gauge data, is produced hourly for each radar on the NWS Hydrologic Rainfall Analysis Project (HRAP) (Fulton, 1998; Reed and Maidment, 1999) grid, using as input the hourly digital precipitation array product from Stage I. Because NWS RFCs have large regional hydrologic forecast areas of responsibility, Stage III is designed to mosaic Stage II multisensor estimates from multiple radars on the national HRAP grid bounding the forecast area of each RFC independently, with the availability of several mosaicking algorithms. Stage III was subsequently superseded by improvements found in the Muti-sensor Precipitation Estimator (MPE) (Seo, 1998; Seo et al., 1999; Breidenbach et al., 1999; Breidenbach and Bradberry, 2001; Kitzmiller et al., 2013; Eldardiry et al., 2017). Gridded MPE (and, previously, Stage-III) precipitation radar rainfall fields are used by most NWS RFCs, which are responsible for flood forecasting in the United

1.2. QPF errors Difficulties with the prediction of future precipitation, referred to as quantitative precipitation forecast (QPF), are compounded by the need to accurately predict occurrences of heavy precipitation accumulations spatially. That is, the location of flood producing rainfall matters significantly, which is demonstrably evident with flash flood scale events, where the occurrence of excessive rainfall, accompanied by flooding, can be hit-or-miss over very short distances with devastating outcomes (Smith et al., 1996; Baeck and Smith, 1998; Smith et al., 2000; Borga et al., 2010; Alfieri et al., 2011; Broxton et al., 2014). 1.3. Temperature errors We have purposefully omitted any consideration of wintertime effects, which can be considerable, to limit the scope of this study. With winter time storms, errors in temperature estimation and prediction can incorrectly identify the physical state of hydrometeors, suggesting the occurrence of rainfall rather than snowfall, or the reverse (Wayand et al., 2017). Mizukami et al. (2013) and Hunter and Holroyd (2002) discuss the implications of mis-typing the physical state of precipitation (rain, snow, ice, hail, etc.) and how such errors lead directly to hydrologic forecast error. Moine et al. (2013), Rössler et al. (2014), and Wayand (2016) show that errors in the estimation of snow accumulation and snow water equivalent become especially problematic during rain-on-snow and significant temperature-driven snowmelt events. With temperature-index based snow models, such as the NWS Snow Accumulation and Ablation model, SNOW-17 (Anderson, 1973), used by NWS RFCs, erroneous temperature estimates can lead to inaccurate snowmelt rates. The effects of wind, terrain, and vegetation on snow estimation (Winstral et al., 2002; Essery and Pomeroy, 2004) and modeling (Essery et al., 1999, Xiao et al., 2000; Bowling et al., 2004; Liston and Elder, 2006) are significant as well. 1.4. Study goals Thiboult et al. (2016) identify three broad sources of total hydrologic forecast uncertainty, namely (1) model structure, (2) initial conditions, and (3) forcing uncertainties. This study focuses on the latter two sources of hydrologic forecast uncertainty. QPE and QPF errors over the 2

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 1. NEXRAD WSR-88D radar locations (black circles) in the NOAA/NWS OHRFC area of forecast responsibility. Refer to Table 1 for details. Also shown are 796 OHRFC modeling subbasins (light gray outlined areas) modeled operationally within the CHPS-FEWS hydrologic forecasting system and, for reference, the Ohio River and major tributaries (black lines). The Greenbrier River basin, WV, discussed below, is shaded gray.

a controlled monte carlo experiment to address a single issue: how should changes in hydrologic response reflect improvements in WPC QPF on the basis spatial placement? Accurate placement of QPF over a watershed is critical in hydrologic forecasting. The results of the monte carlo simulation experiment should demonstrate the range in hydrologic forecast uncertainty associated with past and current expected levels of QPF accuracy. The monte carlo experiment utilizes observed precipitation from a major rainfall event as a surrogate for possible QPF realizations in order to control confounding factors that contribute to QPF error. A summary and discussion of the limitations of the work, as well as implications to hydrologic forecasting, and final conclusions are presented in Section 4.

NOAA/NWS Ohio River Forecast Center (OHRFC) area of responsibility, shown in Fig. 1, are analyzed to identify improvements in QPE and QPF accuracy since 1997 and earlier with QPF. The central question is how QPE and QPF advancements through AHPS implementation have impacted OHRFC hydrologic forecast accuracy. We address the efficacy of AHPS improvements through hydrologic simulation experiments, where hydrologic forecast errors are evaluated with respect to changes in QPE and QPF forcing errors over the periods of record within the OHRFC area of responsibility. QPE and QPF errors are independent of each other and have independent, but similar effects on hydrologic forecast accuracy. We assess the effect of QPE and QPF errors on hydrologic modeling or forecast uncertainty separately. Consequently, in Section 2 we examine the spatial bias patterns of Stage III/MPE precipitation estimates and changes over time for the OHRFC area. A demonstrable improvement in QPE should be reflected in improvements in hydrologic simulations over the period of record. A historical simulation for a representative watershed, using Stage III/MPE precipitation estimates as the principal model forcing, is utilized to investigate possible improvements in hydrologic model simulation resulting solely from changes in precipitation estimation. Improvements in QPE are important for both hydrologic model state initialization and near-term forecasting related to the lag in basin response for most watersheds, which is independent of QPF. Section 3 addresses QPF skill and implications to hydrologic forecasting. Lack et al. (2010) and Rempel et al. (2017), and Gilleland (2019) propose the use of object-based metrics for verification, which includes error measures of (1) magnitude, (2) spatial placement, and (3) spatial pattern of forecasted meteorological fields, including future precipitation. We present results of NOAA/NWS Weather Prediction Center (WPC) and National Precipitation Verification Unit (NPVU) QPF verification to establish a baseline of QPF accuracy spanning the verification period of record. Our principal aim, however, is to quantify expected improvements in hydrologic prediction error that follow directly from QPF accuracy improvements. Consequently, we constructed

2. QPE biases The OHRFC, shown in Fig. 1, has produced radar-based precipitation estimates derived from the NEXRAD network of Weather Surveillance Radar-1988 Doppler (WSR-88D) radars (Crum and Alberty, 1993) since 1996. NEXRAD radars utilized by the OHRFC are shown in Fig. 1 and are listed in Table 1. However, use of NEXRAD data as model forcings for operational hydrologic forecasting did not begin immediately due to significant changes in the OHRFC operational hydrological environment, which included operational implementation of the AWIPS (NRC, 1997) and the NWS River Forecast System (U.S. Department of Commerce, 1972; Adams, 2016). 2.1. QPE timeline Although Stage II/III products were generated beginning in 1996 at the OHRFC, these data were not used in hydrologic forecast operations until 1998. A timeline of the period of use of Stage II & III and MPE by the OHRFC is shown in Fig. 2. An early significant MPE improvement, identified in Fig. 2, was the correction of an algorithmic error that produced truncated rain-rate values, leading to precipitation under3

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Table 1 NEXRAD WSR-88D locations used by the OHRFC in the Stage III and MPE, with the radar commissioning date, ground elevation, and tower height, listed in order of the commissioning date. Location

Longitude

Latitude

Elevation (m)

Sterling, VA St. Louis, MO Louisville, KY Chicago, IL Pittsburgh, PA Cleveland, OH Detroit, MI State College, PA Indianapolis, IN Wilmington, OH Morristown, TN Nashville, TN Blacksburg, VA Charleston, WV Paducah, KY Lincoln, IL Grand Rapids, MI Buffalo, NY Jackson, KY North Webster, IN Ft. Campbell, KY Evansville, IN

−77.4781 −90.6828 −85.9439 −88.0842 −80.2183 −81.8600 −83.4719 −78.0047 −86.2800 −83.8214 −83.4022 −86.5622 −80.2742 −81.7233 −88.7719 −89.3367 −85.5450 −78.7370 −83.3131 −85.7000 −87.2856 −87.7245

38.9753 38.6989 37.9753 41.6044 40.5317 41.4131 42.7000 40.9231 39.7080 39.4200 36.1681 36.2469 37.0239 38.3111 37.0683 40.1503 42.8940 42.9490 37.5908 41.3600 36.7370 38.2603

88.54 197.00 219.15 202.08 361.19 323.56 326.75 733.04 240.79 321.87 407.52 176.48 874.17 329.18 119.48 177.39 237.13 211.23 415.75 292.30 172.00 155.75

Tower Height (m)

Commisioning Date

ID

30 30 30 25 20 25 30 20 25 30 25 25 25 30 30 30 25 30 25 25 10 30

06/15/1994 07/15/1994 11/29/1994 12/16/1994 01/19/1995 02/09/1995 03/23/1995 04/06/1995 05/23/1995 06/01/1995 06/22/1995 07/06/1995 08/03/1995 08/24/1995 09/13/1995 01/03/1996 02/01/1996 04/04/1996 10/25/1996 03/17/1998 05/21/1998 12/16/2004

KLWX KLSX KLVX KLOT KPBZ KCLE KDTX KCCX KIND KILN KMRX KOHX KFCX KRLX KPAH KILX KGRR KBUF KJKL KIWX KHPX KVWX

estimation (Fulton et al., 2003). This error was corrected by the development and field deployment of the Open Radar Product Generator (ORPG) (primarily Builds 1, 3, and 4). Software Build 1 was deployed primarily over the period April-July 2002, Build 3 was deployed during the months April-July 2003, and Build 4, was delivered during the October-December 2003 period (Fulton et al., 2003). ORPG Build 1 contained the most significant improvements to precipitation estimation of the three OPRG software builds. Additional enhancements to NEXRAD precipitation processing have followed (Kitzmiller et al., 2011; Kitzmiller et al., 2013), including the deployment of NEXRAD dual polarization in 2011, which was completed for the OHRFC region before June 2013. Fig. 2 also shows the addition of the KVWX and KHPX NEXRAD radars to those used by the OHRFC, shown in Fig. 1.

stereographic grid. PRISM estimates are developed at a 30-arcsec resolution in geographic (latitude-longitude) coordinates. Consequently, re-projection (Evenden, 1990) and spatial interpolation of monthly PRISM grids to the HRAP coordinate system is necessary for PRISMMPE/Stage III comparisons and analysis. Re-projection of the PRISM grids, employing a bi-linear interpolation algorithm, and bias analyses of the gridded fields use the Geographic Resource Analysis Support System (GRASS) Geographic Information System (GIS) (GRASS Development Team, 2016). The magnitude and spatial patterns of estimation error from Stage III and MPE precipitation estimates are calculated on a pixelby-pixel basis with respect to gridded PRISM estimates over the OHRFC area. MPE/Stage III bias with respect to PRISM for the masked OHRFC region, using Eq. (1):

2.2. Data analysis

bias =

We analyze QPE biases over annual, seasonal, and monthly time scales with respect to spatial bias patterns and mean field bias. The emphasis is to identify long-term spatial and temporal trends over the OHRFC area, which has implications for hydrologic forecasting accuracy improvement. Shorter, hourly or sub-hourly, biases have been analyzed previously by many authors, for example, Krajewski et al. (2003) and Peleg et al. (2013). The HRAP grid is nominally an ∼4.7-km resolution, polar

where bias = 1 is perfect agreement. Gridded HRAP monthly, seasonal, and annual precipitation totals are derived from OHRFC hourly xmrg (refer to Fulton (1998)) format files.

Stage III or MPE PRISM

(1)

2.2.1. Annual variability Making use of Eq. (1), we obtain Table 2 and Fig. 3, which shows the spatial bias pattern of OHRFC Stage III and MPE precipitation

Fig. 2. Timeline for OHRFC implementation of Stage III and MPE with changes to the NEXRAD network, with the addition of KVWX and KHPX radars (see Fig. 1), and precipitation processing algorithm changes. 4

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

which were greatly reduced when MRMS was introduced to OHRFC MPE precipitation processing late in 2012, shown in Fig. 3. Bias variations viewed as an annual series in Fig. 4(a) using boxplots (see Eqs. (2)–(4), in Appendix A) show marked bias reduction from 1997 to 2005. With the introduction of the use of MRMS in late 2012, the variance in bias is substantially reduced over previous years and by 2015–2016 median biases are close to 1. MPE bias results are presented in Fig. 4(b) for the Greenbrier River basin, WV, identified in Fig. 1. Temporal bias patterns over the Greenbrier River basin are presented specifically to aid in the discussion of hydrologic modeling gains that result from MPE estimation improvements, which is discussed in Section 2(2.2)(2.2.4) on hydrologic modeling impacts.

Table 2 Annual OHRFC Stage III/MPE bias statistics by year. Year

Minimum

Maximum

Mean

Standard Deviation

Variance

1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016

0 0 0 0 0 0 0.1913 0.1912 0.4562 0.4018 0.5480 0.4411 0.5606 0.6004 0.6494 0.6757 0.5886 0.7285 0.6361 0.6718

1.4334 1.3750 2.2144 1.2112 1.3814 1.3685 2.0946 1.6529 1.7219 1.5189 1.3898 1.5465 1.3600 1.4369 1.4369 1.9550 1.6463 1.4086 1.2657 1.3090

0.6752 0.8176 0.7328 0.8800 0.9008 0.9382 0.9150 0.9266 1.0100 0.9961 0.9790 0.9884 0.9863 1.0005 1.0421 1.0539 1.0965 1.0936 1.0319 1.0216

0.1134 0.1088 0.1319 0.0932 0.1102 0.1049 0.1019 0.1587 0.1028 0.1112 0.1031 0.1039 0.0971 0.1007 0.0863 0.0987 0.0909 0.0712 0.0569 0.0581

0.0129 0.0118 0.0174 0.0087 0.0122 0.0110 0.0104 0.0252 0.0106 0.0124 0.0106 0.0108 0.0094 0.0101 0.0074 0.0097 0.0083 0.0051 0.0032 0.0034

2.2.2. Seasonal variability Seasonal variability of MPE biases is presented for the years 2015 and 2016 only because these years reflect current best estimates and precipitation processing performance. Summer months are defined as June-July-August (JJA) and winter months as December-JanuaryFebruary (DJF). Seasonal bias values are calculated as monthly JJA and DJF averages using Eq. (1), which are shown in Fig. 5 (a). Bias patterns during the summer months exhibit more of a random pattern compared to the winter months for 2015–2016. These differences are expected due to the prevalence of more isolated, convective rainfall during the summer months and wide-spread, stratiform and synoptic-scale precipitation during the winter months. Also evident during the 2015 and 2016 winter seasons is widespread MPE over-estimation in central and east-central regions of the OHRFC forecast area, quite possibly due to brightband influences (Gourley and Calvert, 2003; Cunha et al., 2013). Martinaitis et al. (2015) examine difficulties with verification and calibration of winter-time, radar-derived QPE, such as MRMS, from gauge based observations related to below-freezing surface wet-bulb temperatures. Such influences could potentially affect winter-time bias estimated in the OHRFC area. Clear differences are seen between winter and summer season biases for the 2015–2016 period with bias density shown in Fig. 5(b). Summer

estimates on an annual basis from January 1, 1997 through December 31, 2016. Two features should be evident, namely that (1) Stage III and MPE precipitation estimates are significantly under-estimated with respect to the PRISM estimates beginning in 1997, but improve significantly by 2002; and (2) the character of the spatial bias pattern changes from an apparent random variation (1997–2001) to one that exhibits distinct polygonal artifacts (2002–2011), to a pattern showing more of a random character (2012–2016). The changes to the bias patterns can be directly attributed to changes in the method used for bias correction initially in Stage III (1997–2001), then MPE (2002–2011), and finally with MPE utilizing initial MRMS estimates (2012–2016). There are also clear indications of persistent beam blockage within the OHRFC area for MPE estimates (2002–2011),

Fig. 3. Spatial pattern of Stage III/MPE precipitation estimate biases with respect to PRISM over the OHRFC forecast area of responsibility, 1997–2016. 5

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 4. Annual time-series of Stage III/MPE precipitation estimate biases with respect to PRISM over (a) the entire OHRFC forecast area of responsibility and (b) only the Greenbrier River basin at Alderson, West Virginia (see Fig. 1), 1997–2016. The horizontal gray line is used for reference with bias = 1.

Fig. 5. OHRFC spatial pattern of MPE precipitation estimation bias by season (a) and bias density by season (b), summer (JJA) and winter (DJF), with respect to PRISM for 2015–2016.

6

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

2.5 2.0 1.5 1.0 0.0

0.5

Stage−3/MPE Bias (xmrg/PRISM)

3.0

3.5

NOAA/NWS OHRFC Stage−3/MPE Precipitation Estimate Bias by Month with respect to PRISM 2015 − 2016

2015−01

2015−03

2015−05

2015−07

2015−09

2015−11

2016−01

2016−03

2016−05

2016−07

2016−09

2016−11

Year−Month

Fig. 6. Monthly time-series of MPE biases with respect to PRISM over the OHRFC forecast area of responsibility, 2015–2016. The horizontal gray line is used for reference with bias = 1.

confounding influences from other factors. Initial, a priori, estimation of RDHM Sacramento Soil Moisture Accounting (SAC-SMA) (Burnash et al., 1973; Burnash, 1995) model parameters is described by Koren et al. (2000). Parameter estimation includes 12 SAC-SMA parameters and several channel routing parameters. RDHM simulations also include use of the SNOW-17 snow accumulation and ablation model to account for wintertime precipitation and snowmelt. SNOW-17 model parameters are also estimated and the model is used uncalibrated. RDHM model simulations begin June 1, 1996 from a cold state, that is, without prior model state initialization that reflect existing basin conditions, using hourly NLDAS-2 precipitation and temperature model inputs. OHRFC Stage III and MPE are used for the full period January 1, 1997 to December 31, 2016. Verification of RDHM simulations is restricted to the January 1998 to December 2016 period, to allow sufficient RDHM SAC-SMA model warm-up, nearly 18 months. Simulation results Using USGS measured discharges and RDHM historical simulation, goodness-of-fit statistics, such as Nash-Sutcliffe Efficiency (NSE), Kling-Gupta Efficiency (KGE), ME, RMSE, R2 , MAE, Normalized Root Mean Square Error (NRMSE), and Percent Bias (PBIAS) are calculated, using Eqs. (5)–(14) (see Appendix A), to assess hydrologic modeling improvement based on MPE improvements. Analyses of the historical flows simulation for the 1998–2016 period compared to USGS measured flows are summarized graphically in Fig. 7. With the exception of Mean Absolute Error (MAE), which is relatively unchanged at about 25 m3 s−1 and Root Mean Square Error (RMSE) at about 55 m3 s−1, all other measures indicate improvement between observed and simulated flow values over the 1998–2016 retrospective simulation period. Improvements in values for Mean Error (ME) (from −30.02 to 6.34), Nash-Sutcliffe Efficiency (NSE) (from 0.44 to 0.84), Kling-Gupta Efficiency (KGE) (from −0.01 to 0.82), Coefficient of Determination (R2) (from 0.61 to 0.84), and Percent Bias (PBIAS) (from −48.60 to 10.00) are notable. Consequently, the results demonstrate that improvements in QPE estimation has lead to significantly improved hydrologic simulations for the Greenbrier River basin in the OHRFC area of forecast responsibility. A comparison of Figs. 4 and 7 shows the association of reduced Stage III/MPE bias to improved hydrologic simulation goodness-of-fit statistical measures for individual years. Also clear is that for years when Stage III/MPE biases are greatest, hydrologic simulation goodness-of-fit statistics worsen. QPF is another major model forcing used in hydrologic forecasting.

biases are very close to 1 and, while the winter season mean bias in not much different from 1, with slight over-estimation, the spread in bias values is much greater during the winter season compared to summer. 2.2.3. Monthly variability The analysis of monthly MPE biases, using Eq. (1), is based on monthly accumulations of hourly xmrg HRAP gridded fields. The bias analysis is restricted to the years 2015 and 2016, which best reflects the current state of MPE measurement accuracy. Fig. 6 shows significant month-to-month bias variability. Monthly median biases are very close to unity, interquartile (25–75 percentile) differences are generally small, but large outliers are evident. This points to the complex nature of both precipitation processes and difficulty in removing estimation biases at short time scales, suggesting continued challenges with correcting estimation biases in real-time. 2.2.4. Hydrologic modeling impacts Our central interest is knowing what benefits, if any, to hydrologic modeling and forecasting is evident from improvements in Stage III and MPE precipitation estimation. We address this issue by using hourly Stage III/MPE precipitation estimates and hourly temperature data obtained from the North American Land Data Assimilation System (NLDAS-2) (Xia et al., 2012a; Xia et al., 2012b) as the primary hydrologic model forcings for a continuous hydrologic simulation spanning the Stage III/MPE historical period of record, 1997–2016. Historical simulation To illustrate the benefits gained from precipitation estimation improvements, a retrospective hydrologic simulation is made using the NOAA/NWS Hydrology Laboratory Research Distributed Hydrologic Model (HL-RDHM) (Koren et al., 2004; Koren, 2006; Koren et al., 2010). The historical simulation spans the full period (1997–2016) of available Stage III/MPE, NEXRAD radar derived precipitation estimates. RDHM simulations are made for the Greenbrier River basin in West Virginia, shown in Fig. 1, with the model defined at the HRAP grid resolution, using an hourly time-step. The Greenbrier River basin defined at Alderson, WV (U.S. Geological Survey, USGS, 05050003), located in the valley and ridge physiographic province of the Appalachian Mountains, has an area of 3533 km2, and ranges in elevation from 466 to 1433 m. Retrospective RDHM simulations are made without prior calibration to avoid biasing model performance to any model period. In this way, model results will best reflect precipitation processing improvements implemented at the OHRFC without 7

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 7. Mean annual goodness-of-fit statistics for the uncalibrated RDHM historical simulation and USGS observed flows, (a) MAE (m3 s−1), ME (m3 s−1), NRMSE (%), PBIAS (%), and RMSE (m3s −1) and (b) NSE, KGE, and Coefficient of Determination (R2), for the Greenbrier River basin, for years 1998–2016.

In Section 3 we examine QPF error and the magnitude of hydrologic forecast error in response to QPF uncertainty.

Table 3 Contingency table for QPF Threat Score calculation. Observed

3. QPF errors Independent QPF verification is not attempted in this study. Instead, we report results from two sources, namely, the NOAA/NWS Weather Prediction Center and National Precipitation Verification Unit (NPVU) to identify trends in QPF accuracy. NPVU verification results provide additional verification measures beyond threat score (often referred to as critical success index (CSI)) and bias available from WPC, including mean absolute error, root mean square error, and correlation coefficient. Additionally, NPVU verification results are presented for several different sources, including several numerical weather prediction (NWP) model forecasts and RFCs, which serve as a useful baseline for comparison against WPC QPF. QPF verification comparing RFC, WPC, and NWP model results demonstrate added value from forecaster contributions. Novak et al. (2014) report improved WPC QPF performance from 1960 through 2012 for days 1, 2, and 3 lead-time, 24-h, 1 in (25.4 mm) forecasts. Methods used for forecaster generation of WPC QPF and comparisons relative to various NWP models are also presented. In

Forecast

Yes

No

Yes No

a c

b d

Novak et al. (2014), WPC QPF improvement is measured in terms of a threat score (TS), TS = a + b + c , from Table 3 (Wilks, 2006). Both WPC and NPVU report QPF verification statistics for a range of precipitation categories, including a maximum of ⩾1.0 in (25.4 mm) for NPVU and categories of ⩾1.0 in (25.4 mm) to < 2.0 in (50.8 mm) and ⩾2.0 in (50.8 mm) for WPC. Since the authors’ study is focused on flood producing precipitation events, it is argued that 24-h, 1.0 in (25.4 mm) precipitation threshold is, in most instances, too low to identify flood producing events. Consequently, the authors believe that an analysis restricted to higher intensity, flood-producing rainfall events, ⩾2.0 in (50.8 mm), is warranted and that WPC monthly QPF verification TS data for accumulations ⩾2.0 in (50.8 mm) is likely to be more relevant to addressing QPF performance relative to meeting hydrologic flood 8

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

accuracy in meteorology, will serve as the basis for evaluating QPF accuracy in the RDHM hydrologic experiments.

forecasting needs. Further justification for the use of ⩾2.0 in (50.8 mm) threshold to identify flood producing rainfall events is made on the basis of the association of rainfall frequency and peak runoff frequency. That is, Larson and Reich (1972) found that while there was high variability for individual years, the rank and recurrence interval of storm rainfall and peak runoff do have a central tendency of equality. So, by looking at rainfall frequency estimates from the NOAA/NWS Hydrometeorological Design Studies Center (Bonnin et al., 2006) in the Ohio River valley, we find that a 1-yr recurrence, 24-h rainfall accumulation generally exceeds 2.0 in (50.8 mm) and ranges between ∼50.8 mm to ∼76.2 mm (about 2–3 inches) for a 2-yr recurrence. By comparison, for shorter duration, higher intensity precipitation events, the 2-yr, 6-h rainfall accumulation is ∼50.8 mm (2-in) in the Ohio River valley. Bankfull conditions, the river stage level for incipient flooding, for small streams and rivers occur with a ∼1.5-yr recurrence (Leopold, 1994) while the mean annual flood is defined as a flood with a 2.33-yr recurrence (Dalrymple, 1960). So, our analysis of flood-producing QPF will be restricted to events with a 24-h QPF threshold of ⩾2.0 in (50.8 mm). Two important issues are, (1) the degree to which errors in operational QPF influence hydrologic prediction and (2) how these influences can be quantified. In an attempt to address these concerns, results from a hydrologic modeling study are presented, noting that:

3.1. WPC Monthly mean bias and TS data for precipitation accumulations ⩾2.0 in (50.8 mm), obtained from WPC (Rausch, 2016), are summarized in Fig. 8 for the period 1970–2015 for Day-1 and 1991–2015 for Day-2 lead-times. Day-2 data were not collected before 1991 by WPC. Annual averages are also shown, based on monthly averaged values. The large variability of monthly bias and TS values is evident. We note that expected Day-1, ⩾2.0 in (50.8 mm) TS values are about 0.06 in 1970 and 0.22 by 2015. 3.2. NPVU Charba et al. (2003) describe a methodology for evaluating QPF forecast accuracy relative to RFC produced Stage III and MPE QPE at WPC, RFCs, and Weather Forecast Offices (WFOs) compared against national guidance produced by operational NWP models run at the NCEP. Also described is the implementation of the National Precipitation Verification Unit (NPVU) to carry-out national QPF verification from the various sources (WPC, WFOs, RFCs, NWP models). NPVU QPF verification data was produced for the period 2001–2012. Fig. 9 shows a time series of NPVU monthly (a) Mean Absolute Error (MAE) and (b) Root Mean Square Error (RMSE) for all NOAA/NWS CONUS River Forecast Centers (rfc), NOAA/NWS Nested Gridded Model (ngm), North American Model (nam), Hydrometeorological Prediction Center (hpc) – now WPC – Global Forecast System (gfs), ETA Model (eta), and Aviation Model (avn) for the period June 2001 to December 2009. The gap in data from late 2003 to mid-2004 was due to an NPVU data processing failure. Several points are noTable (1) seasonal variability for all the QPF sources is clear; (2) while there are differences between QPF sources, there is relatively little discernible MAE or RMSE improvement over the June 2001 to December 2009 analysis period displayed by any of the QPF sources; and (3) errors for

1. We present QPF verification statistics taken from the NOAA/NWS National Precipitation Verification Unit (NPVU) covering the 12 Conterminous U.S. (CONUS) RFCs. The highest precipitation interval used by NPVU is ⩾1.0 in (25.4 mm) and is reported here to serve as a relative basis for comparison against the WPC ⩾1.0 in (25.4 mm) QPF category range, which is available athttp://www.wpc.ncep. noaa.gov/html/hpcverif.shtml; 2. Results from Monte Carlo simulations using the RDHM for the Greenbrier River basin, WV are presented spanning the June 22–24, 2016 flooding episode to assess the range of hydrologic errors in response to expected WPC QPF accuracy; 3. Threat Score (TS), a commonly used statistical measure of forecast

Fig. 8. WPC monthly and annual average QPF Bias and Threat Score, by year, for Day-1 (1970–2015) and Day-2 (1991–2015), for accumulations ⩾2.00 in (50.8 mm). 9

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

avn

1.0 0.5

a

0.0

eta

1.0 0.5 0.0

gfs

1.0 0.5

1.0

hpc

MAE

0.0

0.5 0.0

nam

1.0

Precipitation Interval (in) 0.00 0.01 0.01 0.10 0.10 0.25 0.25 0.50 0.50 1.00 >1.00

0.5 0.0

ngm

1.0 0.5 0.0

rfc

1.0 0.5 0.0 2002

2004

2006

2008

2010

Date 1.5

avn

1.0 0.5

b

0.0 1.5

eta

1.0 0.5 0.0 1.5

gfs

1.0 0.5

1.0

hpc

RMSE

0.0 1.5

0.5 0.0 1.5

Precipitation Interval (in) 0.00 0.01 0.01 0.10 0.10 0.25 0.25 0.50 0.50 1.00 >1.00

nam

1.0 0.5 0.0 1.5

ngm

1.0 0.5 0.0 1.5

rfc

1.0 0.5 0.0 2002

2004

2006

2008

2010

Date

Fig. 9. NPVU (a) Mean Absolute Error (MAE) and (b) Root Mean Square Error (RMSE), by month, for QPF thresholds, ranging from ⩽0.01 (0.254 mm) to ⩾1.00 in (25.4 mm), for the period June 2001 to December 2009 for all NOAA/NWS CONUS River Forecast Centers (rfc), NOAA/NWS Nested Gridded Model (ngm), North American Model (nam), Hydrometeorological Prediction Center (hpc) – now WPC – Global Forecast System (gfs), ETA Model (eta), and Aviation Model (avn).

3.2.1. Implications We can see that deterministic QPF can be quite erroneous, based on a range of statistical verification measures, demonstrated by both NWP model generated QPF and QPF produced with the aid of forecaster input. It has been reported (Novak et al., 2014), correctly, that by some statistical measures, specifically, threat score, QPF accuracy has improved since the beginning of systematic record keeping to the present. Nevertheless, the question must be asked, what benefits have accrued to hydrologic forecasting from QPF improvements? This question is addressed with a hydrologic simulation experiment using the near-record Greenbrier River basin (defined at Alderson, WV, USGS 05050003) flooding event in West Virginia, June 22–24, 2016. The Greenbrier River basin was selected for study due to the significance of the June 2016 event, its basin area, which is large enough to reflect variability in watershed response from spatially variable precipitation forcings, and its location within the OHRFC area of responsibility.

the larger intervals, ⩾1.0 in (25.4 mm), are approximately equal to 1.0 in (25.4 mm) for all QPF sources. The latter point shows that QPF errors are approximately equal in magnitude to the QPF forcings closest to levels that are most impactful for forecasting flood events. Fig. 10 shows the correlation coefficient, R, of QPF versus QPE, aggregated across the June 2001 to December 2009 NPVU analysis period, grouped by QPF source and precipitation interval. It is evident, as reported by Charba et al. (2003), that forecasters add value over NWP modeled QPF, based on HPC (WPC) and RFC results compared to NWP model results. Also apparent is that for 24-h accumulations, especially for the larger precipitation intervals, ⩾1.0 in (25.4 mm), QPF is poorly correlated with observed QPE and that there is considerable correlation spread within the precipitation intervals. Little difference is apparent between HPC and RFC QPF on the basis of R values and spread.

10

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond  ●

●

●

● ● ●



● ● ●

Correlation Coefficient

● ●

●

Precipitation Interval (in)

● ●

●



í í í í í !

● ● ● ●

●



●

í avn

eta

gfs

hpc

nam

ngm

rfc

Forecast Source

Fig. 10. NPVU Correlation Coefficient (R) for QPF thresholds, ranging from ⩽0.01 (0.254 mm) to ⩾1.00 in (25.4 mm), for the period June 2001 to December 2009 for all NOAA/NWS CONUS River Forecast Centers (rfc), NOAA/NWS Nested Gridded Model (ngm), North American Model (nam), Hydrometeorological Prediction Center (hpc) – now WPC – Global Forecast System (gfs), ETA Model (eta), and Aviation Model (avn).

Precipitation (mm)

Threat Score

a

b

Fig. 11. OHRFC forecast area of responsibility (a) (blue shading) showing 1000 randomly generated locations for QPF transposition of the 24-h precipitation accumulation for amounts ⩾50.8 mm (2.0 in) from June 23, 2016 07 UTC to June 24, 2017 06 UTC. Points identifying transposition locations with Threat Scores ⩾0.06 are colored yellow to purple; values < 0.06 are filled white. A closer view (b) shows the reference location, used for storm transposition (identified with a red cross), which is the location of the maximum 24-h precipitation.

accumulation for the period beginning June 23, 2016 0600 UTC within the region, shown in Fig. 11 (a). An example transposed storm is shown in Fig. 12, which shows that all details of the observed MPE field are retained. The June 23, 2016 0600 UTC to June 24, 2016 0600 UTC period was selected because this was the greatest MPE precipitation accumulation for a 24-h synoptic period during the flooding event. The transposed individual hourly grids of the 24-h MPE precipitation serve as QPF for the RDHM simulations. Fig. 11 (b) shows the location of the maximum 24-h MPE grid cell precipitation accumulation, as well as some of the 1000 randomly transposed storm centers closest to the storm center maximum. Only precipitation grid cells with MPE accumulations ⩾50.8 mm (2.0 in) are shown in Fig. 11 (a) and (b). TS values are calculated for each of the randomly transposed 24-h storms based on precipitation amounts ⩾50.8 mm (2.0 in). From these, only storms with TS⩾0.06 are used in the RDHM monte carlo simulations because we are interested in identifying improvement in the

3.3. Hyrdrologic simulation experiments A hydrologic simulation experiment is used to assess how QPF improvements have impacted hydrologic prediction. We use the observed rainfall field from the June 2016 event as if it were QPF, so that if the ’QPF’ were to exactly coincide with the observed precipitation, QPF TS would equal 1.0 and the simulated peak flow would match the observed peak flow, since the RDHM had been calibrated to this event. An underlying assumption is that the observed MPE rainfall perfectly estimates the actual rainfall, which is, of course, unknown. Simulations take the form of a monte carlo experiment, using a storm transposition methodology (Foufoula-Georgiou (1989); England et al. (2014); Wright et al. (2014)). The aim of the experiment is to illustrate the inadequacy of current deterministic QPF for hydrologic forecasting purposes. The experiment first produces 1000 randomly located storm centers relative to the location of the maximum 24-h MPE grid cell precipitation 11

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 12. Example of a transposed storm (shaded blue) relative to the observed MPE storm (yellow); the green region shows overlap between the observed MPE and transposed storm. Also shown are the OHRFC forecast area of responsibility (light blue shading) and 1000 randomly generated locations for QPF transposition of the 24-h precipitation accumulation for amounts ⩾50.8 mm (2.0 in) from June 23, 2016 07 UTC to June 24, 2017 06 UTC. Points identifying transposition locations with Threat Scores ⩾0.06 are colored yellow to purple; values < 0.06 are filled white. The reference location, identified with a red cross, is the location of the maximum 24-h precipitation, from which storm transpositions are made. The heavy black line indicates the transposition vector.

prediction of hydrologic peak flows resulting from WPC QPF improvements since 1970 when WPC TS ≈ 0.06, at the beginning of WPC verification of precipitation accumulation for amounts ⩾50.8 mm (2.0 in). Consequently, 88 randomly transposed storms, shown in Figs. 11 (a) and (b), were identified, meeting the TS ⩾0.06 threshold, and are used in the RDHM monte carlo simulation experiment. The RDHM was initialized from model warm states, that is, model states generated from the 1996–2016 historical simulation discussed previously. However, for the purpose of the monte carlo simulation experiment only, the RDHM was calibrated using a simple trial-and-error process for a few key model parameters against U.S. Geological Survey observed discharge values for the Greenbrier River at Alderson, WV, (USGS 05050003) for the June 22–24, 2016 event. The aim of the RDHM calibration was simply to produce good agreement between observed and simulated peak flows using the observed MPE precipitation for the June 22–24, 2016 event, not comprehensive model calibration, which is an optimization process, that would not necessarily produce good agreement between observed and simulated values for this event. RDHM model calibration here helps to provide hydrological context for the experimental results relative to USGS observed flows for the June 22–24, 2016 event.

0.15–0.25, that reflect WPC QPF skill for Day-1, ⩾50.8 mm (2.0 in) precipitation accumulations in 1970 and 2016, respectively. By doing this we believe we can assess, in relative terms, the gain in hydrologic forecast accuracy reflected by improvements in WPC QPF. It is evident from the Fig. 13 simulations that hydrograph peaks are, in general, greater for storms with TS values ranging 0.15–0.25 than for storms with TS values 0.06–0.15. However, none of the simulated hydrographs exceed the Major Flood level and none approach the observed nearrecord flood peak level. Moreover, there is considerable variability between the simulated hydrographs within the separate 0.06–0.15 and 0.15–0.25 TS categories. A better perspective on the problem inherent with using deterministic QPF in hydrologic forecasting is found by looking at simulation results from all 88 storms used in the RDHM monte carlo experiment with TS⩾0.06. Fig. 14 shows peak flow and storm TS relative to distance from the reference storm center maximum of the observed 24-h MPE, with points identified by color, reflecting storm TS value ranges. Point size indicates the peak flow magnitude. The high degree of peak flow variability within TS categories is illustrated in Fig. 15, which is greatest for the 0.30–0.49 TS interval. Also quite evident is that smaller distances of the transposed storm from the reference storm center does not guarantee either higher TS values or peak flows. In fact, the storm with the highest TS and closest to the reference storm center produced a peak flow approximately the same as other transposed storms with significantly lower TS values and at distances much further from the reference storm center. The monte carlo experimental results show that a large range in

3.3.1. Monte carlo experiment results Fig. 13 shows the USGS observed flows, simulated RDHM flow hydrograph from the calibration, and simulated flows from the transposed storms with TS values ranging 0.06–0.15 and 0.15–0.25. These TS ranges were used to identify reasonable TS value ranges, 0.06–0.15 and 12

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 13. Flow hydrographs for the June 23, 2016 07 UTC to June 25, 2017 12 UTC model experiment period for the Greenbrier River at Alderson, WV, showing USGS observed flows (black circles), RDHM simulated hydrographs derived from observed MPE precipitation (blue circles), and the experimental QPF for Threat Score ranges 0.06–0.15 (cyan lines) and 0.15–0.25 (magenta lines). For reference, the Minor and Major Flood levels are shown as horizontal orange and purple lines, respectively.

Fig. 14. Threat Scores (TS⩾0.06) of 88 randomly transposed QPF instances with respect to distance from a reference location (see Fig. 11 (b)). Maximum flows derived from RDHM simulations are shown by point size and threat score range, by color, for the Greenbrier River at Alderson, WV, within the June 23, 2016 07 UTC to June 25, 2017 12 UTC model experiment period. 13

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 15. Comparison of RDHM simulated peak flows for QPF for Threat Score ranges 0.06–0.15, 0.15–0.25, 0.25–0.30, and 0.30–0.49. For reference, the Minor and Major Flood levels are shown as horizontal orange and purple lines, respectively for the Greenbrier River at Alderson, WV, within the June 23, 2016 07 UTC to June 25, 2017 12 UTC model experiment period. The USGS observed peak flow is indicated as a red line.

4. Summary and conclusions

hydrologic prediction errors result from QPF with TS values at current expected WPC TS levels, TS≈0.22. Much greater hydrologic prediction errors are found with TS≈0.50. Consequently, these results suggest that deterministic QPF is far too uncertain to be used in operational flood forecasting. The large variability of hydrograph response and peak flow relative to the magnitude of the transposed storm TS values, underscores the complex nature of hydrologic response to spatially and temporally variable precipitation forcings. As discussed in Section 1, previous studies have shown that hydrologic response to differences in the location of heavy cores of observed precipitation within a watershed can produce significantly different peak flows. Consequently, large differences in the prediction of flood peak magnitudes using deterministic QPF should be expected simply due to errors in QPF placement relative to a watershed of interest. The consequence of QPF spatial placement errors is reflected in errors in the prediction of hydrograph peaks in the monte carlo experiment. This is reflected in the findings by Rezacova et al. (2009) and Mittermaier and Roberts (2010) who address the difficulty of QPF verification due to the complex structure of observed precipitation fields due to embedded convection.

Results presented demonstrate that (1) NOAA/NWS NEXt generation RADar (NEXRAD) derived QPE has improved dramatically from 1997present for the OHRFC area, which is reflected in significantly improved hydrologic simulations over the 1997–2016 hindcast period and that (2) from the perspective of meeting the needs of hydrologic forecasting, QPF improvements have been marginal over the same period. The monte carlo hydrologic simulation experiment illustrates the sensitivity of hydrologic forecasts to QPF errors, resulting in large peak flow differences within narrow ranges of QPF TS differences. Results from these experiments show that greater QPF TS values do not necessarily produce improved hydrologic forecasts and that considerable variability in hydrologic response should be expected with QPF TS values the same or nearly so, independent of antecedent basin conditions. Improved hydrologic simulations resulting from QPE improvements are important in several ways. First, in large part, hydrologic forecasts are improved with better QPE because of the ability to accurately translate observed precipitation into watershed response through modeling. Second, improved QPE improves model states that better reflect current basin conditions, which reduces the need for an external data assimilation process for model state updating. Furthermore, the need for real-time, ad hoc forecaster adjustments is reduced, which reduces hydrologic forecaster workloads to make model adjustments and decreases the possibility that model adjustments could be made for inappropriate reasons. That is, an observed difference between simulated and observed flows/stage could simply reflect random model error and not indicate divergence between model states and actual basin conditions, thus not warranting curve-fiiting changes to hydrologic model states. Finally, for well-calibrated models, improved QPE provides greater confidence in the initial model states needed for probabilistic hydrologic forecast systems (Cloke and Pappenberger, 2009) that run automatically, without direct forecaster intervention. There are some indications of improved QPF accuracy, shown by

3.3.2. WPC QPF for the June 23, 2016 event As a point of reference, we report results from a RDHM simulation using June 23, 2016 0600 UTC, 24-h WPC QPF. The simulated peak flow was 143 m3 s−1 with TS = 0.30 for WPC QPF for this event. Readers are referred to Fig. 13 for reference. Importantly, no 24-h WPC QPF ⩾50.8 mm (2.0 in), for the June 23, 2016 0700 UTC to June 24, 2016 0600 UTC period, fell within the Greenbrier River basin, shown in Fig. 16. The USGS observed peak flow value, 2285.2 m3 s−1, at Alderson, WV occurred June 24, 2016 0930 UTC. The OHRFC forecast from June 23, 2016 1423 UTC was 180.7 m3 s−1, corresponding to a peak river stage 5.70 feet (1.74 m), which was forecasted to occur June 24, 2016 0600 UTC (Corrigan, 2016).

14

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Fig. 16. Comparison of WPC QPF (blue shades) versus the spatial extent of actual precipitation (yellow regions) for ⩾50.8 mm (2.0 in), for the 24-h period ending June 24, 2016 0600 UTC. The Greenbrier River basin is shown as a heavy black outline.

points to the need for using hydrologic prediction methods that explicitly address the uncertainty in model forcing predictions. Companion research to the current study by Adams and Dymond (2019) investigates limits to the operational use of deterministic WPC QPF, especially as it relates to flood forecasting. Utilizing real-time operational experiments, the authors find that for flood forecasting purposes, QPF durations should not exceed about 12-h. Errors in hydrologic flood forecasts utilizing QPF beyond a 12-h duration grow rapidly. A major source of the real-time, QPF based hydrologic prediction errors found by Adams and Dymond (2019) is identified in this study, namely, the spatial placement of QPF as indicated by TS values. Significant research and development activity (Cloke and Pappenberger, 2009; Adams and Ostrowski, 2010; Demargne et al., 2014) in the hydrologic community has been directed at the implementation of probabilistic/ensemble hydrologic forecast systems with a principal aim to quantify the uncertainties inherent in hydrological forecasting, including uncertainties associated with model forcings, in particular, QPF uncertainties. Consequently, the authors strongly encourage abandoning the use of deterministic QPF in hydrologic forecasting in favor of probabilistic/ensemble methods.

WPC with increased threat score values from 1970-present. However WPC QPF bias statistics do not show improvement. In addition, for a variety of QPF sources, NPVU verification statistics do not reveal discernible improvement for the 2001–2009 analysis period. The RDHM monte carlo experiments demonstrated significant uncertainty in hydrologic response at current expected WPC QPF threat score levels. Importantly, significant uncertainty with deterministic QPF has been widely shown to be problematic (Damrath et al., 2000; Ebert, 2001; Im et al., 2006; Diomede et al., 2008; Cuo et al., 2011). Mascaro et al. (2010) illustrate the need for reliable ensemble QPFs in the context of ensemble streamflow forecasting. The central problem with deterministic QPF is that, in principle, the placement, timing, and magnitude of QPF should all be reasonable estimates for each of the basins shown, for example, in Fig. 1. The RDHM monte carlo hydrologic experiment demonstrates how far current deterministic QPF is from consistently providing the needed skill in hydrologic forecasting. It is only for much larger basins where QPF errors are sufficiently masked by spatial and temporal averaging that deterministic QPF has sufficient skill to have value in hydrologic forecasting. The problem is significantly more difficult for flash floods, with affected areas that are considerably smaller than the OHRFC subbasins shown in Fig. 1, suggesting limits to predictability in real-time hydrologic forecasting. Namely, Figs. 14 and 15, for the Greenbrier River basin (which is comprised of 5 subbasins shown in Fig. 1), illustrate that large hydrologic prediction uncertainty can occur with the use of QPF over medium size watersheds even when QPF threat score levels are high. Moreover, as Fig. 16 shows, WPC QPF entirely missed the placement of ⩾50.8 mm (2.0 in) rainfall within the Greenbrier River basin for the June 23, 2016 0700 UTC to June 24, 2016 0600 UTC period, where near-record setting flooding occurred. This level of QPF error is not at all uncommon as WPC and NPVU verification statistics demonstrate. Fig. 17, showing a histogram of OHRFC subbasin areas, identifies the magnitude of the QPF problem in terms of accurate spatial placement of QPF, where the mean OHRFC subbasin area is 613.5 km2 and median is 476.1 km2 , which are considerably smaller than the Greenbrier River basin (3533 km2). Hydrologic prediction for small basin areas makes the problem with QPF accuracy altogether more acute and

Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments The authors are grateful to Robert “Marty” Rausch at WPC for providing WPC QPF verification statistics, Bill Lawrence at the Arkansas-Red Basin River Forecast Center (ABRFC) for making recent CONUS MPE data available, and Peter Corrigan NOAA/NWS WFOBlacksburg for providing OHRFC RVF products. We also thank the anonymous reviewers for their comments, which helped to improve the manuscript.

15

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Histogram of OHRFC Subbasin Area

Count

75

50

25

0 0

1000

2000

3000

Area (sq km)

Fig. 17. Histogram showing the distribution of subbasin areas for the OHRFC forecast area of responsibility.

Appendix A. Verification equations Analyses utilize R (R Core Team, 2017) verification measures and statistical analyses from the verification (NCAR, 2015) and hydroGOF (Zambrano-Bigiarini, 2014) contributed packages. For R boxplots we have,

IQR = Q3 − Q1

(2)

UpperWisker = min (max (x ), Q3 + 1.5IQR)

(3)

LowerWisker = max (min (x ), Q1 + 1.5IQR)

(4)

where IQR is the interquartile range (box length in R boxplots) of a set of values, x, and Q1 and Q3 are, respectively, the 25th and 75th percentiles (lower and upper quartiles).

ME =

1 n

n

∑ k=1

⎛ ⎞ ⎜yk − ok ⎟ ⎝ ⎠ n

⎞ ⎛ ⎜yk − ok ⎟ ⎠ ⎝

∑ k=1

PBIAS = 100·

(5)

n

∑

(ok )

(6)

k=1

MAE =

RMSE =

1 n

n

⎛ ⎞ ⎜ yk − ok ⎟ ⎝ ⎠

∑ k=1

1 n

(7)

n

∑

(yk − ok )2

(8)

k=1

n 1 n

NRMSE = 100·

∑

(yk − ok )2

k=1

range

(9)

With paired data, namely, predicted, yi , and observed, x i , we have (x1, y1), ⋯, (x n , yn ) and model,

16

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Yj = β0 + β1 x j + ∊ where ∊ is random noise with  ∊ = 0 and Var ∊ = σ 2 . No distribution for ∊ is assumed other than its mean is zero. It is noted that Yj = β0 + β1 x j and Yj = σ 2 . Least squares estimation. β0 and β1 are estimated by minimizing the sum of the squared errors: n

∑

(yj − β0 − β1 x j )2.

j=1

Consequently, we get: n

SSE =

yj2 − β0̂

∑ j=1

n

∑

n

yj − β1

j=1

∑

x j yj (10)

j=1

n

SST =

∑

(yj − y¯)2 (11)

j=1

R2 = 1 −

SSE SST

(12)

where we have the Mean Error (ME), Percent Bias (PBIAS), Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Normalized Root Mean Square Error (NRMSE), and Coefficient of Determination (R2 ), with quantities yk and ok the predicted and observed kth values, respectively, for n total paired values; range = max (ok : k = 1, …, n) − min (ok : k = 1, …, n) . The Nash-Sutcliffe Efficiency (NSE) (Nash and Sutcliffe, 1970), for T periods, where Qo is the observed discharge, Qm is the modeled discharge, and Q0t is the observed discharge at time t, can range from − ∞ to 1. An efficiency of 1 (NSE = 1) corresponds to a perfect match of modeled discharge to the observed data. An efficiency of 0 (NSE = 0) indicates that the model predictions are as accurate as the mean of the observed data; values of NSE less than zero (NSE < 0) occurs when the observed mean is a better predictor than the model. Units of measure for river flow are m3 s−1, unless reported otherwise. Values for ME, MAE, and RMSE = 0 implies perfect agreement, i.e., no error. T

∑ NSE = 1 −

(Qmt − Qot )2

t=1 T

∑

(Qot − Qo )2

(13)

t=1

The Kling-Gupta Efficiency (KGE) (Gupta et al., 2009) is given by:

KGE = 1 −

(r − 1)2 + (α − 1)2 + (β − 1)2

(14)

where r = covso/ σs·σo, covso is the covariance between the simulated and observed flows, σs and σo are, respectively, the standard deviations of the simulated and observed flows, α = σs / σo , and β is the ratio between the mean simulated and mean observed flows (or bias).

flash Floods: Observations and Analysis of Hydrometeorological Controls. URL:https://www.sciencedirect.com/science/article/pii/S0022169410004877. Bowling, L.C., Pomeroy, J.W., Lettenmaier, D.P., 2004. Parameterization of blowing-snow sublimation in a macroscale hydrology model. J. Hydrometeorol. 5 (5), 745–762. Breidenbach, J., Bradberry, J., 2001. Multisensor precipitation estimates produced by National Weather Service forecast centers for hydrologic applications. In: Proc., 2001 Georgia Water Resources Conf. Athens, GA, pp. 179–182. Breidenbach, J., Seo, D.-J., Tilles, P., Roy, K., 1999. Accounting for radar beam blockage patterns in radar-derived precipitation mosaics for River Forecast Centers. In: Proc., 15th Conf. on Interactive Information Processing Systems, Boston, MA, pp. 179–182. Broxton, P., Troch, P.A., Schaffner, M., Unkrich, C., Goodrich, D., 2014. An all-season flash flood forecasting system for real-time operations. Bull. Am. Meteorol. Soc. 95 (3), 399–407. Burnash, R., 1995. The NWS River Forecast System – Catchment Model, first ed. Water Resources Publications. Burnash, R., Ferral, R., McGuire, R., 1973. A generalized streamflow simulation system: Conceptual modeling for digital computers. U.S. Department of Commerce National Weather Service and State of California Department of Water Resources. Cecinati, F., Rico-Ramirez, M.A., Heuvelink, G.B., Han, D., 2017. Representing radar rainfall uncertainty with ensembles based on a time-variant geostatistical error modelling approach. J. Hydrol. 548 (Supplement C), 391–405. URL:http://www. sciencedirect.com/science/article/pii/S0022169417301324. Charba, J.P., Reynolds, D.W., McDonald, B.E., Carter, G.M., 2003. Comparative verification of recent quantitative precipitation forecasts in the national weather service: a simple approach for scoring forecast accuracy. Weather Forecasting 18 (2), 161–183. Ciach, G.J., 2003. Local random errors in tipping-bucket rain gauge measurements. J Atmos. Oceanic Technol. 20 (5), 752–759. Ciach, G.J., Krajewski, W.F., 1999. Radar-rain gauge comparisons under observational uncertainties. J. Appl. Meteorol. 38 (10), 1519–1525. Cloke, H., Pappenberger, F., 2009. Ensemble flood forecasting: a review. J. Hydrol. 375 (3), 613–626. URL: http://www.sciencedirect.com/science/article/pii/ S0022169409003291. Corrigan, P., 2016. Email June 29. personal communication, weather Forecast Office – Blacksburg, VA, NOAA. National Weather Service.

References Adams, T., Ostrowski, J., 2010. Short lead-time hydrologic ensemble forecasts from numerical weather prediction model ensembles. In: Proceedings World Environmental and Water Resources Congress 2010. EWRI, Providence, R.I.. Adams, T.E., 2010. Flood Forecasting in the United States NOAA/National Weather Service. In: Adams, T.E., Pagano, T.C. (Eds.), Flood Forecasting: a global perspective, first ed. Elsevier/Academic Press, New York, NY, pp. 275–325 Ch. 12. Adams, T.E., Dymond, R., 2019. The effect of QPF on real-time deterministic hydrologic forecast uncertainty. J. Hydrometeorol. https://doi.org/10.1175/JHM-D-18-0202.1. Adams, T.E., Pagano, T.C., 2016. Introduction. In: Adams, T.E., Pagano, T.C. (Eds.), Flood Forecasting: a global perspective, first ed. Elsevier/Academic Press, New York, NY, pp. 5–23 Ch. 1. Alfieri, L., Velasco, D., Thielen, J., 2011. Flash flood detection through a multi-stage probabilistic warning system for heavy precipitation events. Adv. Geosci. 29, 69–75. URL: https://www.adv-geosci.net/29/69/2011/. Anagnostou, E., Krajewski, W., Seo, D.-J., Johnson, E., 1998. Mean-field radar rainfall bias studies for WSR-88D. ASCE J. Eng. Hydrol. 3 (3), 149–159. Anderson, E.A., 1973. National Weather Service River Forecast System-Snow Accumulation and Ablation Model. Tech. Rep. NWS-HYDRO-17, U.S. National Weather Service, Office of Hydrology, Hydrology Laboratory. Technical Memo., 1325 East West Highway, Room 8372 Silver Spring, Maryland 20910. Baeck, M.L., Smith, J.A., 1998. Rainfall estimation by the WSR-88D for heavy rainfall events. Weather Forecasting 13 (2), 416–436. Benke, K.K., Lowell, K.E., Hamilton, A.J., 2008. Parameter uncertainty, sensitivity analysis and prediction error in a water-balance hydrological model. Math. Comput. Modell. 47 (11), 1134–1149. URL: http://www.sciencedirect.com/science/article/ pii/S0895717707002373. Bonnin, G.M., Martin, D., Lin, B., Parzybok, T., Yekta, M., Riley, D., 2006. PrecipitationFrequency Atlas of the United States, Ohio River Basin and Surrounding States, NOAA Atlas 14 Volume 2 Version 3.0. Tech. rep., U.S Department of Commerce, Silver Spring, MD. Borga, M., Anagnostou, E., Blöschl, G., Creutin, J.-D., 2010. Flash floods: Observations and analysis of hydro-meteorological controls. Journal of Hydrology 394 (1–2), 1–3.

17

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Im, J.-S., Brill, K., Danaher, E., 2006. Confidence interval estimation for quantitative precipitation forecasts (QPF) using short-range ensemble forecasts (SREF). Weather Forecasting 21 (1), 24–41. Jones, D., Wendland, W., 1984. Some statistics of instantaneous precipitation. J. Climate Appl. Meteorol. 23 (9), 1273–1285. Kitzmiller, D., Cooten, S.V., Ding, F., Howard, K., Langston, C., Zhang, J., Moser, H., Zhang, Y., Gourley, J.J., Kim, D., Riley, D., 2011. Evolving multisensor precipitation estimation methods: their impacts on flow prediction using a distributed hydrologic model. J. Hydrometeorol. 12 (6), 1414–1431. Kitzmiller, D., Miller, D., Fulton, R., Ding, F., 2013. Radar and multisensor precipitation estimation techniques in national weather service hydrologic operations. J. Hydrol. Eng. 18 (2), 133–142. Koren, V., Reed, S., Smith, M., Zhang, Z., Seo, D.-J., 2004. Hydrology laboratory research modeling system (HL-RMS) of the us national weather service. Journal of Hydrology 291 (3), 297–318. catchment modelling: Towards an improved representation of the hydrological processes in real-world model applications. URL:http://www. sciencedirect.com/science/article/pii/S002216940400037X. Koren, V., Smith, M., Cui, Z., Cosgrove, B., Werner, K., Zamora, R., 2010. Modification of Sacramento Soil Moisture Accounting Heat Transfer Component (SAC-HT) for Enhanced Evapotranspiration. Tech. Rep. NOAA NWS Technical Report NWS 53, Department of Commerce, NOAA/NWS. Koren, V.I., 2006. Parameterization of frozen ground effects: sensitivity to soil properties. predictions in ungauged basins: Promises and progress. In: Symposium S7, Seventh IAHS Scientific Assembly. IAHS Publication 303, Foz do Iquacu, Brazil, pp. 125–133. Koren, V.I., Smith, M., Wang, D., Zhang, Z., 2000. Use of soil property data in the derivation of conceptual rainfall-runoff model parameters. In: Conference on Hydrology. AMS, Long Beach, CA. Krajewski, W.F., Ciach, G.J., Habib, E., 2003. An analysis of small-scale rainfall variability in different climatic regimes. Hydrol. Sci. J. 48 (2), 151–162. https://doi.org/10. 1623/hysj.48.2.151.44694. Lack, S.A., Limpert, G.L., Fox, N.I., 2010. An object-oriented multiscale verification scheme. Weather and Forecasting 25 (1), 79–92. https://doi.org/10.1175/ 2009WAF2222245.1. Larson, C., Reich, B., 1972. Relationship of observed rainfall and runoff recurrence intervals. In: Proc. 2nd Int. Hydrol. Symp. Vol. 1. Fort Collins, Colorado, pp. 34–43. Leopold, L.B., 1994. A View of the River, first ed. Harvard University Press. Lin, Y., Mitchell, K.E., 2005. The NCEP Stage II/IV hourly precipitation analyses: Development and applications. In: Pre-prints, 19th Conf. on Hydrology, 1.2. [Available online athttps://ams.confex.com/ams/pdfpapers/83847.pdf.]. Amer. Meteor. Soc., San Diego, CA. Liston, G.E., Elder, K., 2006. A distributed snow-evolution modeling system (snowmodel). J. Hydrometeorol. 7 (6), 1259–1276. Martinaitis, S.M., Cocks, S.B., Qi, Y., Kaney, B.T., Zhang, J., Howard, K., 2015. Understanding winter precipitation impacts on automated gauge observations within a real-time system. J. Hydrometeorol. 16 (6), 2345–2363. https://doi.org/10.1175/ JHM-D-15-0020.1. Martinaitis, S.M., Qi, Y., Cocks, S., Tang, L., Kaney, B., Zhang, J., Howard, K., 2014. Improving MRMS Q3 precipitation estimation in the Western United States: Preliminary results. In: Extended Abstract, 39th Natl. Wea. Assoc. Annual Meeting. Salt Lake City, UT. Mascaro, G., Vivoni, E.R., Deidda, R., 2010. Implications of ensemble quantitative precipitation forecast errors on distributed streamflow forecasting. J. Hydrometeorol. 11 (1), 69–86. https://doi.org/10.1175/2009JHM1144.1. Maurer, E.P., Lettenmaier, D.P., 2003. Predictability of seasonal runoff in the Mississippi River basin. J. Geophys. Res.: Atmos. 108 (D16), 8607. https://doi.org/10.1029/ 2002JD002555. Mittermaier, M., Roberts, N., 2010. Intercomparison of spatial forecast verification methods: identifying skillful spatial scales using the fractions skill score. Weather Forecasting 25 (1), 343–354. Mizukami, N., Koren, V., Smith, M., Kingsmill, D., Zhang, Z., Cosgrove, B., Cui, Z., 2013. The impact of precipitation type discrimination on hydrologic simulation: rain-snow partitioning derived from HMT-west radar-detected brightband height versus surface temperature data. J. Hydrometeorol. 14 (4), 1139–1158. Moine, N.L., Hendrickx, F., Gailhard, J., 2013. Rainfall-runoff modelling as a tool for constraining the reanalysis of daily precipitation and temperature fields in mountainous regions. In: Cold and Mountain Region Hydrological Systems Under Climate Change Towards Improved Projections 1 Proceedings of H02, pp. H02. Nash, J., Sutcliffe, J., 1970. River flow forecasting through conceptual models Part I – a discussion of principles. J. Hydrol. 10 (3), 282–290. URL: http://www.sciencedirect. com/science/article/pii/0022169470902556. National Research Council, 2006. Completing the Forecast: Characterizing and Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts. Tech. rep., Committee on Estimating and Communicating Uncertainty in Weather and Climate Forecasts, Washington, D.C. National Research Council, 2012. The National Weather Service Modernization and Associated Restructuring: A Retrospective Assessment. Tech. rep., National Research Council, Washington, D.C. NCAR, 2015. verification: Weather Forecast Verification Utilities. NCAR - Research Applications Laboratory, r package version 1.42. URL:https://CRAN.R-project.org/ package=verification. Nelson, B.R., Prat, O.P., Seo, D.-J., Habib, E., 2016. Assessment and implications of NCEP stage iv quantitative precipitation estimates for product intercomparisons. Weather Forecasting 31 (2), 371–394. https://doi.org/10.1175/WAF-D-14-00112.1. Newman, A.J., Clark, M.P., Craig, J., Nijssen, B., Wood, A., Gutmann, E., Mizukami, N., Brekke, L., Arnold, J.R., 2015. Gridded ensemble precipitation and temperature estimates for the contiguous united states. J. Hydrometeorol. 16 (6), 2481–2500.

Crum, T.D., Alberty, R.L., 1993. The WSR-88D and the WSR-88D operational support facility. Bull. Am. Meteorol. Soc. 74 (9), 1669–1687. Cunha, L.K., Smith, J.A., Baeck, M.L., Krajewski, W.F., 2013. An early performance evaluation of the nexrad dual-polarization radar rainfall estimates for urban flood applications. Weather Forecasting 28 (6), 1478–1497. Cuo, L., Pagano, T.C., Wang, Q.J., 2011. A review of quantitative precipitation forecasts and their use in short- to medium-range streamflow forecasting. J. Hydrometeorol. 12 (5), 713–728. Dalrymple, T., 1960. Flood-frequency analyses, Manual of Hydrology: Part 3. Tech. rep., U.S Geological Survey, Reston, VA, water Supply Paper 1543-A. Daly, C., Halbleib, M., Smith, J.I., Gibson, W.P., Doggett, M.K., Taylor, G.H., Curtis, J., Pasteris, P.P., 2008. Physiographically sensitive mapping of climatological temperature and precipitation across the conterminous united states. Int. J. Climatol. 28 (15), 2031–2064. https://doi.org/10.1002/joc.1688. Damrath, U., Doms, G., Fruehwald, D., Heise, E., Richter, B., Steppeler, J., 2000. Operational quantitative precipitation forecasting at the german weather service. J. Hydrol. 239, 260–285. Demargne, J., Wu, L., Regonda, S.K., Brown, J.D., Lee, H., He, M., Seo, D.-J., Hartman, R., Herr, H.D., Fresch, M., Schaake, J., Zhu, Y., 2014. The science of noaa’s operational hydrologic ensemble forecast service. Bull. Am. Meteorol. Soc. 95 (1), 79–98. Diomede, T., Nerozzi, F., Paccagnella, T., Todini, E., 2008. The use of meteorological analogues to account for LAM QPF uncertainty. Hydrol. Earth Syst. Sci. 12 (1), 141–157. URL:https://www.hydrol-earth-syst-sci.net/12/141/2008/. Ebert, E.E., 2001. Ability of a poor man’s ensemble to predict the probability and distribution of precipitation. Mon. Weather Rev. 129, 2461–2480. Eldardiry, H., Habib, E., Zhang, Y., Graschel, J., 2017. Artifacts in stage IV NWS real-time multisensor precipitation estimates and impacts on identification of maximum series. J. Hydrol. Eng. 22 (5), E4015003. England, J.F., Julien, P.Y., Velleux, M.L., 2014. Physically-based extreme flood frequency with stochastic storm transposition and paleoflood data on large watersheds. J. Hydrol. 510 (Supplement C), 228–245. URL:http://www.sciencedirect.com/science/ article/pii/S0022169413009189. Essery, R., Li, L., Pomeroy, J., 1999. A distributed model of blowing snow over complex terrain. Hydrol. Process. 13 (14–15), 2423–2438. https://doi.org/10.1002/(SICI) 1099-1085(199910)13:14/15<2423::AID-HYP853>3.0.CO;2-U. Essery, R., Pomeroy, J., 2004. Vegetation and topographic control of wind-blown snow distributions in distributed and aggregated simulations for an arctic tundra basin. J. Hydrometeorol. 5 (5), 735–744. Evenden, G.I., 1990. Cartographic Projection Procedures for the UNIX Environment – A User’s Manual. Tech. Rep. Open-File Report 90–284. U.S. Dept. of the Interior, Geological Survey. Faurès, J.-M., Goodrich, D.C., Woolhiser, D.A., Sorooshian, S., 1995. Impact of small-scale spatial rainfall variability on runoff modeling. J. Hydrol. 173, 309–326. Fogel, M., 1969. Effect of storm rainfall variability on runoff from small semiarid watersheds. Trans. ASAE 12 (6), 808–812. Foufoula-Georgiou, E., 1989. A probabilistic storm transposition approach for estimating exceedance probabilities of extreme precipitation depths. Water Resour. Res. 25 (5), 799–815. https://doi.org/10.1029/WR025i005p00799. Fread, D., Shedd, R., Smith, G., Farnsworth, R., Hoffeditz, C., Wenzel, L., Wiele, S., Smith, J., Day, G., 1995. Modernization in the National Weather Service River and Flood Program. Weather and Forecasting 10 (3), 477–484. https://doi.org/10.1175/15200434(1995) 010<0477:MITNWS>2.0.CO;2. Fulton, R.A., 1998. WSR-88D Polar-to-HRAP Mapping. Tech. Rep. Tech. Memo. Hydrologic Research Laboratory, Office of Hydrology, National Weather Service. Fulton, R.A., Breidenbach, J.P., Seo, D.-J., Miller, D.A., O’Bannon, T., 1998. The WSR-88D rainfall algorithm. Weather Forecasting 13 (2), 377–395. Fulton, R.A., Ding, F., Miller, D.A., 2003. Truncation errors in historical WSR-88D rainfall products. In: 31st Conference on Radar Meteorology. Amer. Meteor. Soc. Gilleland, E., 2019. SpatialVx: Spatial Forecast Verification. R package version 0.6-4. URL:https://CRAN.R-project.org/package=SpatialVx. Goodrich, D.C., Faurès, J.-M., Woolhiser, D.A., Lane, L.J., Sorooshian, S., 1995. Measurement and analysis of small-scale convective storm rainfall variability. J. Hydrol. 173 (1–4), 283–308. URL:http://www.sciencedirect.com/science/article/ pii/002216949502703R. Gourley, J.J., Calvert, C.M., 2003. Automated detection of the bright band using WSR88D data. Weather Forecasting 18 (4), 585–599. Gourley, J.J., Vieux, B.E., 2005. A method for evaluating the accuracy of quantitative precipitation estimates from a hydrologic modeling perspective. J. Hydrometeorol. 6 (2), 115–133. https://doi.org/10.1175/JHM408.1. GRASS Development Team, 2016. Geographic Resources Analysis Support System (GRASS GIS) Software. Open Source Geospatial Foundation, USA. URL:http://grass. osgeo.org. Gupta, H.V., Kling, H., Yilmaz, K.K., Martinez, G.F., 2009. Decomposition of the mean squared error and NSE performance criteria: implications for improving hydrological modelling. J. Hydrol. 377 (1), 80–91. URL: http://www.sciencedirect.com/science/ article/pii/S0022169409004843. Huff, F., 1970. Sampling errors in measurement of mean precipitation. J. Appl. Met. 9 (1), 35–44. Humphrey, M.D., Istok, J.D., Lee, J.Y., Hevesi, J.A., Flint, A.L., 1997. A new method for automated dynamic calibration of tipping-bucket rain gauges. J. Atmos. Oceanic Technol. 14 (6), 1513–1519. Hunter, S.M., Holroyd, E.W., September 2002. Demonstration of improved operational water resources management through the use of better snow water equivalent information. Tech. Rep. R-02-02, U.S. Dept. of the Interior, Bureau of Reclamation, Technical Service Center, River Systems and Meteorology Group, Water Resources Services Div.

18

Journal of Hydrology 579 (2019) 124138

T.E. Adams, III and R.L. Dymond

Tetzlaff, D., Uhlenbrook, S., 2005. Significance of spatial variability in precipitation for process-oriented modelling: results from two nested catchments using radar and ground station data. Hydrol. Earth Syst. Sci. 9 (1/2), 29–41. URLhttps://www. hydrol-earth-syst-sci.net/9/29/2005/. Thiboult, A., Anctil, F., Boucher, M.-A., 2016. Accounting for three sources of uncertainty in ensemble hydrological forecasting. Hydrol. Earth Syst. Sci. 20 (5), 1809–1825. URL:https://www.hydrol-earth-syst-sci.net/20/1809/2016/. U.S. Department of Commerce, 1972. National Weather Service River Forecast System (NWSRFS-Model). Tech. rep., NOAA Technical Memorandum NWS-Hydro-14, Washington, D.C. Vasiloff, S.V., Seo, D.-J., Howard, K.W., Zhang, J., Kitzmiller, D.H., Mullusky, M.G., Krajewski, W.F., Brandes, E.A., Rabin, R.M., Berkowitz, D.S., Brooks, H.E., McGinley, J.A., Kuligowski, R.J., Brown, B.G., 2007. Improving QPE and Very Short Term QPF: An Initiative for a Community-Wide Integrated Approach. Bull. Am. Meteorol. Soc. 88 (12), 1899–1911. https://doi.org/10.1175/BAMS-88-12-1899. Wayand, N.E., 2016. Observation, Simulation, and Evaluation of Snow Dynamics in the Transitional Snow Zone (PhD dissertation). University of Washington. Wayand, N.E., Clark, M.P., Lundquist, J.D., 2017. Diagnosing snow accumulation errors in a rain-snow transitional environment with snow board observations. Hydrol. Process. 31 (2), 349–363. https://doi.org/10.1002/hyp.11002. hYP-16-0027.R2. Wilks, D., 2006. Statistical Methods in the Atmospheric Sciences, second ed. Academic Press. Wilson, C.B., Valdes, J.B., Rodriguez-Iturbe, I., 1979. On the influence of the spatial distribution of rainfall on storm runoff. Water Resour. Res. 15 (2), 321–328. https:// doi.org/10.1029/WR015i002p00321. Winstral, A., Elder, K., Davis, R.E., 2002. Spatial snow modeling of wind-redistributed snow using terrain-based parameters. J. Hydrometeorol. 3 (5), 524–538. Wood, A.W., Lettenmaier, D.P., 2008. An ensemble approach for attribution of hydrologic prediction uncertainty. Geophys. Res. Lett. 35 (14), l14401. https://doi.org/10. 1029/2008GL034648. Wright, D.B., Smith, J.A., Baeck, M.L., 2014. Flood frequency analysis using radar rainfall fields and stochastic storm transposition. Water Resour. Res. 50 (2), 1592–1615. https://doi.org/10.1002/2013WR014224. Xia, Y., Mitchell, K., Ek, M., Cosgrove, B., Sheffield, J., Luo, L., Alonge, C., Wei, H., Meng, J., Livneh, B., Duan, Q., Lohmann, D., 2012a. Continental-scale water and energy flux analysis and validation for north american land data assimilation system project phase 2 (nldas-2): 2. validation of model-simulated streamflow. J. Geophys. Res.: Atmos. 117 (D3), d03110. https://doi.org/10.1029/2011JD016051. Xia, Y., Mitchell, K., Ek, M., Sheffield, J., Cosgrove, B., Wood, E., Luo, L., Alonge, C., Wei, H., Meng, J., Livneh, B., Lettenmaier, D., Koren, V., Duan, Q., Mo, K., Fan, Y., Mocko, D., 2012b. Continental-scale water and energy flux analysis and validation for the north american land data assimilation system project phase 2 (NLDAS-2): 1. Intercomparison and application of model products. J. Geophys. Res.: Atmos. 117 (D3), d03109. https://doi.org/10.1029/2011JD016048. Xiao, J., Bintanja, R., Déry, S.J., Mann, G.W., Taylor, P.A., 2000. An intercomparison among four models of blowing snow. Boundary-Layer Meteorol. 97 (1), 109–135. https://doi.org/10.1023/A: 1002795531073. Young, C.B., Bradley, A.A., Krajewski, W.F., Kruger, A., Morrissey, M.L., 2000. Evaluating NEXRAD multisensor precipitation estimates for operational hydrologic forecasting. J. Hydrometeorol. 1 (3), 241–254. Zambrano-Bigiarini, M., 2014. hydroGOF: Goodness-of-fit functions for comparison of simulated and observed hydrological time series. R package version 0.3-8. URL:https://CRAN.R-project.org/package=hydroGOF. Zhang, J., Howard, K., Langston, C., Kaney, B., Qi, Y., Tang, L., Grams, H., Wang, Y., Cocks, S., Martinaitis, S., Arthur, A., Cooper, K., Brogden, J., Kitzmiller, D., 2016. Multi-radar multi-sensor (MRMS) quantitative precipitation estimation: initial operating capabilities. Bull. Am. Meteorol. Soc. 97 (4), 621–638. Zhang, Y., Adams, T., Bonta, J.V., 2007. Subpixel-scale rainfall variability and the effects on separation of radar and gauge rainfall errors. J. Hydrometeorol. 8 (6), 1348–1363.

Novak, D.R., Bailey, C., Brill, K.F., Burke, P., Hogsett, W.A., Rausch, R., Schichtel, M., 2014. Precipitation and temperature forecast performance at the weather prediction center. Weather Forecasting 29 (3), 489–504. NRC, 1997. An Assessment of the Advanced Weather Interactive Processing system: Operational Test and Evaluation of the First System Build. Tech. rep., National Research Council, Washington, D.C. NRC, 2006. Toward a New Advanced Hydrologic Prediction Service (AHPS). Tech. rep., Committee to Assess the National Weather Service Advanced Hydrologic Prediction Service Initiative, Water Science and Technology Board, Washington, D.C. Peleg, N., Ben-Asher, M., Morin, E., 2013. Radar subpixel-scale rainfall variability and uncertainty: lessons learned from observations of a dense rain-gauge network. Hydrol. Earth Syst. Sci. 17 (6), 2195–2208. URL:https://www.hydrol-earth-syst-sci. net/17/2195/2013/. R Core Team, 2017. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. URL:https://www.R-project. org/. Rausch, R., 2016. Email July 28. personal communication, weather Prediction Center, NOAA, National Weather Service. Reed, S.M., Maidment, D.R., 1999. Coordinate transformations for using nexrad data in gis-based hydrologic modeling. J. Hydrol. Eng. 4 (2), 174–182. Rempel, M., Senf, F., Deneke, H., 2017. Object-based metrics for forecast verification of convective development with geostationary satellite data. Monthly Weather Rev. 145 (8), 3161–3178. https://doi.org/10.1175/MWR-D-16-0480.1. Rezacova, D., Zacharov, P., Sokol, Z., 2009. Uncertainty in the area-related qpf for heavy convective precipitation. Atmos. Res. 93 (1), 238–246. 4th European Conference on Severe Storms. URL:http://www.sciencedirect.com/science/article/pii/ S0169809508003475. Rössler, O., Froidevaux, P., Börst, U., Rickli, R., Martius, O., Weingartner, R., 2014. Retrospective analysis of a nonforecasted rain-on-snow flood in the alps: a matter of model limitations or unpredictable nature? Hydrol. Earth Syst. Sci. 18 (6), 2265–2285. URL:https://www.hydrol-earth-syst-sci.net/18/2265/2014/. Schaake, J., Henkel, A., Cong, S., 2004. Application of PRISM climatologies for hydrologic modeling and forecasting in the western U.S. In: Proc., 18th Conf. on Hydrology. Schröter, K., Llort, X., Velasco-Forero, C., Ostrowski, M., Sempere-Torres, D., 2011. Implications of radar rainfall estimates uncertainty on distributed hydrological model predictions. Atmos. Res. 100 (2), 237–245. uncertainty Propagation in Advanced Hydro-Meteorological Forecast Systems. URL:http://www.sciencedirect.com/ science/article/pii/S0169809510002085. Seo, D.-J., 1998. Real-time estimation of rainfall fields using radar rainfall and rain gage data. J. Hydrol. 208 (1), 37–52. URL: http://www.sciencedirect.com/science/ article/pii/S0022169498001413. Seo, D.-J., Breidenbach, J., Johnson, E., 1999. Real-time estimation of mean field bias in radar rainfall data. J. Hydrol. 223 (3), 131–147. URL:http://www.sciencedirect.com/ science/article/pii/S0022169499001067. Smith, J.A., Baeck, M.L., Morrison, J.E., Sturdevant-Rees, P., 2000. Catastrophic rainfall and flooding in texas. J. Hydrometeorol. 1 (1), 5–25. Smith, J.A., Baeck, M.L., Steiner, M., Miller, A.J., 1996. Catastrophic rainfall from an upslope thunderstorm in the central appalachians: The rapidan storm of june 27, 1995. Water Resources Research 32 (10), 3099–3113. https://doi.org/10.1029/ 96WR02107. Sungmin, O., Foelsche, U., Kirchengast, G., Fuchsberger, J., 2016. Validation and correction of rainfall data from the WegenerNet high density network in southeast Austria. J. Hydrol. URL:http://www.sciencedirect.com/science/article/pii/ S0022169416307648. Taylor, G., Daly, C., Gibson, W., 1993. Development of an isohyetal analysis for Oregon using the PRISM model. In: 8th Conf. on Applied Climatology. Anaheim, CA, pp. 126–127. Taylor, G., Daly, C., Gibson, W., 1995. Development of a model for use in estimating the spatial distribution of precipitation. In: 9th Conf. on Applied Climatology. Dallas, TX, pp. 92–93.

19