A description and evaluation of FAO satellite rainfall estimation algorithm

Atmospheric Research 163 (2015) 48–60

Contents lists available at ScienceDirect

Atmospheric Research journal homepage: www.elsevier.com/locate/atmos

A description and evaluation of FAO satellite rainfall estimation algorithm Tufa Dinku a,⁎, Stefano Alessandrini b,c, Mauro Evangelisti b, Oscar Rojas b a b c

International Research Institute for Climate and Society, The Earth Institute at Columbia University, 61 Rout 9 W, Palisades, NY 10964, USA Foods and Agriculture Organization of the United Nations (FAO), Viale delleTerme di Caracalla, 00153 Rome, Italy NCAR Research Applications Laboratory, National Center for Atmospheric Research, PO Box 3000, Boulder, CO 80307-3000, United States

a r t i c l e

i n f o

Article history: Received 1 May 2014 Received in revised form 15 January 2015 Accepted 19 January 2015 Available online 29 January 2015 Keywords: Satellite Rainfall Algorithm Estimation Climate Data

a b s t r a c t There are ongoing efforts to improve the accuracy of satellite rainfall estimates. One of these efforts comes from the Food and Agriculture Organization (FAO) of the United Nations. The FAO effort involves combining satellite rainfall estimates and meteorological model outputs with station measurements. The algorithm of the FAO satellite rainfall estimates (FAO-RFE) is presented and evaluated by comparing with raingauge data and other satellite rainfall products over eastern and western parts of Africa. The evaluations were done at daily and ten-daily time scales. The FAO-RFE has shown significant improvement over the individual inputs. However, comparison of FAO-RFE with other satellite rainfall products has shown a slight improvement only over areas with good station input. The main weakness of the FAO-RFE is that it overestimates rainfall occurrences, which is attributed to the forecast product used in the algorithm. © 2015 Elsevier B.V. All rights reserved.

1. Introduction Observations from meteorological stations are the main source of rainfall data. However, the coverage of meteorological stations is very limited across most of rural Africa. Satellite proxies have been used to supplement or in lieu of station observations. The main strength of satellite rainfall estimates is that they provide good spatial coverage, including remote areas, and are freely available. However, satellite rainfall estimates also suffer from many shortcomings that include accuracy, particularly at higher temporal resolutions, coarse spatial resolution, short time series, and temporal inhomogeneity due to varying inputs. There are a number of efforts to improve the accuracy of satellite rainfall estimates. Some of these efforts have been summarized by Kidd and Huffman (2011) and Kidd and Levizzani (2011). Recent efforts include Mishra et al. (2011), Chadwick and Grimes (2012), Brocca et al. (2013), Brocca et al. (2014), Lazri et al.(2013), Xu et al. (2013), Lazri et al. (2014), and Xu et al. (2014). The Food and Agriculture Organization (FAO) of the United Nations has also been working on improving satellite rainfall estimates over Africa. The FAO is one of the pioneers in using satellite rainfall estimation (e.g. Hielkema and Snijders, 1994). The current FAO effort involves combining satellite rainfall estimates and meteorological model outputs with station measurements. The main goal of this approach is to benefit from the strengths of the different data sets while overcoming ⁎ Corresponding author. E-mail address: [email protected] (T. Dinku).

http://dx.doi.org/10.1016/j.atmosres.2015.01.020 0169-8095/© 2015 Elsevier B.V. All rights reserved.

their weaknesses. Many of today’s satellite rainfall retrieval algorithms combine data from different sources. Most of these algorithms use different techniques to combine estimates from thermal infrared (TIR-RR) and passive microwave (PMW) sensors (Joyce et al., 2004; Huffman et al., 2007; Kidd and Huffman, 2011; Kidd and Levizzani, 2011). The FAO rainfall algorithm is one of the few approaches that incorporate meteorological model outputs to improve the accuracy of rainfall estimates. The FAO approach combines thermal infrared rainfall (TIR) estimate, the Multi-Sensor Precipitation Estimate (MPE) from EUMETSAT, 24-hour rainfall forecast from the European Center for Medium Range Weather Forecast (ECMWF), and daily Global Telecommunication System (GTS) and other rain gauge data. Algorithm development started in 2008 and now runs operationally at FAO Head Quarters in Rome producing estimates for the entire African continent. Another version has also been installed at the Sudan Meteorological Authority. This paper describes and evaluates the FAO-RFE over East Africa (Ethiopia, Sudan and South Sudan) and Sahel (Burkina Faso, Mali and Niger) at daily and ten-daily timescales. These evaluations are done by comparing the FAO-RFE with reference raingauge data from a relatively dense station networks. The FAO-RFE is also compared with other widely used satellite rainfall estimates. These include rainfall estimate (RFE) from the Climate Prediction Centre (CPC) of the National Oceanic and Atmospheric Administration (NOAA) as described in Xie et al. (2002), the TAMSAT (Tropical Applications of Meteorology using Satellite data) rainfall estimate (Grimes et al., 1999; Thorne et al., 2001), the CPC morphing

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

(CMORPH) algorithm (Joyce et al., 2004), and the TRMM (Tropical Rainfall Measuring Mission) Multi-satellite Precipitation Analysis (TMPA, Huffman et al., 2007). However, the focus here is evaluation of the FAO-RFE as many of the other products have already been evaluated over the study region by different authors. These include Dinku et al. (2008), Hirpa and Gebremichael (2009), Robinson et al. (2009), Dinku et al. (2010a), Dinku et al. (2010b), Roca et al. (2010), Symeonakis et al. (2009), Jobard et al. (2011), Romilly and Gebremichael (2011), and Habib et al. (2012). Section 2 describes the study region and data, while detailed description of the FAO algorithm is given in Section 3. The FAO algorithm is compared with other satellite products in Section 4. Section 5 summarizes the results. 2. Study region and data 2.1. Study region The study areas are located over East and West Africa (Sahel) shown in Fig. 1. The study area over East Africa covers Sudan, South Sudan and western half of Ethiopia. The Ethiopian part has the most complex topography on the continent. Elevation ranges from below sea level to 4620 m. The complex orography plays a significant role in shaping the climate of that region. It also poses a serious challenge to satellite rainfall estimation (Dinku et al., 2011). The climate of the Sudan is more similar to the other part of the Sahel than the nearby Ethiopia. It varies from desert types in the north and tropical semiarid and sub-humid climates in the south. The climate over South Sudan is mostly similar to that of southwestern Ethiopia with tropical rainy climate. The study area over Sahel covers Burkina Faso, Mali, and Niger. This is a mostly flat region. The climate system that brings rainfall over this region is the West African monsoon, which starts with an abrupt northward shift of the intertropical convergence zone (ITCZ) during May to June (Sultan and Janicot, 2003). The region has only one rainy season, mainly during June to August 2.2. Raingauge data Raingauge data from parts of Ethiopia, Sudan, South Sudan, Burkina Faso, Mali and Niger are used for this evaluation. Raingauge data for Ethiopia were obtained from the National Meteorology Agency of Ethiopia. These data come from over 400 stations and cover the period 2007 to 2009. Sixty stations were used for calibration, while the rest are used for validation (Figs. 1 & 2). The Sudan

49

Meteorological Authority has provided daily rainfall data for 2007 to 2009 from about 30 stations. Half of the stations were used for calibrating the FAO-RFE algorithm while the remaining half were used for validations (Figs. 1 & 2). Data made available through the World Meteorological Organization's Global Telecommunication System (GTS) were used for calibrating the algorithm over the Sahel. Measurements from around 30 stations were available in this study area. The validation data set of about 200 stations for the summer months of 2007–2009 were obtained from the AGRHYMET Center. Basic quality check has been performed on all station data to identify and remove suspicious values as well as checking the geolocations. The validation data from Ethiopia and the Sahel were interpolated into regular grids of 0.05° latitude/longitude and then averaged to 0.25° spatial resolution to match the resolutions of some of the satellite products. Simple inverse distance waiting was used for interpolation. The interpolation is a two-step process in which rainfall amounts and rainfall occurrences were interpolated separately and then the two results were combined. Only pixels containing at least one station were used for the evaluation. 2.3. Satellite data The FAO-RFE will be described in detail in the next section. Thus, a brief description the other satellite products used for comparison with FAO-RFE are given here. These include CPC-RFE, TAMSAT, CMORPH and TMPA-3B42. The CPC-RFE is produced by the Climate Prediction Centre (CPC) of the National Oceanic and Atmospheric Administration (NOAA) specifically for United States Agency for International Development (USAID) Famine Early Warning Systems (FEWS) to assist in drought monitoring activities over Africa. The latest version, RFE version 2.0 has been operational since January 2001 (Xie et al., 2002). The current version combines estimates from passive microwave (PM) sensors, estimates from METEOSAT TIR data, and daily rainfall observations from GTS reports. This algorithm produces daily rainfall estimates at a spatial resolution of 0.1°. The TAMSAT rainfall estimate is produced at Reading University in the UK. The TAMSAT method (Grimes et al., 1999; Thorne et al., 2001) is based on the assumption that cold cloud-top temperatures of tropical storms identify raining clouds. These temperatures are obtained from Meteosat thermal-infrared images. The length of time that a satellite pixel is colder than a given temperature threshold is then summed over ten days to create a cold-cloud-duration (CCD) images. It is assumed that that CCD is linearly related to rainfall. This algorithm uses TIR data only. The main strength of the TAMSAT approach is local calibration with gauge data from many

Fig. 1. Calibration stations.

50

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

Fig. 2. Validation stations.

parts of Africa. The daily data used here is an experimental product, and is not yet publicly available. It is drierved by temporal dissagragtion of the satandard dekadal estimates using daily CCD (Ross Maidment, personl communication). TAMSAT is in the process of preparing a manuscrput for publication. The CMORPH algorithm combines different passive microwave (PM) rain retrievals and TIR information. It uses precipitation estimates derived from PM observations and propagates these features in space and time using motion vectors derived from half-hourly geostationary satellite TIR data (Joyce et al., 2004). The objective is to combine the better retrieval accuracy of PM estimates and the higher temporal and spatial resolution of TIR data. CMORPH produces global precipitation analyses at very high spatial (~ 8 km) and temporal (30 min) resolutions starting from December 2002. However, the data used here is three-hourly observation at a spatial resolution of 0.25°. The three-hourly observations are converted to daily totals. The TRMM project at NASA produces rainfall estimates at different spatial and temporal scales. The TRMM Multi-satellite Precipitation Analysis (TMPA, Huffman et al., 2007) 3B42 product (TMPA-3B42) is used here for comparison with FAO-RFE. This algorithm combines TIR data from geostationary satellites and PM retrievals from different sources. The TMPA-3B42 product is available a couple of days after the end of each month. Version 7 of TMPA-3B42 is used here. Table 1 summarizes the different characteristics of the satellite rainfall products used for comparison with the FAO-RFE.

3. The FAO rainfall estimation algorithm 3.1. History The FAO rainfall estimate (FAO-RFE) was initially developed at FAO head quarters for internal use, mainly for monitoring desert locust

movements. The data were made available for download at http:// geonetwork3.fao.org/climpag/FAO-RFE.php. A newer version of FAORFE system was developed in 2010 at the Sudan Meteorological Authority. The algorithm was improved to adapt it to the local needs. The system is operational and its outputs are integrated into the Sudan Agrometeorological Information System. 3.2. Algorithm The FAO-RFE algorithm produces daily rainfall maps by merging different rainfall datasets. The main strengths of this algorithm are that it combines several rainfall estimates including outputs from numerical weather prediction models and that it can incorporate all locally available station data, which are not available outside the countries. The input comes from the following sources: • Rainfall estimate derived from thermal infrared data (TIR-RR) on a 15minute basis; • Multi-sensor precipitation estimate (MPE) from EUMETSAT on 15minute basis; • Daily Global Telecommunication System (GTS) and other raingauge data; and • 24-hour rainfall forecast (of the past days) from ECMWF's global deterministic forecast model. The TIR-RR is generated by FAO and is based on the approach by Vicente et al. (1998), which was used for the GOES (Geostationary Operational Environmental Satellite) rainfall estimation algorithm. The TIR-RR algorithm makes use of Meteosat thermal infrared data and a power-law regression model. The first step is computing the cloud-top temperature trends, which is obtained by examining consecutive TIR images. Then rainfall rates are computed only for those pixels with decreasing cloud-top temperatures. The following power-law regression is used: 11

TIR‐RR ¼ 1:1183  10 Table 1 Descriptions of the different satellite products used for comparison with FAO-RFE at daily and 10-daily time scales. Satellite product

Provider

Spatial resolution

Temporal resolution

Start-End

RFE TAMSAT COMORPH TMPA

NOAA Univ. Reading NOAA NASA

0.10 deg. 0.0375 deg. 0.25 deg 0.25 deg

Daily Daily/10 daily 3 hourly 3 hourly

2001–present 1983–present 1998–present 1998–present

−3:638210−2 T 1:2 Þ  eð

ð1Þ

TIR-RR is the rainfall rate in millimeters per hour and T is the cloud-top brightness temperature in degree Kelvin. The rain rate is then modified by taking atmospheric moisture into account. This is accomplished by using relative humidity (RH) and precipitable water (PW) profiles as proposed by Vicente et al. (1998). In Vicente et al. (1998), the PW and RH profiles were obtained from radiosounding observations. Here those parameters are obtained from ECMWF model analysis, which enables computing the PW and RH

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

profiles even for areas where radio-soundings measurements are not available. The ECMWF forecast model runs 4 times a day (00, 06, 12, 18 UTC) and provides analysis fields for the main meteorological variables including pressure, wind, temperature and humidity at a horizontal resolution of 0.25° at different pressure levels. The vertical integrals of the PW and RH product (PWRH) are computed for each grid point every 6 h. This creates a two-dimensional field of PWRH at 00, 06, 12, 18 UTC. A linear interpolation in time and a bilinear interpolation in space are then performed to obtain PWRH values for each satellite pixel at the required times. Rainfall rates are calculated for each TIR image, and then daily totals are computed as follows. First, the average half hourly rainfall rate is computed using trimean (Wilks, 1995) of three consecutive images as follows: Rainð1=2 hourÞ ¼ ðRainminimum þ 2 Rainmedian þ RainmaximumÞ=4:

51

algorithm runs on a daily basis computing rainfall cumulative amount for the time interval 06 UTC-06 UTC on the geographic window of interest. A weighted linear combination is performed to combine the individual inputs (MPE, TIR-RR, and ECMWF) after global biases have been removed. Mean biases are computed for each of the input rainfall estimates using raingauge measurements. The three ratios hSi hSi hSi BM ¼ hMPE i, BR ¼ hTIR‐RRi, BE ¼ ECMWF are computed, where 〈S〉 is the

average of the daily rainfall values measured at the ground stations and 〈MPE〉, hTIR‐RRi and 〈ECMWF〉 are the average of the daily rainfall estimates at the corresponding pixels. Every pixel of the daily estimates is then multiplied by its correction factor. Then root mean square errors (RMSE) are computed for each rainfall estimate and a weighted linear combination (the weights being inversely proportional to the RMSE) 3

is then performed for every pixel as follows: Ecomb ¼ ∑ wi Ei where wi i¼1

ð2Þ σ −2 i

The daily rainfall rate is then computed by summing the half-hourly rainfall rates. The Multi-sensor Precipitation Estimate (MPE) has been developed by EUMETSAT (Heinemannn et al., 2002) and combines the advantages of the high temporal and spatial resolutions of current Meteosat TIR data with the better accuracy of rainfall from passive microwave sensors. Rainfall rates are calculated for each TIR image. Individual images are downloaded from EUMETSAT website and daily precipitation aggregated from the individual events. The third source of rainfall data is the ECMWF meteorological forecast model running twice a day (00 UTC and 12 UTC). The rainfall forecast is obtained from accumulation between 06 UTC and 06 UTC of the following day using the 00 UTC run. The rainfall field has a horizontal resolution of 0.25° but it is re-gridded at the satellite pixel resolution (~ 3 km) using a bilinear interpolation. The schematic representation of the FAO algorithm is given in Fig. 3. The FAO-RFE

3

and σi are the root mean square errors (RMSE) of each of the

∑ σ −2 i i¼1

three rainfall estimates, where Ecomb represents the total amount of daily rainfall after the merging and Ei are the total amounts from each of the input rainfall estimates after bias removal. The above steps produce a preliminary rainfall field, which is then adjusted by using gauge measurements. The pixel values of the combined estimate (Ecomb) can still be different from station values at those locations. Thus, ratios (Rs) are computed between gauge measurements and corresponding pixel values. The ratios are then spatially interpolated on the satellite pixel grid within a radius of influence from the station. The preliminary rainfall field (Ecomb) is then multiplied by the interpolated ratios. This ensures that the FAO-RFE is equal to the station measurements at the stations locations, but the preliminary rainfall field remains unchanged beyond the radius of influence of the stations. This strategy is known as

Fig. 3. Schematic presentation of the FAO-RFE Algorithm.

52

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

Table 2 Descriptions of validations statistics used in the paper. A, B, C and D represent hits, false alarms, misses, and correct negatives, respectively. G = gauge rainfall measurements, G = average of the gauge measurements, S = satellite rainfall estimate, and N = number of data pairs. Statistics

Formula

Range

Unit

Best value

Probability of detection

A POD ¼ AþC B FAR ¼ AþB AþB FBS ¼ AþC

0 to 1

None

1

0 to 1

None

0

0 to ∞

None

1

ðAD−BCÞ HSS ¼ ðAþCÞð2 CþDÞþðAþBÞðBþDÞ

−∞ to 1

None

1

−∞ to + ∞

mm

0

0 to ∞

None

0

0 to ∞

None

1

−∞ to 1

None

1

False alarm ratio Frequency bias Heidke skill score

1 N ∑ðS−GÞ

Mean error

ME ¼

Mean absolute error

MAE ¼ N

1

∑jðS−GÞj G

Bias ¼ ∑S ∑G ∑ðS−GÞ2 Eff ¼ 1−  2 ∑ G−G

Bias Efficiency

SEDI (Satellite Enhanced Data Interpolation, Hoefsloot, 1995). In order to provide a smoothly vanishing function for each station, a bell-shaped weighting function (wd) of the following form is used:

wd ¼

8 < :

:5 þ :5 cos

 . d

d0

 π

if d ≤d0

0

else

where d0 is the maximum distance of influence for a station. The estimation method has to measure the deviations from R = 1. Thus, the variable to interpolate is (RS − 1) where the index S stands for the station number. A simple weighting scheme for the estimation of RG, where G stands for the grid-pixel number, is applied as follows: ∑ðRS −1ÞwdS where dS is the distance from the pixel G and RG ¼ 1 þ ∑wdS the station S This weighting scheme however has the disadvantage of creating a discontinuity at locations G for which the closest station is at a distance d0. Grid points closer to the stations get weight 1, i.e. RG = RS1 if S1 is the closest station while RG = 1 for grid points far away from station location. In order to circumvent this problem, a virtual station observation RS = 1 is introduced with a relative distance δ = dV/d0 at each grid point. Introducing a smoothing coefficient M = 1 − δ with 0 b M ≤ 1, we get: X ðRS −1ÞwdS

RG ¼ 1 þ X

wdS þ wd ð1−M Þ

The final combined precipitation estimate should be the product of RG and the satellite estimate. There are two alternative radiuses of influences (d 0) used for each station. One is the radius where both FAO-RFE and stations detect rainfall. The other one is the radius where the gauges report no rainfall but FAO-RFE estimate is greater than zero (d0).

Table 3 Comparison of the three different inputs into the FAO-RFE algorithm and the combined product over Sahel. The rainfall threshold used for discriminating between rainy and dry days is 1 mm.

MPE TIR-RR ECMWF MERGED

CC

Bias

MAE

ME

POD

FAR

FBS

HSS

0.45 0.49 0.21 0.49

2.53 3.57 0.71 1.2

9.87 14.00 5.24 4.94

6.66 11.48 −1.31 0.91

0.69 0.77 0.68 0.82

0.34 0.38 0.52 0.44

1.05 1.25 1.43 1.47

0.49 0.47 0.24 0.41

In fact, in case of RS = 0 (Ecomb estimates rain greater than 0 while gauge observes no rain) the maximum distance d0 is reduced to a lower value d00. This just means that stations that observe no rain while the satellite detects rain for the same pixel have only a small radius of influence. In other words, the station observation may be right but not very representative since no rain at a station and low rain in its vicinity does not contradict. Some sensitivity tests, not shown here, have revealed as an optimal choice d0 = 100 km d00 = 50 km and M = 0.25. 4. Evaluation of FAO-RFE The main focus of this work is to assess the performance of the FAO-RFE algorithm over the different validation regions. This is done by comparing FAO-RFE with station data. The relative performance of the FAO-RFE with respect to some other satellite rainfall products has also been evaluated. These evaluations are done at daily and ten-daily (dekadal) time scales. Evaluation of the FAORFE algorithm starts with comparing the three individual inputs (TIR-RR, MPE and ECMWF). The main objective of this evaluation is to assess the relative strengths of the different inputs. One would also need to understand how the different inputs contribute toward improving the accuracy of the final product. Thus, the second part of the evaluation compares FAO-RFE generated using different inputs. For this purpose, the FAO-RFE algorithm was run with three different configurations: (i) single-input with MPE only (MPE); (ii) twoinputs with MPE and TIR-RR (MPE + TIR-RR); and (iii) the standard product with the three inputs (MPE + TIR-RR + ECMWF). The last part compares the standard FAO-RFE with some other satellite rainfall products (RFE, TMPA, CMORPH and TAMSAT). As the TMPA and CMORPH products are available at 0.25° spatial resolutions, all other products are aggregated to this spatial resolution. Different validation statistics are used to compare the different products. Daily satellite rainfall estimates have been shown not to be very good at estimating rainfall amounts, while most products are good in detecting the occurrence of rainfall (e.g. Dinku et al., 2008). Thus, evaluation at daily time scale focuses on assessing the skill of the estimates in detecting rainfall. There are different statistics used for this purpose. The evaluation statistics used here are probability of detection (POD), false alarm ratio (FAR), frequency bias (FBS), and the Heidke Skill Score (HSS). The POD is used to assess the skill of the satellite product in detecting the occurrence of rainfall, while FAR assesses how often the satellite products detect rainfall while there was no rainfall measured at station locations. The FBS compares the rainfall detection frequency of the satellite estimates with that of the raingauge. The HSS statistic measures the accuracy of the estimates accounting for matches due to random

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

53

Fig. 4. Empirical cumulative density function (CDF) of the different rainfall products. Blue lines represent CDF of raingauge data while the red lines represent that of the satellite and forecast products.

chances. The rainfall threshold used for discriminating between rainy and dry days is 1 mm. Linear correlation coefficient (CC), mean absolute error (MAE) and multiplicative bias were also used just to offer an insight into the skill of the products in estimating rainfall amounts. Empirical cumulative distribution function (CDF) is also used to compare the distribution of each product with that of raingauge. At ten-daily time scale, rainfall detection may not be the main issue. There could be a number of rainfall events within a ten-day

period and the satellite product may be able detect at least one of those events. This may not be true when there are few rainfall events within a 10-day period. As a result, the evaluation focuses on evaluating the skill of the products in estimating rainfall amounts. For this we use CC, Bias, MAE, and Efficiency (Eff). The MAE is used to assess random error. It is used instead of the root means square error (RMSE) to avoid the effect of extremely high rainfall values. The efficiency, also known as coefficient of efficiency, shows the skill of the estimates relative to a reference (in this case the gauge mean).

Fig. 5. Sample daily rainfall fields of the three inputs for FAO-RFE algorithm (TIR-RR, MPE, ECMWF) and the final merged product (MERGED) compared with Gauge and gridded gauge (GRD-GG) for 14 July 2007.

54

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

Table 4 Comparison of the different runs of FAO algorithm over Sudan.

MPE MPE + TIR-RR MPE + TIR-RR + ECMWF

CC

Bias

MAE

ME

POD

FAR

FBS

HSS

0.43 0.46 0.49

1.83 1.82 1.28

5.13 4.88 3.64

2.55 2.52 0.84

0.77 0.83 0.90

0.43 0.45 0.55

1.34 1.52 2.03

0.54 0.53 0.43

Scatter plots of gauge vs. each satellite product are also used to compare the different satellite products with the reference raingauge data. The formulas and other descriptions of the different evaluation statistics are given in Table 2.

4.1. Evaluation of the different inputs of FAO-RFE algorithm The evaluation of the individual products was done over the Sahel countries using daily data from the validation stations shown in Fig. 2. Table 3 summarizes the results, which includes the combined (MERGED) product as well. The three different products have different strengths and weakness. The satellite products (MPE and TIR-RR) have better correlation with raingauge observations as well as better detection skills (higher HSS values). However, these products exhibit higher biases and random error (MAE) compared to the ECMWF product. The TIR-only product (TIR-RR) has the highest bias and random error. Even though the satellite products overestimate rainfall amounts, the overestimation of rainfall occurrence is relatively small, particularly for MPE. Thus, satellite products seem to be good in capturing the areal extent of the rainfall, but overestimate rainfall amounts. The forecast product has smaller correlation and underestimates rainfall amounts while overestimating rainfall occurrences (higher FBS and FAR values). The overestimation of rainfall amounts by the satellite products and the underestimation by the forecast products are also shown in Fig. 4, which compares the empirical cumulative density function (CDF) of the different products with that of the reference rain gauge data. The CDFs of the satellite products are very similar to each other but very different from that of the forecast product. The overestimation of the occurrence of low rainfall amounts by TIR-RR and ECMWF is also evident. The combined product has significantly lower bias and random errors and higher POD compared to the individual inputs. However, its frequency bias is the same as that of the ECMWF product. As a result, the combined product overestimates the areal extent of the rainfall. The linear combination performed by FAO-RFE is able to reduce random errors and mean bias in the final product but exaggerates the spatial extent of rainfall fields. Fig. 5 compares sample rainfall fields for the three inputs and the merged products with gauge. This figure is a very good representation of the different characteristics of the three inputs and the merged product. The overestimation of rainfall amounts by the satellite products (particularly TIR-RR), the underestimation of rainfall amounts and overestimation of areal coverage by the forecast product, and contributions of the three inputs to the merged product are depicted clearly. The overestimation of rainfall occurrences by the ECMWF product could be corrected. One could use a mask (0's and 1's) created from the product with the best rain/no-rain discrimination, which is MPE (FBS = 1.05), and then multiplying the merged rainfall field by this

Table 5 Comparison of the different runs of FAO algorithm over Ethiopia.

MPE MPE + TIR-RR MPE + TIR-RR + ECMWF

CC

Bias

MAE

ME

POD

FAR

FBS

HSS

0.34 0.41

1.38 1.22 1.09

8.16 6.67 5.41

2.39 1.35 0.53

0.69 0.77 0.88

0.09 0.09 0.13

0.75 0.84 1.01

0.43 0.50 0.55

mask. The mask is created by assigning “0” to all zero values in an MPE rain field and assigning “1” to all values above zero. This way, the merged field will have rainfall occurrence frequencies close to that of MPE. 4.2. Comparing different runs of FAO algorithm The previous subsection evaluated the accuracy of the individual components of the FAO-RFE algorithm relative to raingauge measurements. This section will try to answer the question “what is the added value of each input of the algorithm?” To answer this question, we start with the MPE product and then add the other two inputs (TIRRR, and ECMWF) one by one. For this, three different runs of the FAO algorithm were performed over The Sudans and Ethiopia. This evaluation was done with daily rainfall data. Table 4 compares the results over both Sudans, while Table 5 presents the different runs over Ethiopia. The addition of TIR-RR to MPE does not show appreciable improvement over Sudan. In fact, there is an increase in FBS. There are some slight improvements over Ethiopia; the more noticeable improvement being in the POD. The addition of ECMWF to the other two inputs improves most of the statistics over both Sudan and Ethiopia. The only problem with the ECMWF input is that it increases false detection (FAR and FBS) and decreases HSS over Sudan. Thus, the addition of ECMWF seems to help in improving the accuracy in rainfall amounts but overestimates rainfall occurrence for Sudan. The situation is different over Ethiopia where the satellite products underestimate rainfall occurrences, which is compensated for by the model product. As a result, the final product does not show frequency bias (FBS ~ 1). The overestimation of rainfall occurrence by the satellite products over Sudan and the underestimation over Ethiopia may be ascribed to differences in the climates as discussed in Dinku et al. (2011) Fig. 6 compares the three products over Ethiopia and the Sudans. The rain area from the combined three inputs is larger than any of the other two cases. This must have come from ECMWF input. Fig. 6 also shows that the MPE-only product overestimates rainfall amounts, particularly over the Sudans. This overestimation can also be seen from Tables 4 and 5, particularly in Table 4, with higher bias values (Bias and ME). These results are very consistent with those of the previous section, which compared the three inputs of FAO algorithm over Sahel. 4.3. Comparison of FAO-RFE with other satellite products This section compares FAO-RFE with other satellite products available and used in Africa. Comparison is done by dividing the station data into calibration and validation sets. In practice, the FAO algorithm would use all available stations for bias removal and for merging with the satellite estimate. Thus, dividing station data into calibration and validation sets may not reflect the true quality of the FAO-RFE. However, it will still be useful to assess the overall performance of the FAO algorithm relative to the other approaches. Some of the satellite rainfall products may incorporate station data that are available through the global telecommunication systems (GTS). These stations are not part of the validation set. 4.3.1. Comparisons at daily time scale The main focus of evaluation at daily time scale would be assessing the rainfall detection skills of the products. Thus, the statistics used are those selected for this purpose. The evaluation will be done over Sudan, Ethiopia, and Sahel separately. 4.3.1.1. Comparison over Sudan. The Comparison over Sudan and South Sudan is done using point raingauge measurements because the available stations are not enough for interpolation and area

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

55

Fig. 6. Comparison outputs from the three different runs of the FAO algorithm for July 7, 2008.

averaging. This should be taken into account when comparing the satellite products, which are area averages, with point gauge measurements. This, for instance, may exaggerate the false alarm ratio or frequency bias statistics as point measurements tend to show more dry days compared to area averages. The statistics comparing the performance of the different products is presented in Fig. 7. The FAO-RFE product has a slightly higher POD, but it also has higher FAR. As a result, its HSS is slightly lower than the other products. The main weakness of FAO-RFE is that it significantly overestimates

Fig. 7. Comparison of validation statistics for the different products against point gauge measurements over Sudan.

occurrence or areal extent of rainfall (FBS = 1.71). This is consistent with the results of Section 4.2. All products have detection skills (POD) above 70%, but the false alarm rates (FAR) are also high (around 40%). All products, except for TAMSAT, overestimate the frequency of rainfall occurrence (FBS N 1). The Heidke Skill Score is above 0.5 for all the products, which is reasonable for daily rainfall. The correlation coefficients are very low for all the products. RFE and TAMSAT underestimate rainfall amounts (bias b 1), while CMORPH and FAO-RFE overestimate rainfall amounts. Fig. 8 compares the empirical CDFs for the different products against that of the station measurements. This would show how well the distributions of the satellite estimates match those of gauge measurements. The CPC-RFE and TAMSAT products miss the occurrence of the high rainfall values, while CMORPH and FAO-RFE overestimate the frequency of low rainfall intensities. The FAO-RFE has good match with that of the raingauge except at very low rain rates, which is consistent with overestimation of rainfall occurrences. The TMPA product has good match with station data at all rain rates. 4.3.1.2. Comparison over Ethiopia. The comparison over Ethiopia was done using area-average station data. This was possible because a dense station network was available for validation (Fig. 2). Fig. 9 presents different statistics comparing the performances of the different products relative to the reference gauge data. All products have relatively good POD values with low FAR. This is also reflected in the higher HSS values. All the products underestimate frequency of rainfall occurrence (FBS b1) except for FAO-RFE. As mentioned in Section 4.2,

56

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

Fig. 8. Cumulative density function (CDF) comparing the different satellite products (red) against gauge measurements (blue) over Sudan.

underestimation of rainfall occurrences over Ethiopia by the satellite estimates is compensated for by overestimation by the ECMWF product. The correlation coefficients are very low for all the products. Most of

Fig. 9. Comparison of validation statistics for the different products against area-average gauge measurements over Ethiopia.

the products underestimate rainfall amounts (bias b 1) except for FAO-RFE. Comparing FAO-RFE with other products, FOA-RFE has slightly better HSS, CC and Bias values. The use of a good number of stations for algorithm calibrations could have contributed to this. The cumulative density functions in Fig. 10 compare the distribution of the satellite rainfall amounts with that of area-average gauge. Except for TMPA and FAO-RFE, the satellite products have similar distribution with the reference gauge data. The FAO-RFE product overestimates the occurrence of high rainfall intensities. This is consistent with corresponding bias values in Fig. 9, which shows that FAO-RFE overestimates rainfall amounts. 4.3.1.3. Comparison over the Sahel. The evaluation over the Sahel uses over 250 raingauge stations, which does not include synoptic stations data available through the GTS system. The GTS data are used for calibrating the FAO-RFE over this region. Fig. 11 compares FAO-RFE with other satellite products. The performance of the FAO-RFE over this region is similar to other satellite products except that it has higher FBS and FAR statistics (FBS = 1.47, FAR = 0.44). The main issue is again overestimation of rainfall occurrence. As a result, FAO-RFE has also a relatively lower skill (HSS). The performance of the other satellite products

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

57

Fig. 10. Cumulative density function (CDF) comparing the different satellite products (red) against are-average gauge measurements (blue) over Ethiopia.

is similar except that CMORPH significantly overestimates rainfall amount (bias = 1.6). The CDF in Fig. 12 also shows that FAO-RFE overestimates the frequency of low rain rates. Otherwise, the CDF of FAORFE matches that of the raingauge measurements. The CDFs for the other products are somewhat similar except that of CMORPH, which exhibits higher rainfall amounts than the ground measurements.

4.3.2. Evaluation at ten-day time scale Here the main focus is evaluating the products in estimating rainfall amounts. Thus, the statistics used are those selected for this purpose.

Fig. 11. Comparison of validation statistics for the different products against area-average gauge measurements over Sahel.

4.3.2.1. Evaluation over Ethiopia. Table 6 presents the different statistics used to compare the performances of the different satellite rainfall products relative to reference raingauge data. The correlation coefficients, though still low, are better than those at daily time scale. The Eff statistic shows positive skill for all products except the TMPA product. All products underestimate rainfall amount except for FAO-RFE. Lower mean bias and slightly lower MAE are the main strength of FAO-RFE relative to the other products. The performance of the other products is similar except that TMPA has lower CC and Eff and exhibits low skill. Fig. 13 compares the scatter plots for the different products vs. pixelaverage gauge. The FAO-RFE exhibits a better performance compared to other products except for a slight overestimation of high rainfall values. Among the other products, TMPA shows wide scatter while TAMSAT misses high rainfall amounts 4.3.2.2. Evaluation over the Sahel. The statistics comparing the performance of the different satellite products over Sahel are presented in Table 7. The correlation coefficients are very similar for all the products. There are big differences among the Eff statistics. The RFE and TAMSAT products have significantly higher Eff values compared to the other products. The Eff values of the other products are lower than what is expected for comparison at ten-daily time scales. The CMORPH product has the lowest skill and the highest bias. Comparing FAO-RFE with the other products, the CC and bias are similar to other products, while its skill is much lower than RFE and TAMSAT. The RFE and TAMSAT products have relatively low MAE values, while CMORPH has the highest. The TMPA and FAO-RFE have similar MAE values. The scatter plot

58

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

Fig. 12. Cumulative density function (CDF) comparing the different satellite products (red) against are-average gauge measurements (blue) over Sahel.

comparing the different products is given in Fig. 14. RFE exhibits the best performance relative to the other products while TMPA, CMORPH and FAO-RFE overestimate high rainfall amounts. The over estimation is more sever for CMORPH. As in the case of Ethiopia, TAMSAT misses high rainfall amounts. These results are similar to those obtained by other research over the same region (e.g. Jobard et al., 2011). 5. Summary and conclusions The FAO satellite rainfall retrieval algorithm combines different rainfall data sets to generate daily estimates at a spatial resolution of about 3 km. The different inputs include a TIR-based rainfall product generated at FAO (TIR-RR), EUMETSAT's multisensor precipitation estimate

Table 6 Comparison of the different satellite products at ten-daily time scale over Ethiopia.

RFE TAMSAT TMPA CMORPH FAO-RFE

CC

Bias

MAE

Eff

0.67 0.69 0.55 0.71 0.70

0.67 0.72 0.75 0.72 0.95

31.1 26.7 30.0 28.2 25.1

0.10 0.24 −0.04 0.22 0.26

(MPE), ECMWF's forecast and station raingauge measurements. The FAO-RFE has been evaluated over Sudan, Ethiopia and the Sahel by comparing it with raingauge data and other widely used satellite rainfall estimates (CPC-RFE, TAMSAT, TMPA and CMORPH). The comparison has been done at daily and ten-daily time scales. Comparison of the individual inputs of the FAO algorithm has shown that the satellite products (particularly TIR-RR) overestimate rainfall amounts while the ECMWF product underestimates rainfall amounts and overestimates the areal coverage of the rainfall field. The contributions of the individual inputs to the final merged FAO-RFE were also evaluated. Combining MPE with TIR-RR did not show significant improvement while adding ECMWF to the other two does improve the merged product except that it increases the frequency bias over Sudan. The comparison of FAO-RFE with the other satellite products did not show significant improvement over Sudan and the Sahel. In fact FAO-RFE overestimates the occurrence rainfall in these areas. This is the main weakness of the FAO-RFE. It has been shown that this weakness comes from the forecast product (ECMWF rainfall field). This problem could be corrected by using a mask (0's and 1's) created from the MPE product, and multiplying the merged rainfall field by this mask. The picture is much better for the FAO-RFE over the Ethiopian highlands. It has a better or slightly better performance than most of the

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

59

Fig. 13. Scatter plots comparing different satellite products against area-average gauge over Ethiopia.

other products. The better performance over Ethiopia can be attributed, at least partly, to the significant number of stations used for the calibration/adjustment over Ethiopia (Fig. 1). The other factor is that overestimation of rainfall occurrence by the ECMWF product is compensated for by underestimation by the satellite products. The FAO-RFE has been developed for specific national applications by using rainfall measurements form local networks to improve the final product. The result from Ethiopia shows the advantage of using data from as many stations as possible. The main weakness of FAORFE in overestimating areal extent rainfall is significantly improved when more stations are used The FAO-RFE algorithm can be run on a common personal computer to produce daily rainfall estimate over a large area. Thus, it can be installed and run at the National Meteorological Agencies in Africa.

Table 7 Comparison of the different satellite products at ten-daily time scale over the Sahel.

RFE TAMSAT TMPA CMORPH FAO-RFE

CC

Bias

MAE

Eff

0.80 0.77 0.69 0.72 0.67

0.95 0.90 1.07 1.55 1.1

14.7 16.0 21.5 31.5 20.6

0.61 0.56 0.10 −0.25 0.08

Acknowledgments We would like to thank the AGRHYMET Centre and the National Meteorological Services of Ethiopia and The Sudan for providing the station data used in this paper. We also thank Michele Bernardi, Rene Gommes to have leaded the FAO RFE development at FAO. Juergen Grieser is also acknowledged to have contributed to the development of FAO RFE in its first step. This research was funding by the EU/FAO program on “Improved global governance for hunger reduction”.

References Brocca, L., Moramarco, T., Melone, F., Wagner, W., 2013. A new method for rainfall estimation through soil moisture observations. Geophys. Res. Lett. 40 (5), 853–858. Brocca, L., Ciabatta, L., Massari, C., Moramarco, T., Hahn, S., Hasenauer, S., Kidd, R., Dorigo, W., Wagner, W., Levizzani, V., 2014. Soil as a natural rain gauge: estimating global rainfall from satellite soil moisture data. J. Geophys. Res. 119 (9), 5128–5141. Chadwick, R., Grimes, D., 2012. An artificial neural network approach to multispectral rainfall estimation over Africa. J. Hydrometeorol. 13 (3), 913–931. Dinku, T., Schdzambwa, S., Ceccato, P., Connor, S.J., Ropelewski, C.F., 2008. Validation of highresolution satellite rainfall products over complex terrain. Int. J. Remote Sensing. 29 (14), 4097–4110. Dinku, T., Ceccato, P., Connor, S.J., 2010a. Comparison of CMORPH and TRMM-3B42 over mountainous regions of Africa and South America. In: Gebremichael, M., Hossaim, F. (Eds.), Satellite Rainfall Applications for Surface Hydrology. Springer, pp. 193–204. Dinku, T., Ceccato, P., Cressman, K., Connor, S.J., 2010b. Evaluation of daily satellite rainfall products over desert locust recession regions. J. Appl. Meteorol. Climatol. 49, 1322–1332. Dinku, T., Ceccato, P., Connor, S.J., 2011. Challenges to satellite rainfall estimation over mountainous and arid parts of East Africa. Int. J. Remote Sensing. 32 (21), 5965–5979.

60

T. Dinku et al. / Atmospheric Research 163 (2015) 48–60

Fig. 14. Scatter plots comparing different satellite products against area-average gauge over the Sahel.

Grimes, D.I.F., Pardo, E., Bonifacio, R., 1999. Optimal areal rainfall estimation using raingauges and satellite date. J. Hydrol. 222, 93–108. Habib, E., Elsaadani, M., Haile, A.T., 2012. Climatology-focused evaluation of CMORPH and TMPA satellite rainfall products over the Nile Basin. J. Appl. Meteorol. Climatol. 51, 22105–22121. Heinemannn, T., Lattanzio, A., Roveda, F., 2002. The EUMETSAT multi-sensor precipitation estimate (MPE). Proceedings of the second International Precipitation Working Group (IPWG) meeting, Madrid, Spain, September 2002. Hielkema, J.U., Snijders, F.L., 1994. Operational use of environmental satellite remote sensing and satellite communications technology for global food security and locust control by FAO: The ARTEMIS and DIANA systems. Acta Astronaut. 32, 603–616. Hirpa, F.A., Gebremichael, M., 2009. Evaluation of high-resolution satellite precipitation products over very complex terrain in Ethiopia. J. Appl. Meteorol. Climatol. 49, 1044–1051. Hoefsloot, P., 1995. Manual for IDA v4.2. Technical Report GCP/INT/578/NET. FAO, Rome. Huffman, G.J., Adler, R.F., Bolvin, D.T., Gu, G., Nelkin, E.J., Bowman, K.P., Hong, Y., Stocker, E.F., Wolf, D.B., 2007. The TRMM Multisatellite Precipitation Analysis (TMPA): Quasi-Global, Multiyear, Combined-Sensor Precipitation Estimates at Fine Scales. J. Hydrometeorol. 8, 38–55. Jobard, I., Chopin, F., Berges, J.C., Roca, R., 2011. An intercomparison of 10-day satellite precipitation products during West African monsoon. Int. J. Remote Sens. 32 (9), 2353–2376. Joyce, R.J., Janowiak, J.E., Arkin, P.A., Xie, P., 2004. CMORPH: a method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeorol. 5, 487–503. Kidd, C., Huffman, G., 2011. Review—global precipitation measurement. Meteorol. Appl. 18, 334–353. Kidd, C., Levizzani, V., 2011. Status of satellite precipitation retrievals. Hydrol. Earth Syst. Sci. 15, 1109–1116. Lazri, M., Ameur, Z., Ameur, S., Mohia, Y., Brucker, J.M., Testud, J., 2013. Rainfall estimation over a Mediterranean region using a method based on various spectral parameters of SEVIRI-MSG. Adv. Space Res. 52 (8), 1450–1466.

Lazri, M., Ameur, S., Mohia, Y., 2014. Instantaneous rainfall estimation using neural network from multispectral observations of SEVIRI radiometer and its application in estimation of daily and monthly rainfall. Adv. Space Res. 53 (1), 138–155. Mishra, A.K., Gairola, R.M., Varma, V.K., 2011. Improved rainfall estimation over the Indian region using satellite infrared technique. Adv. Space Res. 48 (1), 49–55. Robinson, I., Negrón, J., Li, W., Fu, R., Fernandes, K., 2009. Comparison of precipitation datasets over the tropical South American and African continents. J. Hydrometeorol. 10, 289–299. Roca, R., Chambon, P., Jobard, I., Kirstetter, P.E., Gosset, M., Berges, J.C., 2010. Comparing satellite and surface rainfall products over West Africa at meteorologically relevant scales during the AMMA campaign using error estimates. J. Appl. Meteorol. Climatol. 49, 715–731. Romilly, T.G., Gebremichael, M., 2011. Evaluation of satellite rainfall estimates over Ethiopian river basins. Hydrol. Earth Syst. 15, 1505–1514. Sultan, B., Janicot, S., 2003. The West African monsoon dynamics. Part II: The “preonset” and “onset” of the.summer monsoon. J. Clim. 16, 3407–3427. Symeonakis, E., Bonifacio, R., Drake, N., 2009. A comparison of rainfall estimation techniques for sub-Saharan Africa. Int. J. Appl. Earth Obs. Geoinf. 11, 15–26. Thorne, V., Coakeley, P., Grimes, D., Dugdale, G., 2001. Comparison of TAMSAT and CPC rainfall estimates with raingauges, for southern Africa. Int. J. Remote Sens. 22 (10), 1951–1974. Vicente, G.A., Scofield, R.A., Menzel, W.P., 1998. The operational GOES infrared rainfall estimation technique. Bull. Am. Meteorol. Soc. 79, 1883–1898. Wilks, D.S., 1995. Statistical Methods in the Atmospheric Sciences. Academic Press (467 pp.). Xie, P., Yarosh, Y., Love, T., Janowiak, J., Arkin, P.A., 2002. A real-time daily precipitation analysis over southern Asia. Preprints, 16th Conf. of the Hydro., Orlando, FL, Amer. Meteor. Soc. Xu, W., Adler, R.F., Wang, N., 2013. Improving geostationary satellite rainfall estimates using lightning observations: underlying lightning–rainfall–cloud relationships. J. Appl. Meteorol. Climatol. 52 (1), 213–229. Xu, W., Adler, R.F., Wang, N., 2014. Combining satellite infrared and lightning information to estimate warm-season convective and stratiform rainfall. Appl. Meteorol. Climatol. 53 (1), 180–199.