TMCnet News
Extending Extended Logistic Regression: Extended versus Separate versus Ordered versus Censored [Monthly Weather Review](Monthly Weather Review Via Acquire Media NewsEdge) ABSTRACT Extended logistic regression is a recent ensemble calibration method that extends logistic regression to provide full continuous probability distribution forecasts. It assumes conditional logistic distributions for the (transformed) predictand and fits these using selected predictand category probabilities. In this study extended logistic regression is compared to the closely related ordered and censored logistic regression models. Ordered logistic regression avoids the logistic distribution assumption but does not yield full probability distribution forecasts, whereas censored regression directly fits the full conditional predictive distributions. The performance of these and other ensemble postprocessing methods is tested on wind speed and precipitation data from several European locations and ensemble forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF). Ordered logistic regression performed similarly to extended logistic regression for probability forecasts of discrete categories whereas full predictive distributions were better predicted by censored regression. (ProQuest: ... denotes formulae omitted.) 1. Introduction Important applications such as severe weather warn- ings or decision making in agriculture, industry, and finance strongly demand accurate weather forecasts. Usually numerical weather prediction (NWP) models are used to provide these weather forecasts. Unfortunately, because of the only roughly known current state of the atmosphere and unknown or unresolved physical pro- cesses, NWP models are always subject to error. To es- timate these errors many forecasting centers nowadays provide ensemble forecasts. These are several NWP forecasts with perturbed initial conditions and/or different model formulations. However, the perturbed initial conditions do not necessarily represent initial condition uncertainty (Hamill et al. 2003; Wang and Bishop 2003) and some structural deficiencies in the models are also not accounted for. Thus, the ensemble forecasts usually do not represent the full uncertainty of NWP models. Ensemble forecasts therefore typically need to be statis- tically postprocessed to achieve well-calibrated probabi- listic forecasts. In the past decade a variety of different ensemble postprocessing methods have been proposed. Examples are ensemble dressing (Roulston and Smith 2003), Bayesian model averaging (Raftery et al. 2005), hetero- scedastic linear regression (Gneiting et al. 2005), or lo- gistic regression (Hamill et al. 2004). Comparisons of these and other postprocessing methods (Wilks 2006; Wilks and Hamill 2007) showed that logistic regres- sion performs relatively well. Recently, Wilks (2009) ex- tended logistic regression by including the (transformed) predictand thresholds as an additional predictor variable. In addition to requiring fewer coefficients and providing coherent probabilistic forecasts, this extended logistic re- gression allows derivation of full continuous predictive distributions. Extended logistic regression has been used frequently (Schmeits and Kok 2010; Ruiz and Saulo 2012; Roulin and Vannitsem 2012; Hamill 2012; Ben Bouallègue 2013; Scheuerer 2014; Messner et al. 2014) and has been further extended to additionally account for conditional heteroscedasticy (Messner et al. 2014). Recently, several studies noticed that extended logistic regression assumes a conditional logistic distribution for the transformed pre- dictand (Scheuerer 2014; Schefzik et al. 2013; Messner et al. 2014) where this logistic distribution is fitted to selected predictand category probabilities. In this study we compare (heteroscedastic) extended logistic regression with two closely related regression models from statistics that are particularly popular in econometrics (and more broadly in social sciences): 1) (Heteroscedastic) ordered logistic regression also provides coherent forecasts of category probabilities. However, it differs from extended logistic regression in that no continuous distribution is assumed or specified by the model. 2) (Heteroscedastic) censored regression also fits con- ditional logistic distributions to a transformed pre- dictand but employs the full set of training-data points (as opposed to a set of thresholds) for fitting the model. The performance of these statistical models is tested on wind speed and precipitation data from 10 European locations and ensemble forecasts from the European Centre for Medium-Range Weather Forecasts (ECMWF). In addition to heteroscedastic ordered logistic regres- sion, heteroscedastic extended logistic regression, and heteroscedastic censored logistic regression, also separate logistic regressions (Hamill et al. 2004) and for wind speed forecasts heteroscedastic truncated Gaussian regression (Thorarinsdottir and Gneiting 2010) are tested. In section 2 the different statistical models are de- scribed in detail. A brief description of the data can be found in section 3. Finally, section 4 presents the results and section 5 provides a summary and discussion. 2. Statistical models This section describes different statistical models to predict conditional probabilities P(y # qj j x) of a con- tinuous predictand y falling below a threshold qj, given a vector of predictor variables x 5 (1, x 1, x 2, ...)T (i.e., NWP forecasts). Conditional category probabilities of y to fall between two thresholds qa and qb can then easily a. Separate logistic regressions (SLR) Logistic regression was one of the first statistical methods that were proposed to postprocess ensemble forecasts (Hamill et al. 2004). Originally it is a regression model from the generalized linear model framework (Nelder and Wedderburn 1972) to model the probability of binary responses: ... (1) where b 5 (b0, b1, b2, ...)T is a coefficient vector and L(^) 5 exp(^)/[1 1 exp(^)] is notationally equivalent to the cumulative distribution function of the standard lo- gistic distribution. The coefficient vector b is estimated by maximizing the log-likelihood ... (2) as a function of b as defined in Eq. (1), where N is the number of events in the dataset and pi is the predicted probability of the ith observed outcome: ... (3) Often separate logistic regressions (i.e., with separate coefficient vectors b) are fitted for several thresholds qj of interest (e.g., Hamill et al. 2004; Wilks 2006; Wilks and Hamill 2007). This implies that the regression lines for different thresholds can cross, so that for some values of the predictor variables x, P(y # qa jx) .P(y # qbjx) although qa , qb which leads to nonsense negative probability for y to fall between qa and qb. b. Heteroscedastic extended logistic regression (HXLR) To avoid these negative probabilities and to reduce the number of regression coefficients Wilks (2009) proposed to include a transformation of the predictand thresholds as an additional predictor variable in logistic regression: ... (4) where a is an additional coefficient that has to be esti- mated and the transformation g( ) is a monotone function. Equation (4) also differs from standard logistic regression, where b is estimated separately for each threshold, in that here b is the same for all thresholds. Thus, one inter- pretation of Eq. (4) is that it defines parallel regression lines in log-odds space with equal slope but different in- tercepts [uj 5 ag(qj) 2 b0]. Figure 1 shows examples of these regression curves schematically. Extended logistic regression not only avoids the prob- lem of crossing regression lines but also allows for com- puting probabilities for any threshold value qj (and not only the thresholds employed for estimating the model). In other words, Eq. (4) can also be interpreted as a cu- mulative distribution function that describes a full con- tinuous predictive distribution. After some reformulation (see Messner et al. 2014), Eq. (4) can also be written as ... (5) which shows that the predictive distribution of the transformed predictand g(y) is a logistic distribution with location parameter xTb/a and scale parameter 1/a. Thus, the transformation g( ) must be chosen such that thetransformedpredictandcanbeassumedtofollow a conditional (on the predictors x) logistic distribution. To effectively utilize uncertainty information contained in the ensemble spread, Messner et al. (2014) proposed to use additional predictor variables (z 5 1, z1, z2, ...)T (e.g., the ensemble spread) to directly control the dispersion (variance) of the logistic predictive distribution: ... (6) where g 5 (g0, g1, g2, ...)T and d 5 (d0, d1, d2, ...)T are the coefficient vectors that have to be estimated. The exponential function is used as a simple method to ensure positive values (Messner et al. 2014). The coefficient vectors g and d are also estimated by maximizing the log-likelihood function given by Eq. (2). However, the probability of the observed outcome for the multicategorical predictand is ... (7) (Messner et al. 2014), where J is the number of thresholds qj that have been selected for the fitting calculation. c. Heteroscedastic ordered logistic regression (HOLR) Ordered logistic regression-also known as ordered logit, proportional odds logistic regression, or cumulative link model-is a popular regression model from statistics and econometrics for ordinal data, which has not received much attention in meteorology so far. Like extended lo- gistic regression it is an extension of standard logistic regression for multicategorical and ordered predictands. Different from extended logistic regression, separate in- tercepts uj are fitted for each selected threshold instead of modeling them as a linear function of the (transformed) thresholds: ... (8) where the estimated separate intercepts uj are only con- strained to be ordered (u1 # u2 # ? # uJ) for ordered thresholds qj. Because the intercepts of the regression lines are fully determined by uj further intercepts are not needed anymore so that x 5 (x1, x2, ...)T must not contain any constant. Similar to extended logistic regression b is the same for all thresholds. The separate intercepts for each threshold imply the estimation of more coefficients than for extended lo- gistic regression. Furthermore, only the probabilities for the thresholds qj employed in the estimation can be derived, so that Eq. (8) does not specify full continuous predictive distributions. In return, ordered logistic re- gression does not assume a continuous distribution for the transformed predictand. Thus, no (possibly non- existent) transformation has to be determined to fulfill this assumption. Similar to heteroscedastic extended logistic regression, a heteroscedastic version of ordered logistic regression also allows control of the scale (variance) of an under- lying latent distribution with additional predictor vari- ables (Agresti 2002): ... (9) Also, note that here no constant is needed in z 5 (z1, z2, ...)T. Maximum likelihood estimation with the same log- likelihood function as for extended logistic regression [Eqs. (2) and (7)] is used to estimate the coefficients uj, g, and d. d. Heteroscedastic censored logistic regression (HCLR) Above we have shown that extended logistic regression assumes a conditional logistic distribution for the trans- formed predictand. The maximum likelihood estimation with the log-likelihood function given by Eqs. (2) and (7) fits the selected category probabilities. However, if the predictand is given in continuous form, the model de- scribed by Eq. (6) can also be estimated with the log- likelihood function from Eq. (2) with ... (10) where l[^] denotes the likelihood function of the stan- dard logistic distribution. The likelihood is notationally identical to the probability density function [i.e., the derivative of Eq. (6) with respect to g(qj)], but differs because it is a function of the parameter vectors g and d for a fixed predictand value yi, rather than being a func- tion of yi given fixed values for g and d. In this way, the pi employed for fitting the model are not the likelihoods for predictands falling into discrete intervals, but rather the likelihoods that they take on their exact observed values. This model can also be interpreted as a linear regression model with a (heteroscedastic) logistic error distribution. Nonnegative variables (e.g., wind speeds or precipi- tation amounts) are only continuous for positive values and have a natural threshold at 0. This nonnegativity can easily accommodated using censored regression (first discussed by Tobin 1958, for the Gaussian case) where the pi are replaced by ... (11) in Eq. (2). This heteroscedastic censored logistic regression fits a logistic error distribution with point mass at zero to the transformed predictand. While such an error distribu- tion seems reasonable for square root transformed precipitation amounts (Scheuerer 2014; Schefzik et al. 2013), usually other error distributions are assumed for wind speed. For example Thorarinsdottir and Gneiting (2010) proposed to fit a truncated normal distribution to the untransformed wind speed. In this case, in Eqs. (6) and (10) the logistic distribution is replaced with a trun- cated normal distribution and g(y)issettog(y) 5 y.Note that Thorarinsdottir and Gneiting (2010) also called this model heteroscedastic censored regression although actually the data are considered to be truncated and not censored. In the following we therefore denote this model as heteroscedastic truncated Gaussian regression (HTGR), which we also employ as benchmark model for wind speed. e. Comparison Table 1 summarizes the major differences between the four different logistic regression models that were presented above. Extended logistic regression (XLR) and censored logistic regression (CLR) (and their het- eroscedastic versions HXLR and HCLR, respectively) are essentially the same models and only differ in their parameter estimation. They have the fewest parameters of the compared models but imply continuous distribu- tion assumptions. Ordered logistic regression (OLR) and its heteroscedasticversion (HOLR) avoid this continuous distribution assumption but require estimation of more coefficients than (H)XLR and (H)CLR. With its un- constrained slope estimates, separate logistic regressions SLR is more flexible than OLR but requires estimation of even more coefficients. Figure 1 shows schematic parallel regression lines for XLR, CLR, or OLR. In contrast to these models, regression curves from SLR would not be constrained to be parallel and so could potentially cross, which would lead to nonsense negative probabilities. 3. Data To compare the presented ensemble postprocessing methods, we used 10-m wind speed observations (10-min average) and 24-h accumulated precipitation amount from the 10 European weather stations: Wien-Hohe- Warte, Austria (48.2498N, 16.3568E); Paris-Orly, France (48.7178N, 2.3838E); Amsterdam-Schiphol, Netherlands (52.38N, 4.7838E); Berlin-Tegel, Germany (52.558N, 13.38E); Brussels-National, Belgium (50.98N, 4.5338E); Frankfurt-Main,Germany(50.0338N,8.5838E);London- Heathrow, United Kingdom (51.4678N, 20.458E); Lisbon-Geof,Portugal(38.7678N,29.1338E); Madrid- Barajas, Spain (40.4678N, 23.558E); and Rome-Fiumicino, Italy (41.88N, 12.2338E). As input for the statistical models, 10-m wind speed and total precipitation ensem- ble forecasts from the ECMWF were linearly in- terpolated from neighboring grid points to the station locations. The data were available from April 2010 to December 2012 (approximately 1000 days) and separate models were fitted for the lead times 24, 48, and 96 h, respectively. Since the predictands were square root transformed for most regression models (see section 4) we mainly used the mean and standard deviation of square root trans- formed ensemble forecasts as predictor variables. For HTGR the untransformed predictand is used, following Thorarinsdottir and Gneiting (2010). Consequently we employed the mean and standard deviation of the un- transformed ensemble forecasts as input for this model. As thresholds qj we defined J 5 9 climatological deciles that are estimated for each location and predictand variable separately. Note that for precipitation several deciles are 0 and are merged to one threshold so that the effective number of thresholds is smaller (e.g., J 5 4in Wien-Hohe-Warte and J 5 5 in Paris-Orly for pre- cipitation). We found the ensemble standard deviation to im- prove the forecasts of all statistical models, indicating useful spread-skill relationships. Therefore, we only show results for the heteroscedastic models in the fol- lowing. For separate logistic regressions the product of ensemble mean and spread is included as additional predictor variable (Wilks and Hamill 2007). Table 2 lists the different models that are compared in the following in detail. 4. Results Before comparing the performance of the different ensemble postprocessing methods we show how ordered logistic regression can be used to determine appropriate transformations g( ) for extended logistic regression. The crosses and plus signs in Fig. 2 show the fitted in- tercepts from ordered logistic regression (HOLR) for two predictands and two selected locations. For both locations and variables these plots suggest that the in- tercepts can be parameterized as being proportional to the square roots of the thresholds. Thus, we fitted HXLR models with g(qj ) 5 pffiqffiffijffi and added the corresponding HXLR intercept functions uj 5 b0 1 apffiqffiffijffi as curves in Fig. 2. For both predictand variables and locations the HXLR intercept functions fit the HOLR intercepts rea- sonably well. Note that similar figures can also be used to compare the intercepts of extended logistic regression with those of separate logistic regression (e.g., Ruiz and Saulo 2012). However, the varying slope coefficients then complicate the comparison. Figure 2 already suggests that HXLR and HOLR predict similarly well. In the following we compare these and the other statistical models more thoroughly. Be- cause all models provide probabilistic forecasts for dis- crete intervals we mainly employ the ranked probability score (RPS; Epstein 1969; Wilks 2011) to characterize forecast accuracy: ... (12) where J is the number of thresholds and I(^) is the in- dicator function. For each model, forecast location, and lead time we applied 10-fold cross validation to get in- dependent training and test datasets. Therefore, the data are divided into 10 equally sized blocks and in each block the RPS were computed for the models that were trained on the 9 remaining blocks, respectively. Conse- quently, the effective training data length is 9/10 of the full dataset (approximately 900). To estimate the sampling distribution for the average RPS we computed means of 250 bootstrap samples. To compare the models with a reference model we finally computed ranked probability skill scores (RPSS): ... (13) where RPSref is the RPS of appropriate reference forecasts. Figure 3 shows the RPSS relative to HXLR for dif- ferent models, lead times, locations, and predictand variables. HOLR performs equally well or slightly bet- ter than HXLR for all locations, lead times, and pre- dictand variables. For precipitation in Paris forecasts of HXLR and HOLR are nearly identical, which is con- sistent with Fig. 2 where the HXLR intercept function almost perfectly interpolates the HOLR intercepts. SLR generally performs worse than HXLR, exceptions are wind speed forecasts in Wien for 24- and 96-h lead time and precipitation forecasts in Paris for 24-h lead time. However, note that the RPS [Eq. (12)] does not pe- nalize the partly inconsistent forecasts from SLR. HCLRandHTGRalsotendtoperformworsethan HXLR, especially for wind speed. While for Paris HTGR is slightly better than HCLR there is no clear preference for one of these models in Wien or the ag- gregated locations. For Fig. 3 we used nine climato- logical deciles as thresholds qj for estimation and verification. We additionally also tested different other numbers of climatological quantiles. However, apart from SLR and HOLR reaching slightly better skills for fewer quantiles, the results are very similar and therefore not shown. Because the different statistical models differ con- siderably in their number of estimated coefficients (SLR: 3J, HOLR: 2 1 J, HXLR, HCLR, HTGR: 4) it is also interesting to compare their performance for dif- ferent training data lengths. Figure 4 shows RPSS for wind speed and precipitation forecasts for 48-h lead time at Wien and Paris, relative to the raw ensemble interval relative frequencies. Similar to Fig. 3 the RPS are com- puted with 10-fold cross validation but for each test sample, only a subset of the remaining data are used for training. It can be seen that almost all models lose skill with a reduced training dataset. With the largest parameter count SLR clearly loses most. In contrast HOLR generally exhibits comparable skill reductions as HXLR and HCLR in response to decreasing training data although more parameters have to be estimated. Interestingly, for wind speed in Paris the skill of HCLR seems not to depend on the training data length and is therefore superior to the other models for short training datasets. HCLR basically fits the same model as HXLR, with the only difference being that the estimated model pa- rameters optimize either the selected category probabil- ities (HXLR) or the continuous predictive distribution (HCLR). Since the RPS only measures the quality of the selected category probabilities the better RPS of HXLR in Fig. 3 is not surprising. To compare also the quality of the full predictive distributions we therefore employ the continuous ranked probability score (CRPS; Matheson and Winkler 1976; Hersbach 2000; Wilks 2011) that gen- eralizes the RPS to full predictive distributions: ... (14) To avoid finding closed forms of the integral in Eq. (14) we solved these integrals numerically. Figure 5 shows the continuous ranked probability skill score (CRPSS) relative to HXLR for lead time 48 h. Results for the other lead times are very similar and therefore not shown. In contrast to the RPSS (Fig. 3) the CRPSS clearly favors HCLR for both predictand variables. For wind speed, Fig. 5 also shows the CRPSS for HTGR. As in Fig. 3 HCLR and HTGR show similar CRPSS for Wien while HTGR is slightly preferred for Paris and the aggregated locations, which could indicate that the real error distribution is better estimated by a truncated normal than by a censored transformed logistic distribution. Since HXLR fits the selected category probabilities it is also interesting how the choice of the thresholds that define these categories affects the quality of the pre- dictive distribution. Figure 6 shows the CRPSS of HCLR relative to HXLR for different numbers of climatological quantiles that are used to fit HXLR. Since HCLR could also be interpreted as HXLR with infinitesimal category intervals it is not surprising that the CRPS of HXLR and HCLR become more similar with higher numbers of thresholds. Although the patterns look similar for both predictand variables, precipitation forecasts lose much more skill for few thresholds. Finally, Figs. 7 and 8 show reliability diagrams (e.g., Wilks 2011) for the lower and upper climatological deciles, respectively, for 48-h lead time at Wien. With few exceptions the observed conditional relative fre- quencies of both predictand variables lie within the 95% consistency intervals (Bröcker and Smith 2007) with only minor differences between the different statistical models. The refinement distributions in Figs. 7 and 8 show the frequencies of the predicted probabilities. Similar to the calibration function the different models show only minor differences. Only for zero precipitation SLR and HOLR have slightly sharper forecasts than HXLR and HCLR (forecasts more frequently close to 0 and 1). 5. Summary and conclusions Extended logistic regression fits predictand category probabilities by assuming a conditional logistic distri- bution for the transformed predictand (Scheuerer 2014; Schefzik et al. 2013; Messner et al. 2014). However, for some applications the transformed predictand cannot be assumed to follow a logistic distribution. Moreover, fit- ting selected category probabilities implies disregarding available information when the predictand is actually given in continuous form. In this study we compared extended logistic regres- sion with two closely related regression models from statistics and econometrics. Ordered logistic regression is very similar to extended logistic regression but avoids a continuous distribution assumption. On the other hand, censored logistic regression fits the same model as ex- tended logistic regression but uses each individual pre- dictand value in the training dataset instead of the selected category probabilities. As further benchmark models we also employed separate logistic regressions and a truncated Gaussian regression model (Thorarinsdottir and Gneiting 2010). The performance of the different statistical models was tested with wind speed and pre- cipitation data from 10 European locations and ensemble forecasts from the ECMWF. Overall, the logistic distri- bution assumption seemed to be quite appropriate for the square root-transformed predictands for both predictand variables. Thus, the performance differences between ordered and extended logistic regression were only minor. However, because no continuous distribution has to be assumed, ordered logistic regression should generally be preferred if solely threshold probabilities are required. Since extended logistic regression fits selected cate- gory probabilities, it is actually not surprising that RPS skills are higher for this model than for censored logistic regression, which fits the full continuous predictive dis- tribution. For the same reason it is unsurprising that censored logistic regression performed better than ex- tended logistic regression according to CRPS skill, which evaluates accuracy of the full predictive distributions. Extended and censored logistic regression assume cen- sored conditional logistic distributions for the transformed predictand. In contrast, wind speed was assumed to follow a truncated normal distribution in Thorarinsdottir and Gneiting (2010). A comparison between censored and truncated regression models showed that the assumption of a truncated normal distribution resulted in slightly better wind speed forecasts than the assumption of a censored transformed logistic distribution. Our results show that the optimal statistical model strongly depends on the intended application. Ordered logistic regression was best suited for category probability predictions for the forecasts considered here, given suf- ficiently long training series. When the transformed pre- dictand can be assumed to follow a conditional logistic distribution then extended logistic regression provides equally good category probability forecasts while re- quiring fewer coefficients and additionally specifying full predictive distributions. However, if the primary interest is in predicting full continuous probability distributions, censored or truncated regression models should be pre- ferred because they use the information contained in the training data more fully. Acknowledgments. We thank three anonymous re- viewers for their valuable comments to improve this manuscript. This study was supported by the Austrian Science Fund (FWF): L615-N10. The first author was also supported by a Ph.D. scholarship from the Uni- versity of Innsbruck, Vizerektorat für Forschung. Data from the ECMWF forecasting system were obtained from the ECMWF Data Server. Denotes Open Access content. REFERENCES Agresti, A., 2002: Categorical Data Analysis. 2nd ed. John Wiley & Sons, 734 pp. Ben Bouallègue, Z., 2013: Calibrated short-range ensemble pre- cipitation forecasts using extended logistic regression with interaction terms. Wea. Forecasting, 28, 515-524, doi:10.1175/ WAF-D-12-00062.1. Bröcker, J., and L. A. Smith, 2007: Increasing the reliability of re- liability diagrams. Wea. Forecasting, 22, 651-661, doi:10.1175/ WAF993.1. Christensen, R. H. B., 2013: ordinal: Regression Models for Ordi- nal Data, version 2013.09-30. R package. [Available online at http://CRAN.R-project.org/package5ordinal.] Epstein, E. S., 1969: A scoring system for probability forecasts of ranked categories. J. Appl. Meteor., 8, 985-987, doi:10.1175/ 1520-0450(1969)008,0985:ASSFPF.2.0.CO;2. Gneiting, T., A. E. Raftery, A. H. Westveld, and T. Goldman, 2005: Calibrated probabilistic forecasting using ensemble model output statistics and minimum CRPS estimation. Mon. Wea. Rev., 133, 1098-1118, doi:10.1175/MWR2904.1. Hamill, T. M., 2012: Verification of TIGGE multimodel and ECMWF reforecast-calibrated probabilistic precipitation forecasts over the contiguous United States. Mon. Wea. Rev., 140, 2232-2252, doi:10.1175/MWR-D-11-00220.1. _____, C. Snyder, and J. S. Whitaker, 2003: Ensemble forecasts and the properties of flow-dependent analysis-error covariance singular vectors. Mon. Wea. Rev., 131, 1741-1758, doi:10.1175// 2559.1. _____,J.S.Whitaker,andX.Wei,2004:Ensemblereforecasting: Improving medium-range forecast skill using retrospec- tive forecasts. Mon. Wea. Rev., 132, 1434-1447, doi:10.1175/ 1520-0493(2004)132,1434:ERIMFS.2.0.CO;2. Hersbach, H., 2000: Decomposition of the continuous ranked probability score for ensemble prediction systems. Wea. Fore- casting, 15, 559-570, doi:10.1175/1520-0434(2000)015,0559: DOTCRP.2.0.CO;2. Matheson, J. E., and R. L. Winkler, 1976: Scoring rules for con- tinuous probability distributions. Manage. Sci., 22, 1087-1096, doi:10.1287/mnsc.22.10.1087. Messner, J. W., and A. Zeileis, 2013: crch: Censored Regression with Conditional Heteroscedasticity, version 0.1-0. R package. [Available online at http://CRAN.R-project.org/ package5crch.] _____, _____, G. J. Mayr, and D. S. Wilks, 2014: Heteroscedastic extended logistic regression for postprocessing of ensem- ble guidance. Mon. Wea. Rev., 142, 448-456, doi:10.1175/ MWR-D-13-00271.1. Nelder, J. A., and R. W. M. Wedderburn, 1972: Generalized linear models. J. Roy. Stat. Soc., 135A, 370-384, doi:10.2307/ 2344614. Raftery, A. E., T. Gneiting, F. Balabdaoui, and M. Polakowski, 2005: Using Bayesian model averaging to calibrate forecast ensembles. Mon. Wea. Rev., 133, 1155-1174, doi:10.1175/ MWR2906.1. R Core Team, 2013: R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. [Available online at http://www.R-project.org/.] Roulin, E., and S. Vannitsem, 2012: Postprocessing of ensemble precipitation predictions with extended logistic regression based on hindcasts. Mon. Wea.Rev., 140, 874-888, doi:10.1175/ MWR-D-11-00062.1. Roulston, M. S., and L. A. Smith, 2003: Combining dynamical and statistical ensembles. Tellus, 55A, 16-30, doi:10.1034/ j.1600-0870.2003.201378.x. Ruiz, J. J., and C. Saulo, 2012: How sensitive are probabilistic precipitation forecasts to the choice of calibration algorithms and the ensemble generation method? Part I: Sensitivity to calibration methods. Meteor. Appl., 19, 302-313, doi:10.1002/ met.286. Schefzik, R., T. L. Thorarinsdottir, and T. Gneiting, 2013: Un- certainty quantification in complex simulation models using ensemble copula coupling. Stat. Sci., 28, 616-640, doi:10.1214/ 13-STS443. Scheuerer, M., 2014: Probabilistic quantitative precipitation fore- casting using ensemble model output statistics. Quart. J. Roy. Meteor. Soc., 140, 1086-1096, doi:10.1002/qj.2183. Schmeits, M. J., and K. J. Kok, 2010: A comparison between raw ensemble output, (modified) Bayesian model averaging, and extended logistic regression using ECMWF ensemble pre- cipitation reforecasts. Mon. Wea. Rev., 138, 4199-4211, doi:10.1175/2010MWR3285.1. Thorarinsdottir, T. L., and T. Gneiting, 2010: Probabilistic fore- casts of wind speed: Ensemble model output statistics by using heteroscedastic censored regression. J. Roy. Stat. Soc., 173A, 371-388, doi:10.1111/j.1467-985X.2009.00616.x. Tobin, J., 1958: Estimation of relationships for limited de- pendent variables. Econometrica, 26, 24-36, doi:10.2307/ 1907382. Wang, X., and C. H. Bishop, 2003: A comparison of breeding and ensemble transform Kalman filter ensemble forecast schemes. J. Atmos. Sci., 60, 1140-1158, doi:10.1175/ 1520-0469(2003)060,1140:ACOBAE.2.0.CO;2. Wilks, D. S., 2006: Comparison of ensemble-MOS methods in the Lorenz '96 setting. Meteor. Appl., 13, 243-256, doi:10.1017/ S1350482706002192. _____, 2009: Extending logistic regression to provide full-probability- distribution MOS forecasts. Meteor. Appl., 16, 361-368, doi:10.1002/met.134. _____, 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Academic Press, 676 pp. _____, and T. M. Hamill, 2007: Comparison of ensemble-MOS methods using GFS reforecasts. Mon. Wea. Rev., 135, 2379- 2390, doi:10.1175/MWR3402.1. JAKOB W. MESSNER AND GEORG J. MAYR Institute of Meteorology and Geophysics, University of Innsbruck, Innsbruck, Austria DANIEL S. WILKS Department of Earth and Atmospheric Sciences, Cornell University, Ithaca, New York ACHIM ZEILEIS Department of Statistics, Faculty of Economics and Statistics, University of Innsbruck, Innsbruck, Austria (Manuscript received 25 October 2013, in final form 24 February 2014) Corresponding author address: Jakob W. Messner, Institute of Meteorology and Geophysics, University of Innsbruck, Innrain 52f, 6020 Innsbruck, Austria. E-mail: [email protected] DOI: 10.1175/MWR-D-13-00355.1 APPENDIX Computational Details Our results were obtained on Ubuntu Linux using the statistical software R 2.15.2 (R Core Team 2013). Het- eroscedastic extended logistic regression and hetero- scedastic censored logistic regression were fitted using the package crch 0.1-0 (Messner and Zeileis 2013). For ordered logistic regression models we used the package ordinal 2012.09-11 (Christensen 2013). (c) 2014 American Meteorological Society |