TMCnet News
Evaluation of Ensemble Configurations for the Analysis and Prediction of Heavy-Rain-Producing Mesoscale Convective Systems* [Monthly Weather Review](Monthly Weather Review Via Acquire Media NewsEdge) ABSTRACT This study investigates probabilistic forecasts made using different convection-allowing ensemble configurations for a three-day period in June 2010 when numerous heavy-rain-producing mesoscale convective systems (MCSs) occurred in the United States. These MCSs developed both along a baroclinic zone in the Great Plains, and in association with a long-lived mesoscale convective vortex (MCV) in Texas and Arkansas. Four different ensemble configurations were developed using an ensemble-based data assimilation system. Two configurations used continuously cycled data assimilation, and two started the assimilation 24 h prior to the initialization of each forecast. Each configuration was run with both a single set of physical parameterizations and a mixture of physical parameterizations. These four ensemble forecasts were also compared with an ensemble run in real time by the Center for the Analysis and Prediction of Storms (CAPS). All five of these ensemble systems produced skillful probabilistic forecasts of the heavy-rain-producing MCSs, with the ensembles using mixed physics providing forecasts with greater skill and less overall bias compared to the single-physics ensembles. The forecasts using ensemble-based assimilation systems generally outperformed the real-time CAPS ensemble at lead times of 6-18 h, whereas the CAPS ensemble was the most skillful at forecast hours 24-30, though it also exhibited a wet bias. The differences between the ensemble precipitation forecasts were found to be related in part to differences in the analysis of the MCV and its environment, which in turn affected the evolution of errors in the forecasts of the MCSs. These results underscore the importance of representing model error in convection-allowing ensemble analysis and prediction systems. 1. Introduction There continues to be a need for reliable probabilistic forecasts of hazardous convective weather phenomena (e.g., National Research Council 2010), owing to the limited skill of deterministic forecasts of convection (e.g., Zhang et al. 2003). Fortunately, modern comput- ing systems are now making it possible to run ensembles of forecasts at grid spacings where convection is ex- plicitly represented and at least marginally resolved. [Hereafter, we refer to model configurations with hori- zontal grid spacing from 1-5 km as ''convection allowing,'' following Clark et al. (2009).] Past studies have established that even relatively small ensembles at these resolutions tend to produce more skillful forecasts of heavy pre- cipitation than larger, coarse-resolution ensembles (e.g., Clark et al. 2009). However, how to best design, imple- ment, and analyze such storm-scale ensembles remains a subject of active research. For global medium-range ensemble prediction systems, the diversity in forecast outcomes primarily comes from perturbations to the initial conditions (ICs), as these uncertainties outweigh uncertainties in model physics on longer time scales (e.g., Gilmour et al. 2001). But at the temporal and spatial scales of convective weather, uncertainties in the representation of boundary layer and cloud- microphysical processes may be just as important as those associated with the ICs (e.g., Stensrud et al. 2000; Eckel and Mass 2005). Furthermore, there are myriad ways in which the diversity in the ICs and/or physical parameterizations can be implemented in a storm-scale ensemble. For example, the National Centers for Envi- ronmental Prediction (NCEP) runs a regional ensemble, the short-range ensemble forecast (SREF; Du et al. 1997) system, which uses multiple model cores and model physics, but does not have sufficient resolution to explicitly predict convection. This research aims to examine the strengths and weaknesses of some of the methods for incorporating initial-condition uncertainty and diverse physical parameterizations into convection- allowing ensemble forecasts. Several research and operational forecasting centers around the world have been testing convection-allowing ensemble forecast systems in recent years (e.g., Xue et al. 2007, 2011; Hohenegger et al. 2008; Schwartz et al. 2010; Vié et al. 2011; Clark et al. 2012; Leoncini et al. 2013). One such ensemble system that has received substantial attention is the storm-scale ensemble fore- cast (SSEF) system run by the Center for the Analysis and Prediction of Storms (CAPS) at the University of Oklahoma since 2007 (e.g., Xue et al. 2007, 2011; Clark et al. 2012). This system has demonstrated promising results and has led to the development of numerous new analysis and postprocessing methods for ensembles (e.g., Clark et al. 2013). However, considering that the SSEF takes its initial and lateral boundary condition perturbations from selected members of an ensemble (the NCEP SREF) that uses different model physics and runs at coarser resolution, it is possible that improve- ments could be made by employing other methods for initializing the ensemble. In particular, the methods of ensemble-based data assimilation [as reviewed by Meng and Zhang (2011)] provide some attractive benefits for initializing a storm-scale ensemble, in that the assimi- lation of observations, the generation of the ensemble ICs, and the forecasts can all be run with a single inte- grated system. Furthermore, such an approach assimi- lates observations and develops initial perturbations in a flow-dependent manner; in other words, the way that assimilated observations influence their surrounding environment is a function of the atmospheric flow pat- tern at that given time. Recent works by Romine et al. (2013) and Schwartz and Liu (2014) presented the results of using a continu- ously cycled ensemble-based assimilation system to initialize deterministic convection-allowing forecasts. Romine et al. (2013) found that the choice of physical parameterizations in the assimilation system introduced unique biases into the analyses over time. Furthermore, they found that convection-allowing forecasts initialized with this system were generally more skillful than simi- lar forecasts initialized with the NCEP Global Forecast System (GFS) through forecast hours 12-18, but that the GFS-initialized forecasts performed better at longer lead times. The work presented in this study extends the work of Romine et al. (2013) by using a similar as- similation system but initializing convection-allowing ensemble forecasts, rather than single deterministic forecasts. Furthermore, their finding of consistent biases in their analyses resulting from specific parameterization schemes motivates the exploration of using diversity of parameterizations and a partial (rather than continuous) cycling strategy in the ensemble assimilation-forecast system as a potential way to mitigate these biases. The period of study will be 8-11 June 2010, when nu- merous mesoscale convective systems (MCSs) occurred in the central and southern United States that produced heavy to extreme precipitation amounts (Fig. 1a; Higgins et al. 2011; Schumacher et al. 2013). There were two primary focusing mechanisms for organized deep con- vection during this three-day period: a baroclinic zone that was oriented approximately from west to east across the Great Plains, and a mesoscale convective vortex (MCV) that moved northeastward from Texas into Arkansas. Some of the mechanisms responsible for the heavy rainfall associated with the MCV were addressed by Schumacher et al. (2013), including the importance of convective latent heating and low-level evaporative cooling in the maintenance of the MCV and the associated initiation and organization of convection. Schumacher et al. (2013) also presented some of the characteristics of the SSEF forecasts for this time period, finding that there was a wide range of forecast outcomes within the ensemble for the MCV-related rainfall. In this study, we will evaluate the SSEF forecasts in the context of four other ensemble configurations, with a particular focus on how diversity in ICs and physical parameterizations influence the forecasts in terms of both quantitative metrics and the representation of physical processes. Although we will focus on only a three-day period rather than an entire warm season, the variety of at- mospheric processes and heavy rainfall mechanisms that occurred during this time period should provide some insights into the strengths and weaknesses of these configurations. Section 2 provides a brief overview of the 8-11 June 2010 event, and section 3 discusses the configuration of the assimilation and forecast systems. The results of the ensemble-based assimilation itself are presented in section 4, and the evaluation of the ensemble forecasts is given in section 5. Section 6 provides discussion of the implications of these results and their potential appli- cations, and section 7 concludes the paper. 2. Overview of the 8-11 June 2010 period The large-scale 500-hPa flow over the United States during the period of interest was characterized by an upper-level trough over the northwestern part of the country, and a ridge over the central and southern United States. (Fig. 2a). There was also a weak trough over Texas, which is a reflection of the long-lived MCV that moved across this region (e.g., Schumacher et al. 2013). Associated with the midtropospheric height gradient across the central United States was a low-level bar- oclinic zone that extended eastward from the Rocky Mountains to the Ohio Valley (Fig. 2b). There was a broad region of southerly 850-hPa winds across the Great Plains with average speeds exceeding 7.5 m s21, and embedded regions of stronger low-level winds near the Oklahoma and Texas Panhandles and in south Texas near the Gulf of Mexico (Fig. 2b). These areas of strong low-level winds are a regular feature of the warm-season climatology in the Great Plains (e.g., Bonner 1968), and the average wind speeds during this three-day period were slightly stronger than the long-term average (not shown). Intersections of a low-level jet and a baroclinic zone are common breeding grounds for MCSs (e.g., Augustine and Caracena 1994; Tuttle and Davis 2006), and indeed there were MCSs that developed across the central United States during the evenings of 8, 9, and 10 June (Figs. 1b-d). Furthermore, MCSs developed each night in association with the MCV over Texas and Arkansas. On 8-9 June, an MCS developed along the baroclinic zone in eastern Kansas and western Missouri (MCS N1 in Fig. 1b), which moved slowly and produced heavy rainfall. Concurrently, MCS S1 developed and remained nearly stationary over central Texas (Fig. 1b). Both of these systems weakened during the day on 9 June, and two more MCSs developed on the evening of 9 June and persisted through the morning of 10 June, here indicated as MCS N2 in Nebraska and S2 in Texas (Fig. 1c). These MCSs mainly weakened during the day on 10 June, with yet another round of convective development on the evening of 10 June and the morning of 11 June (MCSs N3 and S3 in Fig. 1d). An animation of radar images from this full three-day period is available in the online supplemental material. These six MCSs, along with ad- ditional shorter-lived convective systems, produced broad swaths of more than 50 mm of rain with some local areas receiving more than 300 mm (Fig. 1a). Flash flooding was reported in many of the areas affected by the rainfall from these MCSs, including the deadly flash flood that occurred at the Albert Pike Recreational Area in Arkansas [(National Oceanic and Atmospheric Ad- ministration) NOAA 2013; Schumacher et al. 2013]. 3. Methods a. Data assimilation system and configuration To assimilate observations and develop ICs for en- semble forecasts, we use revision 5780 of the Data Assimilation Research Testbed (DART; Anderson et al. 2009) software package in conjunction with ver- sion 3.4.1 of the Advanced Research version of the Weather Research and Forecasting (WRF) Model (ARW; Skamarock et al. 2008). A 36-member ensemble is run at 36-km horizontal grid spacing and 51 vertical levels (with stretched grid and a model top at 50 hPa) during the assimilation process on a grid covering much of North America (Fig. 3). Multiple configurations of this ensemble are tested in this study, which will be discussed in detail in section 3b. DART is configured to use an ensemble adjustment Kalman filter (Anderson 2001). To maintain spread in the ensemble during the assimilation cycle, spatially and temporally varying in- flation is applied to the prior perturbations about the mean and a sampling error correction is applied (Anderson 2009). To eliminate spurious relationships far from the observation location due to sampling error, horizontal and vertical localization using a Gaspari and Cohn (1999) function is employed, with influence going to zero approximately 580 km away from the observa- tion in the horizontal and 900 hPa away in the vertical. If there are more than 1600 observations within this lo- calization region, the localization length scale is reduced linearly using the method of Torn (2010). The DART options used here are generally similar to those used by Romine et al. (2013), with the primary exceptions being that we use a smaller ensemble (36 vs 50 members) and a somewhat smaller localization radius on a coarser horizontal grid (36- vs 15-km grid spacing) but finer vertical resolution (51 vs 35 vertical levels). b. Data assimilation experiments Four different ensemble and assimilation strategies were tested and then used to initialize convection- allowing ensemble forecasts, and these configurations are summarized schematically in Fig. 4. The configu- rations with the ''single'' prefix use the same set of physics parameterizations for all members, whereas the ''mixphys'' configurations use a variety of physics pa- rameterizations within the ensemble, which are given in Table 1. The parameterizations used in the single ensembles were chosen because they were the param- eterizations used by the SSEF members with the best forecasts of the 8-11 June 2010 MCSs in the south- central United States in Schumacher et al. (2013).The specific choices of parameterizations in the mixed- physics ensemble are somewhat arbitrary, but are de- signed to provide considerable diversity among the schemes available in ARW and as one way to account for model error in the assimilation and forecast process (e.g., Fujita et al. 2007; Wheatley and Stensrud 2010; Meng and Zhang 2011). Romine et al. (2013) established the consistent biases that WRF-DART configurations with single physics parameterizations can introduce in data assimilation (along with the effects those biases can have on convection-allowing forecasts), and the mixphys experiments are designed to identify differences be- tween single-physics and mixed-physics assimilation and forecast systems. The single-physics configuration cho- sen here will, of course, introduce its own biases, though how those biases compare to other potential choices of parameterizations in a single-physics assimilation and forecast system has not been extensively tested here. The use of a mixed-physics configuration that includes a mix of single-moment and double-moment micro- physical parameterizations means that the number concentration variables cannot be used as state variables for the ensemble filter. This is perhaps not a desirable scenario, but experimentation showed that the influence of the initial number concentrations on the precipitation forecasts were less than those that would be seen with a run initialized from a ''cold start,'' in which no hy- drometeors are present in the ICs. The results of these experiments, along with other discussion of this issue, are presented in appendix A. For consistency with the mixed-physics results, the number concentrations are not used as state variables in the single-physics config- uration either. The configurations with the ''cc'' suffix are continu- ously cycled starting at 0000 UTC 6 June 2010, whereas the ''24hr'' configurations are initialized every 24 h starting at 0000 UTC 7 June. As in Torn and Hakim (2008), all of the ensemble assimilation configurations are initialized with the 36-h forecast of the NCEP GFS valid at that time, which are perturbed with random draws from the NCEP background error covariance using the WRF data assimilation system (Barker et al. 2012) to produce an initial ensemble. In the cc configu- rations, this process only takes place once, at 0000 UTC 6 June, whereas in the ''24hr'' experiments this process is repeated every 24 h. These experiments are designed to test two possible ensemble configurations that can ini- tialize a convection-allowing forecast ensemble, and to examine differences between a continuously cycled system and one that draws the initial ensemble from an independent source more frequently. All of the WRF-DART configurations also use per- turbed 36-h GFS forecasts as lateral boundary condi- tions (LBCs) using the fixed covariance perturbation method described by Torn et al. (2006). The pertur- bations are applied every 6 h through the assimila- tion cycle with the 36-h GFS forecast valid at that time, and the same perturbed LBCs are used for all of the configurations. c. Assimilation of observations The WRF-DART system primarily assimilates ob- servations from the NCEP Global Data Assimilation System (GDAS), which are summarized in Table 2. This includes radiosondes (wind components, temperature, specific humidity, and altimeter setting are assimilated), surface observations from ships and buoys (wind com- ponents, temperature, specific humidity, and altimeter setting are assimilated), Aircraft Meteorological Data Relay (AMDAR; wind components and temperature are assimilated), and atmospheric motion vectors (AMVs; Velden et al. 2005; wind components are as- similated). Surface observations over land and NOAA wind profiler observations, which are processed by the GDAS but not assimilated by that system, are also as- similated here, as are global positioning system (GPS) radio occultation observations (Kursinski et al. 1997) from the Constellation Observing System for Meteo- rology, Ionosphere and Climate (COSMIC) program. Specific humidity was used as the variable for assimilated moisture observations, consistent with Schwartz and Liu (2014). The observation errors assigned for land surface humidity are relatively large, but are consistent with those assigned by the GDAS for humidity from radio- sonde and marine-surface observations. Because most stations observe relative humidity directly before con- verting to specific humidity or dewpoint, the assigned observation error is a function of relative humidity. Based on a suggestion from an anonymous reviewer, we tested the sensitivity of the results to using dewpoint rather than specific humidity as the assimilated mois- ture variable, and to using a smaller observation error for surface moisture over land. The results of these tests are presented in appendix B. The assimilated observations were also subject to additional processing in the same manner as described by Romine et al. (2013), including ''superobbing'' of AMDAR and AMV observations, exclusion of surface observations in complex terrain, and inflation of obser- vation errors for observations near the model lateral boundaries. The horizontal intervals for superobbing were 30 km for AMDAR observations and 60 km for AMVs. Observations more than four standard de- viations away from the prior ensemble mean value were rejected. A representative spatial distribution of assim- ilated observations is shown in Fig. 5 for 0000 UTC 10 June 2010. Neither radar data nor satellite radiances were assimilated in this study; the radar observations were not assimilated because of the focus on forecasts beyond 6 h, where the impact of radar assimilation is small (e.g., Stratman et al. 2013), and satellite radiances were not assimilated because these capabilities are still in their infancy in the DART system (e.g., Jones and Stensrud 2012). d. Convection-allowing forecast ensemble To evaluate their performance as ICs for convection- allowing ensemble forecasts, the analyses from each of the four WRF-DART configurations discussed in section 3c were used to initialize 24-member, 36-h ensemble forecasts. These forecasts were initialized at 0000 UTC 8, 9, and 10 June 2010, using 4-km horizontal grid spacing over a domain covering much of the south-central United States shown in Fig. 3. This domain was designed as a balance between the computational cost of running multiple convection-allowing ensembles and the desire to encompass the features of interest (i.e., both the southern and northern MCSs) in the interior of the domain. The four convection-allowing ensembles were ini- tialized on a single domain by downscaling the WRF- DART analyses to a 4-km grid using the WRF ''ndown'' program. They used identical model configurations to their parent assimilation system with the exception of the smaller grid spacing and domain, and the explicit (rather than parameterized) representation of cumulus convection for the 4-km forecasts. In other words, member 1 of the mixphys_cc and mixphys_24hr fore- cast ensembles used the same physical parameterizations (except for cumulus) as were used in member 1 of the WRF-DART system, and so on. However, only the first 24 members of the ensemble were used to initialize the 4-km forecasts. For the mixed-physics ensembles, ini- tializing the ensemble with the first 24 members (rather than 24 members meeting some other criteria) may mean that the chosen members are not a truly representative sample of the larger ensemble, but the potential effects of this on the forecasts have not been tested. LBCs for the 4-km forecasts were provided by the GFS forecast initialized 6 h prior to the WRF forecast (i.e., 1800 UTC the previous day). As with the LBCs used for the WRF-DART assimilation system, the LBCs for the 4-km forecasts were perturbed using the fixed covariance perturbation method. All of the ensemble configurations used the same LBC perturbations. A sensitivity test using unperturbed LBCs was also con- ducted and will be discussed briefly in section 5c.As a point of comparison, these ensemble forecasts will also be evaluated against the SSEF, which was run and evaluated in real time during June 2010. e. Evaluation of analyses and forecasts Several methods will be used to evaluate both the analyses generated by the ensemble-based data assimi- lation system and the output of the 4-km ensemble forecasts. The WRF-DART analyses will be evaluated using standard statistical methods, including the root- mean-square (RMS) error, bias, and ensemble spread of state variables against the observations over the entire domain, as well as by focusing on the analysis of the MCV in the southern United States. The evaluation of the 4-km ensemble forecasts will primarily focus on precipitation, but relationships between the precipi- tation forecasts and other atmospheric fields will also be presented where appropriate. Two primary methods for evaluating the probabilistic precipitation forecasts are the area under the receiver operating characteristic (ROC) curve and rank histograms (e.g., Mason and Graham 2002; Wilks 2011). For all precipitation forecast evaluation, the NCEP stage-IV precipitation analysis (ST4; Lin and Mitchell 2005) was used as the ''truth,'' and the WRF Model output was regridded to the stage- IV grid using a neighbor-budget method. For the area under the ROC curve, a simple ''neigh- borhood'' probability of exceeding a given precipitation threshold was first calculated from the ensemble fore- casts by using Eq. (5) of Schwartz et al. (2010) with a neighborhood encompassing the six grid points on each side of a given grid point, for a 13 3 13 gridpoint (52 3 52 km2) neighborhood. The WRF-DART forecasts used all 24 members in the calculation of the probabilities, whereas the SSEF had 15 core members that were used (Xue et al. 2011). The ROC curves, and the areas under those curves, were then calculated based on these neighborhood probabilities and unsmoothed stage-IV analyses using the method of Mason and Graham (2002) and subject to a bootstrap procedure with 500 resam- plings to calculate confidence intervals. To examine the spread and bias characteristics of the 4-km ensemble precipitation forecasts, rank histograms of the preci- pitation accumulations at different lead times are also presented. Individual ensemble members were also evaluated using the fractions skill score (FSS; Roberts and Lean 2008), also using a 52 3 52 km2 neighborhood. These neighborhood methods have been shown to bal- ance the objective evaluation of forecasts at very small scales with subjective evaluations of forecast quality (e.g., Ebert 2008; Schwartz et al. 2010). The scores were calculated over two regions: a large region covering nearly the full forecast domain, and a subdomain cov- ering Texas, Arkansas, Oklahoma, and Louisiana (Fig. 3) to isolate the heavy precipitation associated with the MCV. The scores were calculated using the Mete- orological Evaluation Tools (MET) software package, version 4.0. f. MCV tracking A vortex-tracking method was used to calculate the path of the MCV in the ensemble forecasts and to di- agnose the environmental conditions near the MCV. The model-predicted vortices were tracked using the same method as Schumacher et al. (2013),wherebythe vortex location is defined by center of a box with the greatest circulation in the 700-500-hPa layer. For the 4-km forecasts, a 183 km 3 183 km (61 3 61 model grid point) box was used, whereas for the model ICs at 36-km grid spacing, a 324 3 324 km2 (9 3 9 grid point) box was used. The vortex location in the North Ameri- can Regional Reanalysis (NARR) was used as ''obser- vations'' here because it is an independent analysis and because it is available at 3-h intervals. For the NARR, which is on a 32-km grid, a 288 km 3 288 km, or 9 3 9 grid point, box was used for calculating the circulation. The sizes of the boxes used for the NARR and 4-km forecasts are the same as in Schumacher et al. (2013), and the size of the box for the 36-km ICs was chosen to be similar to that used for the NARR. 4. Results: Analyses In this section, similarities and differences between the analyses produced by the four WRF-DART config- urations are presented, beginning with the characteristics of the analyses for state variables over the full domain, followed by more detailed aspects of the analyses of the MCV over the south-central United States. Because we are using both continuously cycled configurations and those only cycled for 24 h, the comparisons between analyses focus primarily on a single 24-h period (e.g., five assimilation cycles) from 0000 UTC 9 June to 0000 UTC 10 June, but the results for earlier time periods are generally similar, and a few results showing the evolu- tion of the assimilation diagnostics with time will also be shown in this section. Consistent with the results of Romine et al. (2013) and Schwartz and Liu (2014), there is a tendency for the ''first guess'' (i.e., 6-h model forecasts) in all of the WRF-DART systems to have a warm bias with respect to radiosondes through much of the troposphere, but a cool bias with respect to AMDAR temperature ob- servations, especially above 500 hPa (Figs. 6a,b). How- ever, Ballish and Krishna Kumar (2008) point out that the AMDAR temperature observations may themselves have a warm bias. With respect to radiosonde temper- ature observations, the configurations with one day of cycling have a greater warm bias than those that are continuously cycled. This is also true with respect to AMDAR temperatures above 700 hPa. The continu- ously cycled configurations have lower values of both RMSE and total spread (i.e., the sum of the observation error standard deviation and the ensemble spread) in their 6-h forecasts than those cycled for only 24 h, an issue that will be revisited later in this section. After assimilating observations, all of the WRF-DART analyses accordingly show better fit to those observa- tions. With respect to radiosonde temperature, the analyses still show a slight warm bias, especially below 800 hPa and above 400 hPa (Fig. 6c). The analyses all fit the AMDAR temperatures well in the midtroposphere, with a slight cool bias aloft, again similar to the results shown by Romine et al. (2013). At the radiosonde sites, the total spread is reduced after observations are as- similated, and the spread in temperature is slightly higher than the RMSE through most of the troposphere (Figs. 6c,d). The first-guess zonal winds exhibit a slow bias, partic- ularly with respect to radiosondes above approximately 700 hPa (Figs. 7a,b). This is also generally consistent with the results of Romine et al. (2013) and Schwartz and Liu (2014). The magnitude of this bias is greater in the sys- tems cycled for only 24 h. As with the radiosonde tem- peratures, there is less spread and lower RMSE in the continuously cycled systems. After assimilating the ob- servations, only a very minimal slow bias remains (Figs. 7c,d). There is considerably less bias in the first guesses for the meridional (y) wind component (not shown). Both the first guesses and the analyses of spe- cific humidity show a slight dry bias at low levels with a slight moist bias aloft (Figs. 8a,b). The mixed-physics configurations are consistently drier than the single- physics configurations. The total spread (dashed lines in Figs. 8a,b) is considerably larger than the RMSE for low-level specific humidity. This is an undesirable arti- fact of the assumed observation error being compara- tively large at high values of specific humidity, which are more common at low levels. The 24-h cycling period does appear sufficient to achieve a magnitude of prior spread that is similar to that in the continuously cycled configuration, at least for the midlevel wind and temperature fields shown in Fig. 9. Other vertical levels (not shown) demonstrate similar behavior. This suggests that it is possible to ''spin up'' the ensemble assimilation system over a fairly short period of time, but Fig. 9 also shows that it would not be desirable to use a spinup time that was any shorter than 24 h for this type of assimilation system, as the initial ''random'' spread is comparatively large and the prior spread in the ''24hr'' runs does not approach that of the ''cc'' runs until 24 h of cycling is complete. Diagnostics examining the assimilation of surface observations primarily echo the results presented by Romine et al. (2013), with the continuously cycled analyses being nearly unbiased for surface temperature, but exhibiting a moist bias that is greatest during the daytime hours (Fig. 10). As with the radiosonde mois- ture observations, the total spread in the surface specific humidity is larger than the RMSE because of the large observation error assigned at high values of specific humidity; this represents a difference from Romine et al. (2013) and Ha and Snyder (2014), where the total spread in surface moisture was closer in magnitude to the RMSE. Prominent differences between the mixed- physics and single-physics configurations are that the mixed-physics systems consistently had much larger ensemble spread in these fields, and they also had larger analysis increments in the surface specific humidity field at the 1800 UTC analysis time. However, the reasons that the increments were larger at this specific time are not clear. When visualized spatially, there is considerably more spread in the midtropospheric mass fields in the mixed- physics configurations than in the single-physics analyses. For example, the ensemble-mean 500-hPa height analysis at 0000 UTC 10 June 2010 is very similar across most of the domain in all four configurations, but the spread in the interior of the domain is greater in the mixed-physics configurations (Figs. 11b,d) than in the single-physics configurations (Figs. 11a,c). (The large spread near the domain edges is a result of the perturbations applied at the lateral boundaries, discussed in section 3b.) On the other hand, the analyses from the continuously cycled and 24-h cycling configurations were generally similar, consistent with Fig. 9. All of the assimilation configura- tions showed a closed 500-hPa height contour over northeast Texas in the ensemble mean, indicative of the MCV, but again the mixed-physics configurations had more spread than the single-physics systems around this feature. Considering the influence of convective and microphysical processes on the development and maintenance of MCVs (e.g., Davis and Trier 2002) and the well-known uncertainties in the representation of these processes by numerical models, a closer look at the analyses of this MCV by the different assimilation con- figurations is warranted. Unsurprisingly, the configurations that include multiple microphysics parameterizations in the assimilation system provide a greater diversity of vertical profiles of microphysical properties than the configurations using only a single microphysics parameterization. Particu- larly in the prior (6-h forecast), the shape of the vertical profiles of hydrometeor mixing ratios are similar across the configurations using only the Morrison microphysics scheme (Figs. 12a,c), with the differences between members primarily appearing in the values of the mixing ratios of the different species. On the other hand, the mixed-physics configurations (Figs. 12b,d) show diversity in both the shapes of the profiles and the magnitudes of the values. In particular, the height of the freezing level (which can be visualized as the transition region be- tween the liquid and ice hydrometeor species) ranges from approximately 600 to 400 hPa across the ensemble members in the mixed-physics configurations, whereas there is a much narrower range around 600 hPa in the single-physics forecasts and analyses. The variability in the shapes and structures of the hydrometeor profiles continues to be greater in the mixed-physics configurations than the single-physics configurations after assimilation, although the overall spread is reduced in all four as ex- pected (Figs. 12e-h). Because latent heating and cooling processes are key to the vertical distribution of potential vorticity (PV) in MCVs (e.g., Haynes and McIntyre 1987; Davis and Trier 2002), the representation of these processes in the as- similation system are also important in how the MCV is analyzed. Consistent with the greater spread in the mi- crophysical profiles (especially in the first-guess fields), there is also greater spread in the area-averaged vertical structure of PV (Fig. 13), and also in the spatial extent of the higher spread in the midlevel PV (Fig. 14) in the mixed-physics configurations, although the mean pro- files and structure of PV are similar among all of the configurations. In particular, the mixed-physics runs demonstrate more spread in upper-level PV than the single-physics ensembles, but there are relatively small differences between the ''cc'' and ''24hr'' configurations. This likely reflects the importance of diversity in the microphysical parameterizations in producing diversity in the PV field. Although it is not clear from this brief diagnosis which analysis of the microphysical and PV profiles is closest to the truth, the forecasts initialized from these analyses will provide some evidence as to the strengths and limitations of the different assimilation methods. 5. Results: Ensemble forecasts a. Precipitation We now turn attention to the convection-allowing ensemble forecasts that were initialized from each of these WRF-DART assimilation configurations. This section will begin with an overview of the results ob- tained from evaluating the different ensemble config- urations, and then delve into some detailed aspects of the forecasts. As discussed in section 3e, a neighborhood method was used to derive probabilistic forecasts for which the area under the ROC curve was calculated over two domains: one covering most of the forecast domain, and a subdomain of that larger domain focusing on the southern Great Plains (Fig. 3). The ROC area was calculated at 6-h intervals over each of these domains at the thresholds of 12.5, 25, 50, and 100 mm (6 h)21, and these results are presented in aggregate in Fig. 15 and separated by forecast lead time in Fig. 16). On the whole, all of the ensemble configurations were found to produce skillful forecasts in the 6-36-h time- frame, with aggregate ROC areas above 0.8 for all of the ensembles (Fig. 15), and ROC areas well above 0.7 for most of the ensembles at most times (Fig. 16). The ag- gregate results (Fig. 15) show that the mixed-physics forecast ensembles had larger ROC areas than the single-physics ensembles at all thresholds over the full domain and at all but the largest threshold for the sub- domain. The ROC area for the SSEF generally fell in between the mixed-physics and single-physics WRF- DART ensembles. There was no consistent signal sug- gesting that the ''cc'' configurations had larger ROC areas than the ''24hr'' configurations or vice versa. At the lower precipitation thresholds, the ROC areas for all ensembles were smaller over the subdomain than those for the full domain, whereas areas were larger over the subdomain than the full domain at the higher thresholds for all but the mixphys_24hr and SSEF ensembles. This suggests that the regions of very heavy rainfall near the MCV (which was contained within the verification subdomain) were better predicted than those elsewhere, even though the opposite was true for forecasts of lighter precipitation. When breaking the results down by lead times, no ensemble configuration was found to be consistently superior or inferior to the others at all lead times and thresholds (Fig. 16). However, consistent with the ag- gregate results, the ensemble forecasts using mixed physics were typically at or near the top for most lead times in both verification domains, and the two single- physics configurations were consistently at or near the bottom. The SSEF system also performed well, al- though it does carry the limitation of only being run out to 30 h (instead of 36 for the WRF-DART forecasts) and having an ensemble membership of only 15 for these calculations (cf. 24 members in the WRF-DART ensembles.) However, Clark et al. (2011) showed that increases in precipitation forecast skill were minimal when increasing the SSEF ensemble size beyond ap- proximately 10 members. For both the full verification domain and the sub- domain at the 6-12-h lead time, the WRF-DART en- sembles (with the exception of single_24hr on the full domain) outperformed the SSEF (Fig. 16), with these differences being statistically significant on the sub- domain (Figs. 16b,d). For 12-18-h forecasts, all of the WRF-DART ensembles outperformed the SSEF when considering the full evaluation domain, but the SSEF showed comparable or better performance over the Arkansas-Texas verification subdomain. In contrast to the short lead times, the SSEF was the best, or nearly the best, forecast system in terms of ROC area for 24-30-h precipitation forecasts. At the longest lead times considered here (30-36 h), the single-physics configura- tions show an increase in ROC area relative to the mixed-physics ensembles. Possible reasons for this will be examined in the following subsections. Because the ROC area is insensitive to biases in probabilistic forecasts, an examination of the spread and bias in the different ensemble configurations is also needed. At short lead times, the single-physics con- figurations have rank histograms that are somewhat ''U shaped,'' which indicates that the observed precip- itation values often fall outside the ensemble envelope (Figs. 17a,f). In other words, these ensembles have in- sufficient spread at short lead times. The mixed-physics ensembles, on the other hand, have much flatter rank histograms (Figs. 17k,p). The better dispersion charac- teristics of the mixed-physics ensembles are likely tied to the greater spread in the mixed-physics analyses of the MCV discussed previously and shown in Figs. 12-14. At all lead times, the SSEF has rank histograms that are right skewed, indicating that this system has a consistent wet bias (Figs. 17u-x). This result is in line with the results of Clark et al. (2009) and Schumacher et al. (2013,theirFig. 6) for full seasons of SSEF forecasts. In comparison, the WRF-DART configurations have relatively flat rank histograms at longer lead times, again suggesting that they have more representative spread than the SSEF (Fig. 17). The reasons for the ''flattening'' of the rank histograms with increasing lead time for the single-physics forecasts are not immedi- ately clear. Over the three-day period of heavy rainfall in Texas and Arkansas, the predictive skill of each of the en- semble systems generally increased with each run, sim- ilar to the findings of Davis et al. (2002), owing to the heavy precipitation becoming more widespread, and the MCV becoming better defined and more robust with time. When considering the full domain, however, the skill was more consistent from run to run. One il- lustration of this is in the 24-30-h forecasts shown in Fig. 18, which show low probabilities of 25 mm of rain in south-central Texas (MCS S1) in each of the ensemble systems for the forecast initialized on 8 June (Figs. 18a-f), generally higher probabilities in the region where the heavy rain fell for the forecast initialized on 9 June (MCS S2; Figs. 18g-l), and a focused region of low to moderate probabilities near the Texas-Arkansas border in the run initialized on 10 June (MCS S3; Figs. 18m-r). Even for the 9 and 10 June initializations, the exceedence proba- bilities were less than 30% in the region of the MCV in all ensembles. On the other hand, the locations with high potential for heavy rainfall in the central and northern plains were captured in the runs (with probabilities greater than 30%) from both 8 June and 10 June (MCSs N1 and N3), presumably because the baroclinic zone in that area was already well established and was analyzed well by the assimilation systems. Although not shown here, the heavy precipitation probabilities tend to be focused over smaller areas with higher probabilities at earlier lead times, and then disperse over larger spatial areas as the solutions of the individual ensemble mem- bers diverge. b. MCV track and structure Schumacher et al. (2013) showed the close relation- ship between the MCV and the heavy rainfall over the course of this event, and therefore a discussion of skill and uncertainty for the precipitation forecast necessarily raises the question of the skill and uncertainty for the forecast of the MCV. Visualizing and evaluating ensemble forecasts of tropical cyclones has become commonplace, but similar investigations for continental vortices such as MCVs are limited in the literature [Hawblitzel et al. (2007) and Clark et al. (2010) being two notable ex- ceptions]. This study provides an attempt to evaluate ensemble MCV forecasts in a similar way. Identifying the MCV during 8 June 2010 was diffi- cult, as that was the day it became more organized and moved inland, and efforts to adequately track the vortex in the model forecasts initialized on 8 June were unsuccessful. In the forecast initialized at 0000 UTC 9June,therewasatendencyforthemodel-predicted MCVs to move slightly north and west of the NARR- analyzed track; in other words, a left-of-track error (Fig. 19a). (The NARR is used as observations here because it is an independent analysis and because it is available at 3-h intervals.) There were no clear biases or differences between the different model configurations, aside from the fact that the single_cc and mixphys_cc analyses included several members with the initial po- sition of the MCV much farther south than the other analyses or the NARR (Fig. 19a). This left-of-track er- ror in the WRF-DART-initialized ensembles is consis- tent with the precipitation forecasts shown in Figs. 18g-j, in which the highest precipitation probabilities were also displaced slightly to the north and west of where the heavy rainfall was observed in north Texas. Un- fortunately the full three-dimensional output of the SSEF was unavailable for this case to analyze the MCV track in that ensemble. For the ensemble forecasts initialized at 0000 UTC 10 June, the ensemble envelope nearly symmetrically surrounded the NARR-analyzed track. The generally accurate track forecasts for the MCV on 10-11 June also contributed to the placement of higher precipita- tion probabilities near the Texas-Arkansas border in all of the ensembles. Almost all of the SSEF members predicted the MCV to move too slowly (Fig. 19b), which is consistent with the highest precipitation probabil- ities in the SSEF being located farther southwest than the WRF-DART ensembles (Figs. 18m-r). The con- nection between the MCV location and the precipitation forecast is further illustrated in Figs. 20a,b, which shows that the ensemble members with the best precipita- tion forecasts on 11 June (MCS S3 in Fig. 1d)were those that predicted the MCV to be located in south- western Arkansas and northeast Texas at 0300 UTC (Fig. 20a) and in central Arkansas at 0900 UTC (Fig. 20b). However, at 0300 UTC, the predicted MCV locations in these members were slightly south of the NARR-analyzed location. Furthermore, although the members with poor forecasts of the MCV track gen- erally also had poor precipitation forecasts, there were many members that had poor precipitation forecasts yet an MCV track prediction very similar to other members that had good precipitation forecasts. This suggests that factors beyond simply the track of the MCV were important to the accurate prediction of heavy precipitation. For the SSEF, Schumacher et al. (2013) address some of these possible factors, and the factors that were important in these WRF-DART en- semble forecasts will be addressed in future research. Finally, although relatively few members of the single_cc ensemble had skillful forecasts of the MCV-related precipitation in the 24-30-h forecast (associated with thelowROCareaforthisensembleinFigs. 16b,d), multiple members of this ensemble then intensified the MCV and predicted heavy rainfall near the observed location in the 30-36-h forecast (Fig. 20b), improving the forecast skill of that ensemble at this lead time (Figs. 16b,d). Although space considerations do not make it fea- sible to show the evolution of the MCV and the pre- cipitation forecasts across the full ensembles, Fig. 21 illustrates some of the differences between the fore- casts using a selected single member of all four en- semble configurations (which had identical physics schemes and LBCs) for the heavy rain event on 11 June 2010. In this member of each ensemble, a closed circulation and vorticity maximum was located over Arkansas at 0600 UTC 11 June, though the intensity and specific location of the vorticity maximum varied among these forecasts. In mixphys_24hr, there is a concentrated region of vorticity in southern Arkansas (Fig. 21d),andanassociatedregionoflong-livedheavy precipitation, which was located just south of the observed MCS in the 24-30 and 30-36-h forecasts (Figs. 21h,l). In this member in the other three en- sembles, the forecast MCV was weaker than that in mixphys_24hr (Figs. 21a-c) and there was too little precipitation in the 24-30-h forecast (Figs. 21e-g). Al- though there was an increase in precipitation in these members as the MCV moved northeast in hours 30-36 (Figs. 21i-k), there continued to be an underprediction of rainfall surrounding the MCV. c. Sensitivity to lateral boundary condition perturbations Some past studies of convection-allowing ensemble forecasts have found that the errors and uncertainties owing to the LBCs can dominate over those of the ICs in relatively short periods of time [on the order of 12 h; e.g., Hohenegger et al. 2008; Vié et al. 2011]. With this in mind, 12 members of the mixphys_24hr ensemble fore- cast were run for 36 h using unperturbed GFS forecasts for the LBCs, and were then compared to the forecasts discussed previously that used the perturbed LBCs. It was found that, although there were nontrivial differences later in the runs using the perturbed versus unperturbed LBCs, these differences were much smaller than the differences between members owing to the ICs (not shown). In particular, the members that produced accu- rate forecasts of heavy rainfall at 24-36-h lead times with perturbed LBCs were the same members that did so with the unperturbed LBCs, and similarly those with poor predictions were the same in both experiments. This sug- gests that although LBC perturbations had an influence on the forecast, they did not overwhelm the IC uncertainties even through 36 h. The discrepancy between this research and the above-cited studies is at least in part attributable to the sizes of the convection-allowing domains in those studies; both used domains covering only a small section of Europe (less than 108 latitude-longitude on a side), whereas this study used a much larger domain that cov- ered approximately the central two-thirds of the conti- nental United States (more than 208 latitude-longitude). In other words, the influence of LBC perturbations is likely very sensitive to the size of the domain chosen, as well as the length of the model integration. 6. Discussion This study provides an intercomparison of multiple methods of configuring and running a convection- allowing ensemble forecast system. As such systems continue to come into greater use owing to increased computing capabilities and the need for probabilistic information about convective weather hazards, it will be important to evaluate them for both their forecast skill and their representation of uncertainty. This will allow for the systems to be designed such that they optimally use the substantial computing needs required to run them. This study found that overall, ensemble configurations with diversity in both their ICs and their physical pa- rameterizations provided better probabilistic forecasts of heavy rainfall than those with changes to the ICs but only a single set of parameterization for all members. This result is similar to the findings of Stensrud et al. (2000) and Fujita et al. (2007), among others. These re- sults provide further evidence in support of the idea that uncertainty in the representation of physical processes by models must be included in an ensemble prediction sys- tem that explicitly predicts convection. In particular, cir- culation features such as MCVs that are intimately tied to convective processes may not be analyzed properly by ensemble-based assimilation systems, nor predicted ade- quately by high-resolution ensembles, if this model un- certainty is not represented in the assimilation and forecast system. However, diversity in physical parame- terizations can also complicate the interpretation of the forecasts, and can complicate quantitative methods such as ensemble-based synoptic and mesoscale analysis (e.g., Hakim and Torn 2008), as those methods assume that all ensemble members are equally likely, which may not al- ways be the case for a mixed-physics ensemble. There are also other methods for including model uncertainty without using a diversity of parameterizations, including stochastic perturbations within those parameterizations (e.g., Bouttier et al. 2012), using ensemble data assimi- lation for parameter estimation (e.g., Nielsen-Gammon et al. 2010), or stochastic kinetic energy perturbations (e.g., Berner et al. 2011). Although these methods have not been thoroughly tested for data assimilation and forecasting on convection-allowing spatial and temporal scales, they may provide means for incorporating model uncertainty in the future. The results presented here also provide some evi- dence supporting the design and implementation of the SSEF, which has been run in real-time during several recent spring seasons. Although its design is somewhat ad hoc, it performed comparably to, or better than, the ensemble systems using ensemble data assimilation to initialize their forecasts, especially at longer lead times. For practical purposes this is an important finding, considering that the SSEF is, in a relative sense, com- putationally inexpensive because it exploits analyses and ensemble perturbations from other readily available sources rather than requiring an ensemble data assimi- lation system of its own. Of course, this study only presents the results of three cases with somewhat similar large-scale meteorological conditions, so the results should not be generalized too broadly. The SSEF also exhibits a consistent wet bias, suggesting that there may be changes to the configuration of some members, or statistical postprocessing methods, which could reduce this bias and improve forecasts further. Finally, the results of this study suggest that a relatively short cycling period (e.g., 24 h) may lead to comparable forecasts than a con- tinuously cycled system, but again the applicability of this result is limited by the small sample size in this study. The results are consistent with Wheatley et al. (2012),who found that running an ensemble data assimilation system for even a very short time could potentially provide forecast improvements. More research with larger sample sizes is required to better address these issues. 7. Summary and conclusions In this study, four different ensemble-based data assimilation and forecast systems were run and their performance was evaluated during a period of six heavy- rain-producing MCSs over three days in June 2010. The four systems varied in the length of time over which they were cycled, and in whether they had a single set of physical parameterizations or a diversity of parameteri- zations. All four assimilation configurations produced results that were generally similar to those reported in previous studies using the WRF-DART system, although there were important differences in the vicinity of the convective systems. In particular, the shapes of the vertical profiles of microphysical variables in the vicinity of an MCV in the southern Plains were more uniform in the single-physics systems but had greater diversity in the mixed-physics systems, which then influenced the structure of the vortex in the analyses. Convection-allowing ensemble forecasts were then initialized from these analyses, and were also compared to a real-time ensemble from CAPS, known as the SSEF. Using the ROC area as an evaluation method, the systems with mixed physics provided more skillful probabilistic forecasts of the heavy precipitation, though this result is somewhat dependent on the precipitation threshold and lead time being considered. The contin- uously cycled and partially cycled systems performed similarly to one another for precipitation forecasts. The WRF-DART-initialized forecasts generally out- performed the SSEF at shorter lead times, whereas the SSEF provided the most skillful forecasts, or nearly so, at longer lead times, although it also had a consistent wet bias. The skill of the precipitation forecasts in Arkansas and Texas was partially tied to the accuracy of the forecast track of the MCV, but other factors also influenced the skill of these forecasts. Ongoing work is aimed at further analyzing the processes responsible for the differences between the ensemble systems, and also the differences among the members within each of the ensembles, with an aim of both improved proba- bilistic prediction of, and increased understanding of the processes responsible for, extreme warm-season precipitation. Acknowledgments. The authors thank Glen Romine, David Novak, and Wallace Hogsett for helpful discus- sions and suggestions regarding this work. The authors also thank Ming Xue and Fanyou Kong of CAPS for making the SSEF output available, and for their efforts in developing and running the SSEF. Three anonymous reviewers provided numerous constructive suggestions that substantially improved the quality of this manu- script. Computing resources were provided by the Na- tional Center for Atmospheric Research, which is sponsored by the National Science Foundation. NARR analyses were obtained from the NOAA/Physical Sci- ences Division, and stage-IV analyses were provided by NCAR. R. Schumacher was supported by National Science Foundation Grant AGS-1157425. *Supplemental information related to this paper is avail- able at the Journals Online website: http://dx.doi.org/10.1175/ MWR-D-13-00357.s1. REFERENCES Anderson, J. L., 2001: An ensemble adjustment Kalman filter for data assimilation. Mon. Wea. Rev., 129, 2884-2903, doi:10.1175/ 1520-0493(2001)129,2884:AEAKFF.2.0.CO;2. -, 2009: Spatially and temporally varying adaptive covariance inflation for ensemble filters. Tellus, 61A, 72-83, doi:10.1111/ j.1600-0870.2008.00361.x. -, T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 1283-1296, doi:10.1175/2009BAMS2618.1. Augustine, J. A., and F. Caracena, 1994: Lower-tropospheric precursors to nocturnal MCS development over the central United States. Wea. Forecasting, 9, 116-135, doi:10.1175/ 1520-0434(1994)009,0116:LTPTNM.2.0.CO;2. Ballish, B. A., and V. Krishna Kumar, 2008: Systematic differences in aircraft and radiosonde temperatures. Bull. Amer. Meteor. Soc., 89, 1689-1708, doi:10.1175/2008BAMS2332.1. Barker, D., and Coauthors, 2012: The Weather Research and Forecasting model's community variational/ensemble data assimilation system: WRFDA. Bull. Amer. Meteor. Soc., 93, 831-843, doi:10.1175/BAMS-D-11-00167.1. Berner, J., S.-Y. Ha, J. P. Hacker, A. Fournier, and C. Snyder, 2011: Model uncertainty in a mesoscale ensemble prediction system: Stochastic versus multiphysics representations. Mon. Wea. Rev., 139, 1972-1995, doi:10.1175/2010MWR3595.1. Bonner, W. D., 1968: Climatology of the low level jet. Mon. Wea. Rev., 96, 833-850, doi:10.1175/1520-0493(1968)096,0833: COTLLJ.2.0.CO;2. Bouttier, F., B. Vié, O. Nuissier, and L. Raynaud, 2012: Impact of stochastic physics in a convection-permitting ensemble. Mon. Wea. Rev., 140, 3706-3721, doi:10.1175/MWR-D-12-00031.1. Chen, F., and J. Dudhia, 2001: Coupling an advanced land surface- hydrology model with the Penn State-NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea.Rev.,129,569-585,doi:10.1175/1520-0493(2001)129,0569: CAALSH.2.0.CO;2. Clark, A. J., W. A. Gallus Jr., M. Xue, and F. Kong, 2009: A comparison of precipitation forecast skill between small convection-allowing and large convection-parameterizing ensembles. Wea. Forecasting, 24, 1121-1140, doi:10.1175/ 2009WAF2222222.1. -, -, -, and -, 2010: Convection-allowing and convection-parameterizing ensemble forecasts of a meso- scale convective vortex and associated severe weather en- vironment. Wea. Forecasting, 25, 1052-1081, doi:10.1175/ 2010WAF2222390.1. -, and Coauthors, 2011: Probabilistic precipitation forecast skill as a function of ensemble size and spatial scale in a convection-allowing ensemble. Mon. Wea. Rev., 139, 1410- 1418, doi:10.1175/2010MWR3624.1. -, and Coauthors, 2012: An overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program spring ex- periment. Bull. Amer. Meteor. Soc., 93, 55-74, doi:10.1175/ BAMS-D-11-00040.1. -, J. Gao, P. T. Marsh, T. Smith, J. S. Kain, J. Correia Jr., M. Xue, and F. Kong, 2013: Tornado pathlength forecasts from 2010 to 2011 using ensemble updraft helicity. Wea. Forecasting, 28, 387-407, doi:10.1175/WAF-D-12-00038.1. Davis, C. A., and S. B. Trier, 2002: Cloud-resolving simulations of mesoscale vortex intensification and its effect on a serial me- soscale convective system. Mon. Wea. Rev., 130, 2839-2858, doi:10.1175/1520-0493(2002)130,2839:CRSOMV.2.0.CO;2. -, D. A. Ahijevych, and S. B. Trier, 2002: Detection and prediction of warm season midtropospheric vortices by the Rapid Update Cycle. Mon. Wea. Rev., 130, 24-42, doi:10.1175/ 1520-0493(2002)130,0024:DAPOWS.2.0.CO;2. Du, J., S. L. Mullen, and F. Sanders, 1997: Short-range ensemble forecasting of quantitative precipitation. Mon. Wea. Rev., 125, 2427-2459, doi:10.1175/1520-0493(1997)125,2427: SREFOQ.2.0.CO;2. Ebert, E. E., 2008: Fuzzy verification of high-resolution gridded forecasts: A review and proposed framework. Meteor. Appl., 15, 51-64, doi:10.1002/met.25. Eckel, F. A., and C. F. Mass, 2005: Aspects of effective mesoscale, short-range ensemble forecasting. Wea. Forecasting, 20, 328- 350, doi:10.1175/WAF843.1. Fujita, T., D. J. Stensrud, and D. C. Dowell, 2007: Surface data assimilation using an ensemble Kalman filter approach with initial condition and model physics uncertainties. Mon. Wea. Rev., 135, 1846-1868, doi:10.1175/MWR3391.1. Gaspari, G., and S. E. Cohn, 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc., 125, 723-757, doi:10.1002/qj.49712555417. Gilmour, I., L. A. Smith, and R. Buizza, 2001: Linear regime duration: Is 24 hours a long time in synoptic weather forecasting? J. Atmos. Sci., 58, 3525-3539, doi:10.1175/1520-0469(2001)058,3525: LRDIHA.2.0.CO;2. Ha, S.-Y., and C. Snyder, 2014: Influence of surface observa- tions in mesoscale data assimilation using an ensemble Kalman filter. Mon. Wea. Rev., 142, 1489-1508, doi:10.1175/ MWR-D-13-00108.1. Hakim, G. J., and R. D. Torn, 2008: Ensemble synoptic analysis. Synoptic-Dynamic Meteorology and Weather Analysis and Forecasting: A Tribute to Fred Sanders, Meteor. Monogr., No. 55, Amer. Meteor. Soc., 147-161. Hawblitzel, D. P., F. Zhang, Z. Meng, and C. A. Davis, 2007: Probabilistic evaluation of the dynamics and predictability of the mesoscale convective vortex of 10-13 June 2003. Mon. Wea. Rev., 135, 1544-1563, doi:10.1175/MWR3346.1. Haynes, P. H., and M. E. McIntyre, 1987: On the evolution of vorticity and potential vorticity in the presence of diabatic heating and frictional or other forces. J. Atmos. Sci., 44, 828-841, doi:10.1175/1520-0469(1987)044,0828: OTEOVA.2.0.CO;2. Higgins,R.W.,V.E.Kousky,andP.Xie,2011:Extremepre- cipitation events in the south-central United States during May and June 2010: Historical perspective, role of ENSO, and trends. J. Hydrometeor., 12, 1056-1070, doi:10.1175/ JHM-D-10-05039.1. Hohenegger, C., A. Walser, W. Langhans, and C. Schär, 2008: Cloud-resolving ensemble simulations of the August 2005 Alpine flood. Quart. J. Roy. Meteor. Soc., 134, 889-904, doi:10.1002/qj.252. Hong, S.-Y., and J.-O. J. Lim, 2006: The WRF single-moment 6-class microphysics scheme (WSM6). J. Korean Meteor. Soc., 42, 129-151. Janji^c, Z. I., 1994: The step-mountain eta coordinate model: Fur- ther developments of the convection, viscous sublayer, and turbulence closure schemes. Mon. Wea. Rev., 122, 927-945, doi:10.1175/1520-0493(1994)122,0927:TSMECM.2.0.CO;2. -, 2000: Comments on ''Development and evaluation of a convection scheme for use in climate models.'' J. Atmos. Sci., 57, 3686-3686, doi:10.1175/1520-0469(2000)057,3686: CODAEO.2.0.CO;2. -, 2002: Nonsingular implementation of the Mellor-Yamada level 2.5 scheme in the NCEP Meso model. NCEP Office Note 437, 61 pp. Jones, T. A., and D. J. Stensrud, 2012: Assimilating AIRS tem- perature and mixing ratio profiles using an ensemble Kalman filter approach for convective-scale forecasts. Wea. Fore- casting, 27, 541-564, doi:10.1175/WAF-D-11-00090.1. Kain, J. S., 2004: The Kain-Fritsch convective parameteriza- tion: An update. J. Appl. Meteor., 43, 170-181, doi:10.1175/ 1520-0450(2004)043,0170:TKCPAU.2.0.CO;2. Keyser, D., cited 2014: PREPBUFR processing at NCEP. NOAA/NWS/NCEP/EMC. [Available online at http:// www.emc.ncep.noaa.gov/mmb/data_processing/prepbufr.doc/ document.htm.] Kuo, Y.-H., T.-K. Wee, S. Sokolovskiy, C. Rocken, W. Schreiner, D. Hunt, and R. A. Anthes, 2004: Inversion and error esti- mation of GPS radio occultation data. J. Meteor. Soc. Japan, 82, 507-531, doi:10.2151/jmsj.2004.507. Kursinski,E.R.,G.A.Hajj,J.T.Schofield,R.P.Linfield,and K. R. Hardy, 1997: Observing Earth's atmosphere with radio occultation measurements using the Global Positioning System. J. Geophys. Res., 102, 23429-23465, doi:10.1029/ 97JD01569. Leoncini, G., R. S. Plant, S. L. Gray, and P. A. Clark, 2013: En- semble forecasts of a flood-producing storm: Comparison of the influence of model-state perturbations and parameter modifications. Quart. J. Roy. Meteor. Soc., 139, 198-211, doi:10.1002/qj.1951. Lin, Y., and K. E. Mitchell, 2005: The NCEP stage II/IV hourly precipitation analyses: Development and applications. 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2. [Available online at http://ams.confex.com/ams/pdfpapers/ 83847.pdf.] Mason, S. J., and N. E. Graham, 2002: Areas beneath the relative operating characteristics (ROC) and relative operating levels (ROL) curves: Statistical significance and interpretation. Quart. J. Roy. Meteor. Soc., 128, 2145-2166, doi:10.1256/ 003590002320603584. Mellor, G. L., and T. Yamada, 1982: Development of a turbulence closure model for geophysical fluid problems. Rev. Geophys., 20, 851-875, doi:10.1029/RG020i004p00851. Meng, Z., and F. Zhang, 2011: Limited-area ensemble-based data assimilation. Mon. Wea. Rev., 139, 2025-2045, doi:10.1175/ 2011MWR3418.1. Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87, 343-360, doi:10.1175/ BAMS-87-3-343. Morrison, H., J. A. Curry, and V. I. Khvorostyanov, 2005: A new double-moment microphysics parameterization for applica- tion in cloud and climate models. Part I: Description. J. Atmos. Sci., 62, 1665-1677, doi:10.1175/JAS3446.1. National Research Council, 2010: When Weather Matters: Sci- ence and Services to Meet Critical Societal Needs. Board on Atmospheric Sciences and Climate, National Academies Press, 181 pp. Nielsen-Gammon, J. W.,X.-M.Hu, F. Zhang, and J. E. Pleim, 2010: Evaluation of planetary boundary layer scheme sensitivities for the purpose of parameter estimation. Mon. Wea. Rev., 138, 3400-3417, doi:10.1175/2010MWR3292.1. NOAA, cited 2013: Storm events database. [Available online at http://www.ncdc.noaa.gov/stormevents/.] Noh, Y., W. G. Cheon, S. Y. Hong, and S. Raasch, 2003: Im- provement of the K-profile model for the planetary boundary layer based on large eddy simulation data. Bound.-Layer Meteor., 107, 401-427, doi:10.1023/A:1022146015946. Roberts, N. M., and H. W. Lean, 2008: Scale-selective verification of rainfall accumulations from high-resolution forecasts of convective events. Mon. Wea. Rev., 136, 78-97, doi:10.1175/ 2007MWR2123.1. Romine, G. S., C. S. Schwartz, C. Snyder, J. L. Anderson, and M. L. Weisman, 2013: Model bias in a continuously cycled assimi- lation system and its influence on convection-permitting forecasts. Mon. Wea. Rev., 141, 1263-1284, doi:10.1175/ MWR-D-12-00112.1. Schumacher, R. S., A. J. Clark, M. Xue, and F. Kong, 2013: Factors influencing the development and maintenance of nocturnal heavy-rain-producing convective systems in a storm-scale ensemble. Mon. Wea. Rev., 141, 2778-2801, doi:10.1175/ MWR-D-12-00239.1. Schwartz, C. S., and Z. Liu, 2014: Convection-permitting forecasts initialized with continuously cycling limited-area 3DVAR, ensemble Kalman filter, and hybrid variationalensemble data assimilation systems. Mon. Wea. Rev., 142, 716-738, doi:10.1175/MWR-D-13-00100.1. -, and Coauthors, 2010: Toward improved convection- allowing ensembles: Model physics sensitivities and optimizing probabilistic guidance with small ensemble membership. Wea. Forecasting, 25, 263-280, doi:10.1175/ 2009WAF2222267.1. Skamarock, W. C., and Coauthors, 2008: A description of the Advanced Research WRF version 3. NCAR Tech. Note NCAR/TN-4751STR, 113 pp. [Available online at http:// www.mmm.ucar.edu/wrf/users/docs/arw_v3_bw.pdf]. Smirnova, T. G., J. M. Brown, S. G. Benjamin, and D. Kim, 2000: Parameterization of cold-season processes in the MAPS land- surface scheme. J. Geophys. Res., 105, 4077-4086, doi:10.1029/ 1999JD901047. Stensrud, D. J., J.-W. Bao, and T. T. Warner, 2000: Using initial condition and model physics perturbations in short-range en- semble simulations of mesoscale convective systems. Mon. Wea. Rev., 128, 2077-2107, doi:10.1175/1520-0493(2000)128,2077: UICAMP.2.0.CO;2. Stratman, D. R., M. C. Coniglio, S. E. Koch, and M. Xue, 2013: Use of multiple verification methods to evaluate forecasts of convection from hot- and cold-start convection- allowing models. Wea. Forecasting, 28, 119-138, doi:10.1175/ WAF-D-12-00022.1. Tao, W.-K., J. Simpson, and M. McCumber, 1989: An ice-water saturation adjustment. Mon. Wea. Rev., 117, 231-235, doi:10.1175/1520-0493(1989)117,0231:AIWSA.2.0.CO;2. Thompson,G.,P.R.Field,R.M.Rasmussen,andW.D. Hall,2008: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part II: Implementation of a new snow parameterization. Mon. Wea. Rev., 136, 5095-5115, doi:10.1175/2008MWR2387.1. Tiedtke, M., 1989: A comprehensive mass flux scheme for cu- mulus parameterization in large-scale models. Mon. Wea. Rev., 117, 1779-1800, doi:10.1175/1520-0493(1989)117,1779: ACMFSF.2.0.CO;2. Torn, R. D., 2010: Diagnosis of the downstream ridging associ- ated with extratropical transition using short-term ensemble forecasts. J. Atmos. Sci., 67, 817-833, doi:10.1175/ 2009JAS3093.1. -, and G. J. Hakim, 2008: Performance characteristics of a pseudo-operational ensemble Kalman filter. Mon. Wea. Rev., 136, 3947-3963, doi:10.1175/2008MWR2443.1. -, -, and C. Snyder, 2006: Boundary conditions for limited- area ensemble Kalman filters. Mon. Wea. Rev., 134, 2490- 2502, doi:10.1175/MWR3187.1. Tuttle, J. D., and C. A. Davis, 2006: Corridors of warm season precipitation in the central United States. Mon. Wea. Rev., 134, 2297-2317, doi:10.1175/MWR3188.1. Velden, C., and Coauthors, 2005: Recent innovations in deriving tropospheric winds from meteorological satellites. Bull. Amer. Meteor. Soc., 86, 205-223, doi:10.1175/BAMS-86-2-205. Vié, B., O. Nussier, and V. Ducrocq, 2011: Cloud-resolving ensemble simulations of Mediterranean heavy precipitating events: Un- certainty on initial conditions and lateral boundary conditions. Mon. Wea. Rev., 139, 403-423, doi:10.1175/2010MWR3487.1. Wheatley, D. M., and D. J. Stensrud, 2010: The impact of assimi- lating surface pressure observations on severe weather events in a WRF mesoscale ensemble system. Mon. Wea. Rev., 138, 1673-1694, doi:10.1175/2009MWR3042.1. -, -, D. C. Dowell, and N. Yussouf, 2012: Application of a WRF mesoscale data assimilation system to springtime se- vere weather events 2007-09. Mon. Wea. Rev., 140, 1539-1557, doi:10.1175/MWR-D-11-00106.1. Wilks, D. S., 2011: Statistical Methods in the Atmospheric Sciences. 3rd ed. Academic Press, 704 pp. Xue, M., and Coauthors, 2007: CAPS realtime storm-scale en- semble and high-resolution forecasts as part of the NOAA Hazardous Weather Testbed 2007 spring experiment. 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Park City, UT, Amer. Meteor. Soc., 3B.1. [Available online at http://ams.confex.com/ams/ pdfpapers/124587.pdf.] -, and Coauthors, 2011: Realtime convection-permitting ensemble and convection-resolving deterministic forecasts of CAPS for the Hazardous Weather Testbed 2010 spring experiment. 24th Conf. on Weather Analysis and Forecasting/20th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., 9A.2. [Available online at https://ams.confex.com/ams/91Annual/ webprogram/Manuscript/Paper183227/Xue_CAPS_2011_ SpringExperiment_24thWAF20thNWP_ExtendedAbstract.pdf.] Zhang, C., Y. Wang, and K. Hamilton, 2011: Improved represen- tation of boundary layer clouds over the southeast Pacific in ARW-WRF using a modified Tiedtke cumulus parameteri- zation scheme. Mon. Wea. Rev., 139, 3489-3513, doi:10.1175/ MWR-D-10-05091.1. Zhang, F., C. Snyder, and R. Rotunno, 2003: Effects of moist con- vection on mesoscale predictability. J. Atmos. Sci., 60, 1173-1185, doi:10.1175/1520-0469(2003)060,1173:EOMCOM.2.0.CO;2. RUSS S. SCHUMACHER Department of Atmospheric Science, Colorado State University, Fort Collins, Colorado ADAM J. CLARK Cooperative Institute for Mesoscale Meteorological Studies, University of Oklahoma, and NOAA/OAR/ National Severe Storms Laboratory, Norman, Oklahoma (Manuscript received 6 November 2013, in final form 4 August 2014) Corresponding author address: Russ Schumacher, Department of Atmospheric Science, Colorado State University, 1371 Campus Delivery, Fort Collins, CO 80523. E-mail: [email protected] APPENDIX A Influence of Microphysical Fields in Initial Conditions on Convection-Allowing Forecasts As discussed in the text, the mixed-physics ensem- bles used in this research include both single- and double-moment microphysical parameterizations. As such, it is not possible to use hydrometeor number concentrations as a state variable during the assimila- tion in these ensembles, and for an equitable com- parison, they are not used as state variables in the single-physics assimilation systems either. In this ap- pendix, we present the results of tests to evaluate the importance of this potential limitation on the resulting forecasts. As a simple test of the influence of using number con- centration as a state variable, we ran the single_24hr assimilation for five cycles from 0000 UTC 9 June to 0000 UTC 10 June, identically to how they were run in the experiments reported in the main part of the man- uscript, except with number concentrations of rain, graupel, cloud ice, and snow as additional state variables. Then, we initialized forecasts from a selected single member (member 4) using these initial conditions, along with two experiments: one with all number concentra- tions set to zero, and one with all hydrometeor mixing ratios and number concentrations set to zero. The first experiment was designed to be the most drastic pertur- bation one might apply, where the hydrometeor mixing ratios are nonzero but the number concentrations are zero. (In the actual configuration used in the manuscript, the perturbation is much less drastic as the number concentrations remain at the value in the prior.) The second experiment was designed to determine how much influence the initial microphysical profiles have on the forecasts, considering that they are being inter- polated from a coarse to a fine grid, and no direct mi- crophysical observations (e.g., radar reflectivity) are being assimilated here. This is akin to a cold start that is often used to initialize WRF forecasts using initial con- ditions from other modeling systems. We focus on the first 12 h of the forecasts, when the initial microphysical profiles are likely to have the most direct impact on the forecast. The 0-6-h precipitation forecasts are nearly in- distinguishable between these four runs (not shown). By the 6-12-h forecasts, there are nontrivial differ- ences between the runs, especially in the vicinity of the MCV in northeast Texas, but these differences are quite small in the context of the variability within the ensemble as a whole (Fig. A1). Comparing Figs. A1a,b demon- strates the differences arising from using microphysics number concentrations as a state variable during the WRF-DART assimilation cycles. (Thus, there are also differences in all of the other model state variables in the initial conditions of these two forecasts.) The primary differences are small spatial shifts in the location of the heaviest rain in the MCSs in Nebraska and Texas, and also changes in the maximum rainfall amounts from these MCSs. The MCV-related rainfall in the manuscript configuration (Fig. A1a) is slightly less displaced from the observed rainfall location (Fig. A1e). In the forecast with the number concentra- tions set to zero in the initial condition, the MCSs are in nearly identical locations to those with realistic number concentrations (cf. Figs. A1b,c). The maximum rainfall in northeast Texas is somewhat higher in the run with the initially zero number concentrations, whereas the precipitation from the MCS in Nebraska is very similar. The differences in precipitation amounts are somewhat larger in the cold start run (Fig. A1d), although the locations of the MCSs are again nearly the same. These results, along with manual inspection of the initial conditions in other ensemble members (not shown) suggests that the effects of not using number concentration as a state variable in the ensemble data as- similation are relatively small in relation to other effects for the configurations used in this study. Thus, configura- tions with both single- and double-moment microphysics parameterizations are used in the mixed-physics ensem- bles here. With this being said, for applications where microphysical variables are a more integral part of the data assimilation system (such as when radar reflec- tivity is assimilated), these conclusions may not hold, and additional testing is suggested for such configu- rations. There are also some additional differences between these experiments at longer lead times, but whether these differences arise directly from micro- physical sensitivities or through feedbacks with other processes in the model is not clear and would warrant additional investigation that is beyond the scope of this study. APPENDIX B Sensitivity of Results to Assimilated Moisture Variable Based on the suggestion of a reviewer, we conducted a limited test to evaluate the sensitivity of the forecasts to the method of assimilating moisture in the DART system. Similar to the tests described in appendix A,we ran the single_24hr assimilation for five cycles from 0000 UTC 9 June to 0000 UTC 10 June, except with dewpoint instead of specific humidity as the assimilated moisture variable. The magnitude of the observation error was kept the same, but converted to dewpoint. Then, a 30-h convection-allowing ensemble forecast (i.e.,from0000UTC10Juneto0600UTC11June)was initialized with these analyses. Comparing the ROC area for this ensemble forecast to the one reported in the manuscript shows that the forecasts using the analyses with dewpoint as the assimilated variable were comparable at early lead times, and worse at longer lead times, than those using specific humidity (cf. black and red lines in Fig. B1). We then conducted the same experiment, except that the assigned observation error for land surface dewpoint was reduced by half in the DART assimilation process. The ensemble forecast initialized from these analyses did show higher ROC area in the 6-12-h forecast than the analyses using specific humidity, but lower ROC area by the 24-30-h forecast (cf. black and green lines in Fig. B1). These tests provide evidence that assimilating specific humidity as the moisture variable is not degrading the resulting ensemble forecasts, at least for this ini- tialization time. Uncovering the reasons for these dif- ferences is beyond the scope of this study, and despite the mixed results of this limited testing, recent work has offered some reasons why it may be more desirable to assimilate dewpoint rather than specific humidity (e.g., Fujita et al. 2007; Ha and Snyder 2014). Continued re- search on the implications of assimilating different moisture variables for the use in convection-allowing forecasts is encouraged. (c) 2014 American Meteorological Society |