Known Issues

Known Issues

The following are known issues for the August 2021 release.

User Interface Issues

1. Only NDVI and ET are available as raster views

    The requirements for visualization of satellite data products developed by the OpenET partners and working group members only included NDVI and ET for the raster view. Additional data layers will be added to the raster layer viewer in the future. 

    2. Model selection for the monthly timeseries graphs is reset with each new query

      Model selection for the monthly timeseries graphs is reset with each new query to avoid overloading the server with unnecessary queries and to increase response time for individual queries. Model selection may be changed in the future to be “sticky” based on query response times and user feedback. Queries for multiple models for large numbers of fields should primarily be handled through the API in the future.

      3. Only monthly data are available

      At this time, only monthly data are available through the Data Explorer. The OpenET is currently finalizing implementation of the daily data products, and these will be added to the OpenET Data Explorer and API in the near future.

      General Data Issues

      1. EToF values do not agree well, even when ET values do agree across the ensemble

        The fraction of reference ET (EToF) is calculated as the ratio of ET to ETo (grass reference ET). When ETo is low, small variations in estimated ET will translate to large differences in EToF. In general, EToF values from different models should agree well during the growing season. Large differences in EToF during the winter are expected, even when there appears to be generally good agreement among the ET values. 

        2. The range of the model ensemble (the difference between the lowest and highest model) appears large for fallow fields, pasture, and other polygons with low ET.

            The OpenET team has observed greater disagreement across the ensemble of models for low ET conditions that include fallow fields, partially irrigated pasture lands, and other fields with low ET. The OpenET team is currently working to explain and resolve these differences. Because the y-axis in the graphing windows scales based on the maximum ET values for a given field, fields with lower ET values will have a smaller range in the y-axis and, as a result, smaller differences in ET among the models will be exaggerated in the graphs relative to fields with higher monthly and annual ET values.

            3. Crop type information for the current year is incorrect, or crop type information for California appears to be incorrect for years other than 2016 and 2018.

            Crop type information for the current year is derived from the previous year, and is provided to assist with interpretation of ET values. Any inaccuracies or inconsistencies may be ignored. Outside of California, all crop type information is calculated from the USDA Cropland Data Layer.

            Crop type information for California for 2016 and 2018 is based on the CA Department of Water Resources (DWR) Land Cover database developed by LandIQ. Data for California for all other years are from the USDA Cropland Data Layer which has lower classification accuracies for many of the specialty crops grown in California. Crop type information will be updated for California as additional data updates are made available by CA DWR.

            Any errors in crop type do not affect the accuracy of the ensemble ET value, or the five models that use thermal infrared data as an input for calculation of ET.
            Crop type information is not used as an input for five of the models on the OpenET platform (ALEXI/DisALEXI, eeMETRIC, geeSEBAL, PT-JPL, SSEBop). The sixth model, SIMS, model does use crop type information to separate annual and perennial crops, and to adjust model parameters for stomatal resistance and maximum crop height for perennial crops. However, errors in crop type classification for annual crops don’t affect the SIMS ET calculations, and the crop-specific adjustments for perennials usually have a small influence on SIMS ET relative to other inputs. In cases where crop type errors do have an effect on SIMS ET data (e.g., misclassification of a perennial crop as an annual crop), SIMS is typically flagged as an outlier and removed from the ensemble prior to calculation of the ensemble ET value.   

            4. Field boundaries / data appear to be missing

            OpenET uses publicly available field boundary datasets. In some locations, these datasets are incomplete or missing large numbers of fields. The OpenET team is currently working with partners to improve and update these datasets. The Data Explorer ‘Raster View’ currently allows users to define any location of interest up to 1000 hectares (2,470 acres) and request data for the user-defined region. In the near future, the API and the Custom Reporting interface will allow users to define locations of interest and upload shapefiles to obtain data summaries for regions of interest.

            5. ET estimates do not agree well over open water bodies

            This is a known issue and the OpenET team is currently collaborating with other scientists to implement a method specifically for calculation of open water evaporation from satellite data and meteorological data. OpenET data over open water currently provided through the OpenET Data Explorer are provisional and likely to change in future updates.

            6. There are some locations where ET is zero in desert environments

            Evaluation of initial results from application of the Mean Absolute Deviation (MAD) approach for outlier detection highlighted a limitation in sparse vegetation desert environments during the warm season, where ET rates are very low. In these cases, the majority of models within the ensemble have a tendency to consistently estimate ET values of zero, while one or two other models provide estimates of low, but non-zero ET values. Requiring a minimum of four models allows non-zero values to be included in the ensemble average in most cases.  However, in some locations, at least four models are providing ET estimates of zero.

            7. There are some locations where data is missing for a month for one or more models.

            There are some time periods and locations where data is missing for a month for one or more models. This is usually due to the lack of high quality satellite observations due to snow cover or persistent cloud cover during the missing month. Ongoing work by the OpenET team is focused on addressing these data gaps through data fusion techniques and increased use of data from Sentinel-2. The recent launch of Landsat 9 is also expected to reduce the number of data gaps moving forward.


            8. The OpenET Ensemble Value appears low for small agricultural regions in very arid environments

            From the limited number of cropland in-situ flux stations located in very arid environments, it is evident that some models have a systematic low bias for smaller agricultural areas in very arid regions, and the MAD outlier filtering approach does not filter outliers as desired due to the large range is model estimates. This may result in a low bias in the ensemble average. These areas are often indicated by fields with a wide range of ET values across the ensemble of ET models. Over the coming months, the OpenET team will continue to conduct additional research in these more challenging settings and develop a Best Practices Manual that will provide more region and application specific guidance. The ensemble value is likely to evolve in the coming year as the team conducts additional research and designs more region-specific approaches for calculation of the ensemble ET value. We encourage users to rely upon their knowledge of local conditions in applying the ensemble ET value, or selecting a single model or subset of models for use in their application.

            Model-specific Data Issues

            1. ALEXI/DisALEXI

            DisALEXI was recently ported to Google Earth Engine as part of the OpenET framework, and refinement is ongoing. A few known issues in the current version of DisALEXI are noted here, which the team is working to resolve.

            Underestimation in semi-arid regions

            In the current version of ALEXI, ET is underestimated in some arid and semi-arid grassland regions in the western United States.  This will impact the DisALEXI ET values via disaggregation of the ET in small-scale patches of irrigated crops or pastures embedded in these grasslands. This underestimation is associated with a modification of the soil resistance formulation in the Two-Source Energy Balance (TSEB) land-surface representation (Norman et al., 1995) used in ALEXI and DisALEXI, which will be refined in future updates to reduce the dry bias in semi-arid grasslands.  In addition, an empirical aridity correction in the TSEB will be replaced in future work by the more physically based Priestley-Taylor alpha adjustment factor that was added to PT-JPL.  These adjustments will serve to increase ET in western grasslands and in small-scale irrigation districts that are significantly impacted by advection. Biases in these regions are somewhat stronger in 2020, apparently related to transition between GOES-W instrumentation. The source of the bias is being investigated and will be addressed in future updates. 

            Topographic biases

            DisALEXI exhibits biases in areas of strong topographic slope, with high ET bias preferentially on shaded hillslopes.  Future refinements will include the slope and aspect corrections to solar radiation load used in eeMETRIC.  This will improve hillslope ET estimates, as well as disaggregated fluxes from flat surfaces in ALEXI pixels containing high topographic variability which are effectively biased low in compensation.

            CFSR meteorological inputs and insolation monthly scaling flux

            To be consistent with continental-scale ALEXI execution, DisALEXI is currently using meteorological inputs from the Climate Forecast System Reanalysis (CFSR) – a coarse-resolution global dataset. Differences between the meteorological forcings (insolation in particular) will lead to differences in model output at the daily timestep – for example, DisALEXI may be significantly lower than other models on Landsat overpass days when CFSR has differences in cloud cover.

            Following the ALEXI procedure, interpolation to monthly ET for DisALEXI uses daily insolation rather than gridMET ETo as the scaling flux.  This will also introduce differences in comparison with other monthly datasets from other models, although these differences are not large in most regions of the modeling domain.

            In future work, DisALEXI will migrate to the common insolation and ETo scaling flux datasets used by the other OpenET models.  This will help to reduce non-model related differences with respect to the ensemble. 

            Coastal areas

            A high bias is evident in DisALEXI monthly fluxes along the Pacific Coast and the Great Lakes.  This is related to enhancement in ALEXI 4km-resolution ET in coastal pixels extending into the ocean, serving to similarly enhance disaggregated 30m fluxes within that and surrounding ALEXI pixels. These high coastal pixels are excluded as outliers in the ensemble product and do not affect monthly ensemble averages.  The high bias will be corrected in future work.

            2. eeMETRIC

            Some uncertainty in monthly estimates by eeMETRIC, as well as with other methods, is caused by infrequent clear satellite images during cloudy periods or in locations that are prone to clouds. Infrequent images may exhibit wetness or dryness associated with timing of precipitation events that is not representative of a monthly period.  During winter periods, thermal contrasts within an image may be too small for eeMETRIC to distinguish differences caused by ET.  

            Evaporation from open water is estimated by eeMETRIC using an aerodynamic equation rather than using a surface energy balance. This is done due to large uncertainties associated with heat storage fluxes into water bodies. However, there is uncertainty associated with the estimation of vapor pressure over water that is required in a full aerodynamic method, as well as whether wind speeds during the satellite overpass time are representative of those over a month. Estimation of evaporation from open water will progress in eeMETRIC over the next year.

            3. PT-JPL

            For croplands, wetlands, and riparian areas in arid and semiarid environments where advection is prevalent, a PT alpha adjustment layer was developed based on the ratio of ASCE reference ET and Priestley-Taylor ETw, following principles of the complementary relationship of evaporation (Kahler and Brutsaert, 2006; Szilagyi, 2007; Huntington et al., 2011). ASCE reference ET and ETw estimates were developed using the gridMET dataset. The alpha adjustment layer was calculated as the ratio of the growing average bias corrected ASCE reference ET to the growing season average ETw. Growing seasons were defined for each gridMET grid cell based on cumulative growing degree days of daily average air temperature from January 1 and killing frost thresholds of 300 C and -2 C, respectively. Adjustment factors were limited to a minimum of 0.79 and maximum of 2 to account for uncertainty in weather data and model performance in locations with extreme climate and weather (e.g. Death Valley, coastal regions). Adjusted alpha values range from 1 to 2.5, and fall within the range of alpha values previously reported across arid and humid settings (Priestley and Taylor, 1972; McAneney and Itier, 1996; Weiß and Menzel, 2008; Yang et al., 2009; Tabari and Talaee, 2011). Application of adjusted alpha values were limited to cropland, wetland, and riparian areas as defined by the USDA cropland data layer (CDL).

            Results from Phase II of the OpenET Accuracy Assessment and Intercomparison study indicated that application of the PT alpha adjustment layer increased overall accuracy of PT-JPL for cropland, wetland, and riparian areas. In cases where land cover information is incorrect, the correction may not have been applied, resulting in a low bias in estimates from PT-JPL. Future enhancements to PT-JPL will include the slope and aspect corrections to short and longwave radiation and LST inputs, similar to eeMETRIC, and further improvements to the PT alpha adjustment layer.

            4. SIMS

            The SIMS model generally performs well over irrigated agricultural areas and shows strong agreement with the ground-based ET datasets used in the OpenET Intercomparison and Accuracy Assessment. However, a positive bias in SIMS ET estimates for rainfed croplands, deficit irrigated fields, and fallow agricultural lands is a known issue. The SIMS model is primarily responsive to satellite measurements of green vegetative cover and assumes well-watered conditions. As a result, SIMS can overestimate ET from partially irrigated or deficit irrigated fields and long-term fallow areas, especially when grasses or weeds are present but not irrigated. The team has implemented a gridded soil water balance model on Earth Engine to improve estimation of evaporation from bare soil and identify periods when available soil water in the root zone is likely to serve as a constraint on ET for non-irrigated lands. 

            SIMS was developed to support ET estimation and irrigation management over agricultural lands. For the purposes of providing continuous ET estimates across the western U.S., a simple reflectance-based model was implemented for non-agricultural land cover types. This approach was based on a linear transformation of NDVI, and did not incorporate the full density-coefficient approach used for agricultural lands. As a result, it frequently produced ET estimates with a very high positive bias for non-agricultural land cover types and these data were excluded from inclusion in the OpenET ensemble value. The data is included in the SIMS archive and accessible via Data Explorer queries in ‘raster view’ or the OpenET API. For non-agricultural areas, SIMS ET data should be interpreted as an upper limit on ET and a measure of potential ET that accounts for the vegetation density and condition. Future work by the SIMS team will extend the density coefficient approach used by SIMS to non-agricultural land cover types, and incorporate the soil water balance model to account for soil water limitations on ET. 

            5. geeSEBAL

            Implementation of geeSEBAL within the OpenET framework was just recently completed. Overall, geeSEBAL performance is dependent on the domain area, including topography and climate conditions, and the endmembers selection for automated calibration. The team is currently working to improve these limitations to increase model accuracy at multiple climate conditions and complex topographic landscapes. Some of the limitations are discussed below.

            Complex topographic landscapes

            geeSEBAL frequently exhibits a bias in areas of high altitude and steep slopes. These topographic features affect the model’s endmember selection algorithm. A review of the surface temperature lapse rate, combined with the addition of terrain slope and aspect considerations to correct land surface temperature and to compute solar radiation will potentially improve results in such areas.

            These corrections will also improve the hot and cold endmembers temperatures for internal calibration, improving ET estimates in the model.

            Desert and arid areas

            In desert areas and arid or semiarid climates, geeSEBAL tends to yield lower ET estimates, producing ET estimates of zero or nearly zero over sparse vegetation areas. On the other hand, some vegetated areas can yield higher values, overestimating evapotranspiration in these conditions. This is possibly caused by the elevated temperatures in some land cover classes, such as bare soils or sparsely vegetated areas. The geeSEBAL team is currently working on improving the endmembers selection algorithm to provide more accurate ET estimations in desert and sparsely vegetated areas.

            Large water bodies

            The geeSEBAL team members are currently working to investigate and characterize model uncertainties and data gaps over large water bodies. One primary issue is related to masking geeSEBAL meteorological inputs (NLDAS and gridMET) over large water bodies (coastal areas and continental waters) for interpolation, to avoid high bias over land surfaces. Future improvements to geeSEBAL will include separate solutions for open water evaporation.

            6. SSEBop

            The following four known issues, or causes of SSEBop ET model error, have been identified for user interpretation and awareness. Future improvements will address high and low biases observed for different land cover types.

            Cloud and atmosphere contaminants
            The SSEBop model does not include additional cloud buffering techniques during processing. The source Landsat image quality band used for masking bad pixels is applied without advanced screening such as cloud- or shadow-buffering techniques. In some instances, model estimates may at times result in abnormally high ET outliers because of imperfect cloud masks. 

            Puget Sound Underestimation

            SSEBop is underestimating ET over agricultural lands in the Puget Sound region (WA state) and some coastal areas with complex topography. The underestimation on croplands is tied to model parameters being influenced (during spatial smoothing of parameters) by adjacent hills and mountains with different hydroclimatic conditions. Future improvements will address these shortcomings. 

            Barren lands and bright surfaces

            Model calibration related to land surface temperature and simplified surface energy balance parameterization is attributable to an observed high bias in ET estimates for high albedo, bare soil, and sparse shrubland areas. Examples include parts of northern and southern Arizona (AZ), Holloman AFB (NM) surrounding areas, Big Bend National Park (TX), and many playas in the Great Basin region. These are limited areas and not well-vegetated. The SSEBop team is currently improving the model parameterization scheme to correct for these known issues. 

            Missing emissivity input data
            Gaps in images may exist related to missing emissivity source data in land surface temperature Level-2 model input data products (LST Data Caveats).