Old Monitoring: Description
This page contains the NRT monitoring of data up to 1st May 2017. Improved NRT monitoring of RO data can be found at NRT monitoring starting at 1st May 2017.
(O-B)/B plots | Time-trend | (O-B)/B histogram | Occultations map | Zonal (O-B)/B | 1D-Var stats | Correlation/Covariance | Matched occultations | Co-located radiosonde monitoring | Delay histograms | Data availability
Definitions, acronyms and abbreviations:
B Forward modelled refractivity or bending angle (referred to as background)
The monitored radio occultation data are provided to the Met Office as refractivity and bending angle profiles processed by the ROM SAF (for the GRAS instrument), GFZ (CHAMP and GRACE-A satellites) and from UCAR (FORMOSAT-3/COSMIC, SAC-C and C/NOFS). The GRAS observation data used in the plots are obtained from DMI BUFR files, which are disseminated in NRT via the GTS. Any GRAS plots specified as ECMWF data will be using only bending angle data, which is sourced from EUMETSAT BUFR files (the ECMWF statistics are calculated at ECMWF and received at the Met Office already processed to (O-B) statistics). Most plots are based on the comparison of observation data to forward modelled Met Office global model (operational) 6 hour forecast data and forward modelled ECMWF (operational) 12 hour forecast data (upgrades to these NWP systems are logged here). The refractivity and bending angle forward models are currently both 1D local operators (see ROPP documentation for bending angle operator description).
The Met Office refractivity forward model simulates refractivity as a function of geopotential height and is relatively straightforward. It requires model total pressure and specific humidity on geopotential heights horizontally interpolated to observation location, from which temperature can be calculated and hence the refractivity (using an empirical formula). This is followed by a vertical interpolation to the observation heights. The bending angle operator follows the same method up to producing refractivity on model levels, the bending angle is then calculated using the inverse Abel transform (as in ROPP). The ECMWF data use a very similar bending angle forward model to the one employed by the Met Office.
The IDL monitoring is automated so that the plots on the web site are updated every 6 hours with a delay of roughly 8 hours since the last model run, and is based on roughly the previous 30 days data. Note that this system is not operational and there may be short periods when the plots do not get updated. The majority of the monitoring plots are based on (O-B)/B statistics (using the Met Office global model 6 hour forecasts as B unless otherwise stated). The plots should represent the quality of the data we receive at the Met Office (and therefore any other NWP centre) taking into account the idiosyncrasies of the model (unique biases). However, it should be noted that some quality control is performed to remove extreme outliers. Firstly we disregard any unphysical data in the observation and background e.g. refractivity, bending angles and associated height co-ordinates less than zero. We then do a check for extreme outliers in the (O-B)/B on a level-by-level basis: we reject any values with |(O-B)/B| > 10*ob_error% where ob_error is the observation plus forward model assumed errors. For bending angle the assumed errors are the same as those used at ECMWF.
Data Quality Monitoring:
(O-B)/B profile plots
These plots allow us to compare the refractivity (or bending angle) calculated using the Met Office's global model fields with the refractivity (or bending angle) data given in a corresponding GPSRO observation. The fractional difference in refractivity (or bending angle): (O-B)/B, is calculated for each occultation profile at a fixed set of altitude levels (200 m intervals, the data were interpolated on to these levels using a log-linear method). The altitude data are geopotential height with respect to the geoid for refractivity and impact height with respect to WGS-84 ellipsoid for bending angle.
The mean of (O-B)/B over many occultations for each altitude level is calculated. Over many thousands of occultations this mean indicates the typical differences (biases) between the observed and modelled refractivity (or bending angle). We also show the standard deviation of (O-B)/B, indicating the width of the distribution. The plots show the number of occultations used in each mean and standard deviation calculation i.e. number that got past the QC.
The (O-B)/B plots are separated into: global data, latitudinal bands; northern/southern hemispheres/tropics, land/sea, setting/rising and nominal/no-nominal. Bending angle (O-B)/B statistics provided by ECMWF have been added to the global and northern/southern hemispheres/tropics (O-B)/B plots. The profile plots are useful for comparing satellites/models/processing centres and thus for making decisions on which data to assimilate into NWP models. The land/sea bias plots were created using a land-sea mask on a resolution of 1 degree in latitude and longitude.
These plots show how global refractivity (O-B)/B statistics vary with time on altitude levels. The mean (O-B)/B over a 6-hour period is plotted as a the colour of a box for a given time and altitude, this is repeated for all times and altitudes for the data over the defined period. Similarly plots of standard deviation and a count of the number of occultations used in the mean and standard deviation calculations are produced. The plots allow changes in the data quality of the observations or background to be seen. Also available are ECMWF and Met Office bending angle statistics against time on the same plot for a selection of impact heights; with Met Office statistics plotted every 6 hours, ECMWF every 12 hours. They help to determine whether changes with time are related to observations or the models.
These plots show the refractivity (and bending angle) (O-B)/B mean, (O-B)/B standard deviation and observation count averaged over latitudinal bands around the globe. Each band is 4 degrees wide in latitude to ensure sufficient data for the average. The zonal averages are calculated on each altitude level, and the result is shown as a contour plot.
(O-B)/B histogram plots
Histograms of (O-B)/B values are produced for all the altitude levels for refractivity and bending angle. The histograms are normalised (so the area under the histogram at any particular height level is 1), hence the axis label PDF (probability density function, a reasonable estimate given a large enough sample). Given a histogram for each level we then plot each one against altitude with the value of the PDF shown as a colour. The plot gives a lot more information on the spread of the (O-B)/B compared to simply the standard deviation i.e. the shape of the distribution with altitude can be seen.
At the Met Office we pass radio occultation data through a 1D-Var procedure for QC purposes prior to assimilating the data in 4D-Var; the aim of this is to black-list poor quality occultations. We run this 1D-Var in a non-operational capacity for monitoring. We set the 1D-Var to have no vertical cut-off (typically 0 to 60 km) and to use all the data received at the Met Office i.e. no QC prior to 1D-Var is done.
The 1D-Var hist plot shows histograms of the number of iterations required to converge, 2J/m (where J =cost function at solution and m = number of observations in a profile), 2J/m when J is the initial cost function and DFS (degrees of freedom of signal) which is calculated as the trace of AKTO-1K where A is the solution error covariance matrix, K is the forward model Jacobian matrix and O is the observation plus forward model error covariance matrix (see Inverse Methods for Atmospheric Sounding C.D. Rodgers pages 33,37).
The 1D-Var time series plot shows the trend of the 1D-Var statistics: the number rejected (and why) and the 2J/m value on first and final iterations. 2J/m should be roughly 1 if the background and observation plus forward model errors are specified correctly. Another plot shows 2J/m vs latitude, i.e. the final 2J/m against tangent point latitude of the occultation. Both plots are useful for determining the quality of data.
The plot shows the position on a global map of the occultations for the chosen satellite over the designated time period. The colour of the marker for the occultation position corresponds to the lowest geopotential height provided with the refractivity profile. The height can be read from the colour bar provided below the map. The shape of the marker indicates whether it is a rising or setting occultation. The plot provides an indication of the global spread of the data and allows one to see any spatial patterns in lowest processed height. Another map (Gradient map) plot shows the altitude at which the observation and model background gradient of refractivity is less than -100 N/km shown at the locations of the occultations. This indicates where on the globe and a what heights super-refraction is likely.
To get a qualitative feel for observation error covariance, the vertical correlation and vertical covariance matrices of (O-B)/B for refractivity (bending angle) plotted against geopotential height (impact height) are provided. Although clearly some of the correlation of (O-B)/B comes from B (the model refractivity or bending angle) we can still get a feel for the observation part of error correlation by performing comparisons between satellites/processing centres, when B should be the same (although different satellites sample different local times). Also the difference between refractivity and bending angle provide an interesting comparison.
Matched occultation plots
Given an occultation one can search for other occultations that lie within a specified time-range and distance from the reference occultation. Statistical comparisons of the refractivity/bending angle observation data from these matches can then be made. The plots show how the mean and standard deviation of the fractional difference vary with latitude between different GPSRO receivers and processing centre combinations. Currently the criteria for a match is that the great-circle distance is less than 300 km and the time difference is less than 3 hours. Surrounding the mean in a dashed line is the 95% confidence interval on the mean to help judge the significance of the result.
Co-located radiosonde monitoring
Given an occultation one can search for radiosondes that lie within a specified time-range and distance from the reference occultation. Statistical comparisons of the refractivity GPSRO observation data to the radiosonde-derived refractivity can then be made. The plots show how the mean and standard deviation of the fractional difference vary with latitude between different GPSRO receivers and radiosonde type combinations. Currently the criteria for a match is that the great-circle distance is less than 300 km and the time difference is less than 3 hours. Surrounding the mean in a dashed line is the 95% confidence interval on the mean to help judge the significance of the result. The plots are also split by day/night-time to help distinguish the effects of diurnal radiosonde biases. For the calculation of radiosonde refractivity we use the commonly used Smith & Weintraub formula, given the radiosonde temperature and dew-point temperature as a function of pressure. The geopotential heights are calculated by summing thicknesses between radiosonde significant level data using the hypsometric equation. The calculated radiosonde refractivity is interpolated onto the standard set of GPSRO geopotential heights used in the other plots using the same log-linear method.
Data Flow Monitoring:
These plots show statistics of the time delay between the observation time and the time of receipt at the Met Office.
These plots are produced for yesterday's data and for a 28-day (monthly) period of data.They are important for judging how quickly the data is (or could be) assimilated into NWP models.
It should be noted that the flow of data provided to the Met Office can stop occasionally for reasons beyond our control, hence why some of the plots may be blank.