Product Quality of Offline & CDR products

These pages summarize the quality assessment of ROM SAF Climate Data Records (CDRs), version 1.0, as well as the ROM SAF Offline & ICDR products.
In general the following semantics are followed: O: Observation; S: ID-Var solution; A: Model Analysis; B: Background, i.e. the closest model forecast interpolated in time and space which is independent of the occultation itself (see 1D-Var ATBD for details); ERA-I: ERA Interim forecast or analysis data; ECMWF: Daily operational EWMWF forecast or analysis data.


 Plot Description

1D Monthly Profiles

The 1D monthly profile plots show differences between the retrieved GNSS-RO variables and equivalent values calculated using fields from ERA-I or operational ECMWF data. Comparisons with both model analysis (A) and model forecast (B) are made available.

The variables temperature, dry temperature, pressure and specific humidity are compared in terms of difference between absolute values, and refractivity, bending angle, optimized bending angle, pressure and specific humidity are compared in terms of their fractional differences.

In addition a small suite of diagnostic plots related to performance of 1D-Var are made available at the end of the drop down menu. These are the mean adjustment in refractivity space, the 4 Desrozier -ratios and the solution error relative to the a priori error.(These could need a separate explanation.)

Profiles are interpolated (cubic spline interpolation) to a fixed altitude grid following this (quadratic) form: [0 2 4 6 .. 284]2 m, i.e. [0, 4,16,36,64,100,144 .. 79524] m, (143 levels). Impact height, geometric altitude and geopotential height are is used as independent vertical variable for Level 1B, Level 2A and Level 2B respectively.

The fractional differences in bending angle, refractivity and pressure are calculated as (X-Y)/Y, where Y refer to either analysis or forecast from ERA-I or operational ECMWF. These plots are called "relative". Due to relatively large variability in specific humidity, the fractional differences for specific humidity are calculated as (X-Y)/mean(B), where B (background) refers to the ERA-I forecast, i.e., the specific humidity difference is normalized with respect to the monthly mean of the ERA-I forecast. These plots are called "normalized". In this particular calculation the background specific humidity (B) is interpolated linearly to the common grid instead of using splines.

For all three plot types - absolute, relative and normalized - the monthly statistics - mean, standard deviation, median and median absolute deviation - are calculated after interpolation and possible division with reference profile.

Solid black lines show mean values, dashed black lines show standard deviations, solid red lines show median and dashed red lines show the median absolute deviation (MAD). MAD(x) = k median(|x-median(x)|), where k = 1.4826 is chosen such that the MAD equals the standard deviation for normally distributed data.

The plot on the right shows the percentage of data points valid at a given height. The number of occultations used in the statistics for that month, i.e. number that were accepted by the quality control is printed in the title.

Product Service Specifications

The schematic graphs show with a color code whether different product specifications have been violated during a given month and altitude interval, for a given variable and satellite mission. If an area is hatched it means that the service specifications for the variable were violated on average during the corresponding period. The profiles are divided into latitudinal zones: tropics (30° S-30° N), mid-latitudes (30° S-60° S and 30° N-60° N), and polar regions (60° S-90° S and 60° N-90° N). Each zone is further separated into height layers, which depend on the product type. Monitoring of the tropopause height performance is found the scalar timeseries.

Compliance with specifications:

For each variable and day we calculate the mean and standard deviation as follows: (i) Bending angle and refractivity: "observation - background", (ii) 1D-Var products: "solution - ERA-I analysis". If monthly STD value exceeds the product requirements or service specification at one or more altitudes, the color code for that violation is used for the corresponding altitude range.

Product requirements and service specifications are calculated in many different ways, based on mean and standard deviations.These are summarized in the PRD and SeSp documents found at Three product requirement accuracy levels are specified: threshold, target, and optimal. The color codes seen in the color bar indicate that the given requirements are met, i.e., not violated; e.g., the color yellow means that threshold level is met but the target level has been violated.

2D Time Series

These plots show monthly standard deviation or mean of the chosen variable comparison, satellite and area. The plots include all data from the selected RO mission that passed the QC screening (nominal data). The monthly STD and mean used to produce these plots are exactly the same as those used in "1D Monthly Profiles". The standard deviation plots of temperature, humidity and pressure, compared to ERA-I analysis, are overlayed with contours enclosing areas where requirements (solid lines) or service specifications (hatched areas enclosed by purple lines) are violated. The requirements and specifications are summarized in the PRD and SeSp documents.

Scalar Time Series

The scalar time series plots show differences between retrieved GNSS-RO scalar variables and equivalent values calculated using fields from ERA-I or operational ECMWF data. Comparisons are made available in 8 different latitudinal zones. For the tropopause an additional narrow tropical zone (15° S-15° N) is also monitored.

The available variables, surface pressure, tropopause height and scaled costfunction, are of different kinds and therefore treated differently:

The pressure is compared in terms of absolute difference with both model analysis (A) and model forecast (B). Both standard deviation and mean are shown in the same plots.

The tropopause height and temperature are viewed in separate mean and standard deviation plots (that is 4 plot types). Three tropopause heights are compared to ERA-I analysis: Refractivity based, dry temperature cold point based and dry temperature lapse rate based. For tropopause temperature there is only two types presented (dry temperature cold point and and dry temperature lapse rate).

Instead of evaluating the difference between all tropopause estimates and all model tropopause estimates, we show the difference between different model based tropopause estimates and ERA-I lapse rate tropopause in the "Model Lapse Rate Tropopause" and "Model Cold Point Tropopause" plots.

The scaled cost function at the end of the left drop down menu is a diagnosis of how well the 1D-Var retrieval of temperature, specific humidity and surface pressure performs, in a well tuned 1D-Var its value should be close to unity. For more informatin, see the 1D-Var ATBD which is found here

Vertical Mean Time Series

The vertical mean plots are designed to monitor the Level 1B and Level 2A products with respect to the service specifications, which are defined as monthly vertical mean values for (O-B) mean and standard deviation. Here O is the retrieval and B is the ERA-I forecast. Service Specifications are marked with red dashed line (only for single missions - not for combined data sets).

Product Quality of Offline & ICDR products

DMI's logo IEEC's logo Met Office's logo
ECMWF's logo