Assimilation of PC-reconstructed radiances in NWP

Hyperspectral missions have been providing key information to numerical weather prediction (NWP) for about two decades, e.g. from AIRS, IASI or CrIS instruments. The observations are disseminated to the users in the form of spectra made of thousands of channels. However, only small subsets of radiances (of the order of 100 channels) are actually assimilated in NWP models.

Principal Components Analysis enables high compression rates, while preserving the thermodynamic information and reducing the instrument noise in individual channels. The IASI principal components (PC) products from EUMETSAT achieve a compression factor of about 30 and have been operational since 2011. However, no operational experience has been established in NWP as users continued using the original radiances from the complete spectra, which were still disseminated in parallel.

The future geostationary sounder MTG-IRS will provide an unprecedented 4D look into the atmosphere. However, the data volume resulting from the unique spatial and temporal sampling demands that spectral information be compressed and disseminated in principal components scores. In this study, IASI is used as proxy for IRS to address a priori perceptions within the NWP community regarding the utilisation of principal components products.

The study performed at ECMWF shows that radiances reconstructed from PC products can be seamlessly ingested instead of original radiances in an operational assimilation system, with minimal adaptation. It also shows that the underlying eigenvector basis can be seamlessly updated over time without any system recalibration, ensuring operational continuity. The forecast’s performances with reconstructed radiances remain as high as with original radiances in all experiments. Therefore, principal components products form a plausible alternative route for the exploitation of high spectral resolution infrared sounders in NWP.

Objectives

The overarching question to be addressed was: Can an assimilation system set-up for conventional IASI radiances be switched to using reconstructed-radiances instead, without system retuning or forecast performance loss?

Study objectives:

• Set up and run assimilation experiments of reconstructed radiances using different eigenvector configurations in the integrated forecast system (IFS) of ECMWF.
• Evaluate the practicalities and impact of assimilating reconstructed, instead of original radiances by comparing the forecast skills against external references.
• Evaluate operational continuity in case of an eigenvector basis update.
• Share with the user community the conclusions and guidelines on the generation and utilisation of PC products for IRS.

Overview

System preparation and assimilation experiments definition
Radiance monitoring and cloud detection
Assimilation in a depleted environment
Assimilation in a full observing system
Assimilation run including an update of the eigenvector basis
Conclusions
Optional phase — Follow up on new methodologies to determine the observation error matrix

System preparation and assimilation experiments definition

In this study, two sets of PC products were used, resulting from two different eigenvector basis:

1. v1.04 currently operational
2. v2.01 to be released in the upcoming IASI L2 processor v6.7, second half of 2022.

The assimilation experiments with the original radiance, and the two sets of reconstructed radiances, first ran in a depleted observing system and then in a full observing system. They included observations from the three Metop platforms.

A pre-processing step to reconstruct the radiances from the principal components products was introduced. This was the only adaption made to the assimilation system. The same configuration as in normal operations with original radiances was used for the assimilation of reconstructed radiances. This includes the channel selection, variational bias correction (varBC), cloud screening and observation error.

Radiance monitoring and cloud detection

• The standard deviations of the background departures are reduced for temperature channels with reconstructed radiances, whereas they remain similar to statistics with original radiances for water vapour channels.
• The biases are generally unchanged for all channels.
• Comparable data volumes pass the cloud detection algorithm with original and reconstructed radiances.

Assimilation in a depleted environment

In the first phase, the assimilation experiments were carried out in a depleted environment.

• Assimilation trials in a depleted system (no high-resolution IR sounder) suggest that reconstructed radiances (v1.04 and v2.01 baselines) can deliver as much thermodynamic information as IASI conventional radiances.
• Both original and reconstructed radiance experiments produce similar forecast skill impact, and show statistically significant improvements over the depleted control.
• No statistical difference in performance between the reconstructed experiments based on v1.04 and v2.01 eigenvectors bases.

Assimilation in a full observing system

The control run in the full observing system includes all observations available in nominal operations, including the infrared sounders, with the exception of IASI. It is expected that the impact of original and reconstructed radiances on the forecasts is less than in a depleted environment. But the relative impact of original and reconstructed radiances can be evaluated.

Assimilation run including an update of the eigenvector basis

In this last series of experiments, the eigenvectors basis of the PC products was updated ‘on-the-fly’, halfway through the assimilation period, while keeping all system configuration unchanged. The subsequent forecasts skills, having updated the eigenvector basis, are compared to assimilation experiments where the basis remained unchanged.

Conclusions

✔ No loss of performance switching from original to reconstructed radiances in a depleted environment

• Assimilation of IASI reconstructed radiances working with minimal adaptation in the IFS.
• Positive impact on analyses and forecasts, like the conventional IASI radiances.
• Very promising results using the current dedicated operational observation errors for IASI radiances.
• Plausible route for the exploitation of high-resolution IR data in NWP.

✔ No indications of performance loss switching from RAD to REC_RAD in the full system.

• Full system experiments conducted to assess REC_RAD impacts when analysis errors are small.
• Positive impact on analyses and forecasts, similar to RAD; Short and medium-range impact differences between REC_RAD systems and RAD are not statistically significant.
• OSEs assessment was complemented by an analysis of Forecast Sensitivity to Observation Impact diagnostics.

✔ No indications of performance loss switching between REC_RAD eigenvectors basis, ensuring operational continuity.

• REC_RAD eigenvectors base was changed 'on-the-fly' from v.104 to v2.01 to simulate a necessary update in an operational context.
• Impact on analyses/forecasts is similar to RAD.

Optional phase — Investigating new methodologies to derive suitable observation error matrices for new instruments with spectral correlation

The success of assimilating reconstructed radiances was also permitted by the pre-existing suitable, well-tuned observation error matrix for IASI radiances.

Further work on the determination of a functional observation error matrix would be very beneficial for any new instrument data presenting error correlations. The problem is not specific to reconstructed radiances, whose operational potential was shown in the first phase.

Therefore, an additional exploration has been undertaken to accelerate the generation of a functional observation error matrix and hence accelerate the operational uptake of MTG-IRS in NWP after commissioning. This includes exploring parametric approaches and investigating the observation and model subspaces.