Assimilation of satellite sounder profiles in NWP

About

An accurate description of the state of the atmosphere is crucial to initialise NWP, for which satellite observations provide key information. In particular hyperspectral IR sounders have become one of the most important contributors in the last two decades, with the AIRS, IASI and CrIS missions.

As outlined in [Eyre et al. 2019], geophysical parameters retrieved from satellite observations were assimilated in numerical models up to the 1980s. The NWP centres then changed their approach and started to assimilate radiances (L1) in the 1990s instead of retrieved physical quantities (L2). One motivation at the time was to avoid degrading the model background with inaccurate information from the null space, e.g. with small-scale vertical structures or from atmospheric layers for which these passive sounders have little to no sensitivity. This was guaranteed by the utilisation of the radiative transfer model as an observation operator.

Despite the operational maturity and success of this approach, it still faces scientific and technical challenges, which limits the exploitation of the IR sounders to their full extent:

• Uncertainties due to the parameters not accurately modelled: trace gases, surface emissivity, levels above the model top, misclassified clouds…;
• Radiative transfer in clouds being still an active area of research;
• The heavy computational load incurred by the radiative transfer calculations.

Consequently, only a limited subset of channels are assimilated, typically from a few 10s to about 200 channels depending on the numerical models and the scenes. The selection is usually limited to cloud-free pixels or to those channels peaking above the clouds. Because of the uncertainties in the radiative transfer modelling, channels peaking in the lower troposphere tend to be excluded or down-weighted, in particular for humidity and over land, where even fewer channels are assimilated. The definition of a functional observation error matrix, required in the assimilation process, has been mostly empirical and is still subject to active studies, especially for spectrally correlated observations, like with hyperspectral sounders [Assimilation of reconstructed radiances].

The present study explored the operational potential for NWP – both from a scientific and a functional perspective - of assimilating all-sky L2 products from satellite sounders, with a scene-dependent error characterisation and observation operators. It builds on the promising results from preliminary studies at ECMWF and Météo-France, where retrieved profiles were assimilated as pseudo-sondes. Though not accounting at the time for the vertical error correlations and the vertical sensitivity functions, the L2 yielded positive impacts on forecasts, including with cloudy retrievals [Preliminary assimilation of L2].

The atmospheric temperature and humidity profiles used here are retrieved with a prototype evolution of the operational machine learning method [An introduction to the PWLR], from which it is also possible to compute associated averaging kernels and error estimate covariance. In this new approach, the L2 products are represented in a limited set of principal components (typically 10 to 25), which already accounts for the intrinsic vertical resolution of the sounder. The study was performed with retrievals using IASI measurements only – i.e. without exploiting the companion microwave sounders on-board Metop- to serve as proxy for MTG-IRS products.

The aim was to convey more geophysical information to the NWP system than through radiances because the retrievals exploit the full spectral range measured and because of the provision of sounding in cloudy pixels. Furthermore, as it does not necessitate a Radiative Transfer Model (RTM), the assimilation of such L2 products is expected to be computationally significantly less expensive than with L1 radiances. This technique could hence be particularly interesting in contexts where strict trade-offs exist between stricter timeliness requirements in NWP, cost of CPU resources and the ever rising amount of satellite information in NWP. MTG-IRS, in particular, will produce 100 times more observations than current missions like IASI.

Objectives

The study objectives were to:

• Evaluate the practicalities of assimilating atmospheric profiles in PC scores, with scene-dependent observation operators;
• Evaluate the precision of the sounding products (L2) and the pertinence of the associated uncertainty estimates;
• Set-up, run and evaluate assimilation experiments of L2 products in the ECMWF IFS, compared to the assimilation of radiances;
• Provide feed-back and guidelines to the users and product developers in view of their potential utilisation in NWP.

Overview

• System preparation and assimilation experiments definition
• Passive monitoring: products precision, uncertainty estimates and observation operators
• First assimilation experiments: depleted environment, maritime, clear-sky
• Optimising the observation error
• Assimilating cloudy retrievals
• Full system, assimilating maritime all-sky retrievals
• Conclusions
• Outlook
• Please link bullets above to sections below

System preparation and assimilation experiments definition

In this study, the elements of the cost function minimised during data assimilation

J(x) = (x-x_b)^TB^-1 (x-x_b) + (y – H[x])^TR^-1 (y – H[x])

are

• x is the geophysical state vector resulting from the assimilation – here T and q profiles
• x_b is the background state vector, the forecast from the previous run
• B is the background error covariance matrix
• y is the observation vector – here the L2 products T/q in PC
• H[x] is the observation operator – here transforming the vertical profiles of T/q in PCs
• R is the observation error covariance matrix

The observation operator H writes

H[x] = (U^TSU)^-1/2U^TE^T(x-<x>)

where

• x is the geophysical state vector analysed during the assimilation – here T and q profiles
• <x> is the average state vector
• E are the truncated eigenvectors (leading directions) transforming the profiles in PCs
• U is the left singular-vector of the retrieval averaging kernels
• S is the retrieval uncertainty covariance matrix

E, U and S are scene-dependent, defined for different observations classes. The classification is obtained by k-mean clustering applied to the observations [An introduction to the PWLR]. By construction, the error covariance matrix of the observations is expected to be the Identity matrix.

The IFS framework has been successfully adapted to ingest the L2 products in PC space. The assimilation experiments are composed as follows:

Passive monitoring: products precision, uncertainty estimates and observation operators

The main outcome of the passive monitoring of the IASI L2 against ECMWF forecasts, in profile (Fig.1) and PC (Fig.2) space are:

• Good quality retrievals in profile space;
• Reliable auxiliary information for quality control and data acceptance - cloudiness intensity (OmC, OBS-CALC estimates in window channels) and uncertainty estimates (QI);
• Best precision, within or below 1K all along in the vertical, is achieved with clearest pixels (|OmC| < 1 K) and best quality (uncertainty QI_T < 2 K);
• The quality in the lower atmosphere decreases as the cloudiness and error estimates increase (consistent with IASI only providing information from above the clouds in strong overcast);
• The O-B variance for temperature in PC is overall close to 1 as per theoretical expectations for most situations. However, it exceeds 1 in the trailing directions and more surprisingly in the first leading direction in clear-sky;
• the O-B variance for humidity in PC is lower than 1 overall, suggesting that the error estimates on IASI L2 humidity might be slightly overestimated.

First assimilation experiments: depleted environment, maritime, clear-sky

The first assimilation experiments were carried out in IFS with:

• L2 temperature and humidity in PC - both observation errors set to Identity matrix;
• L1c radiances as per operational baseline;
• Clear-sky (different cloud mask, based on L1 and L2 information respectively);
• Maritime pixels.

Outcome:

• Positive impact of temperature and humidity forecasts with L2;
• Forecast performance with L2 lower but close to L1 in the free troposphere;
• L2 yields comparable positive impact as L1 on temperature in the lower troposphere;
• L2 yields stronger positive impact than L1 on humidity in the lower troposphere.

Optimising the observation error

As explained in the first paragraph, the observation error covariance matrix should be 1, by construction of the observation operator. However, Figure 3 hints that the error estimates provided along the L2 products may be overestimated for humidity. Though the statistics of the difference model vs L2 also include forecasts errors in addition to L2’s errors, it may be that the L2 uncertainty estimates are underestimated in some directions for temperature.

A series of experiments was therefore attempted to try and optimise:

• The information conveyed to the model, i.e. truncating or using the full 15 eigenvectors;
• The observation error, i.e. decreasing or inflating the observation errors in step between 0.25 and 1.75.

Outcome:

This best forecast scores (see Figure 5) were obtained with:

• All 15 directions retained for temperature and humidity;
• The error on temperature set to 1.5 in all PC directions;
• The error on humidity set to 0.75 in all PC directions;

This forms the baseline for the rest of the study.

Assimilating cloudy retrievals

This series of experiments aimed at increasing the amount of information conveyed to the model by assimilating cloudy pixels. Relying on the error estimates provided together with the products, no further adjustments to the observation error matrix was made here. The experiments gradually included observations with stronger and stronger cloud signal as indicated by the OmC indicator (OBS-CALC estimates retrieval in window channels).

Outcome:

• Including cloudy retrievals further improves the forecast’s performance;
• Assimilation including cloudy retrievals outperforms L1-assimilation for humidity
• The uncertainty estimates in cloudy pixels appears reliable and functional;
• Assimilating the cloudiest retrievals starts degrading the forecasts, especially temperature in the UTLS.

Increasing spatial density, with intelligent data thinning

In the nominal operational baseline, only 1 pixel in a 125 km x 125 km box is currently assimilated. Increasing the density of IASI pixels assimilated could not be experimented as part of this study.

Full system, assimilating maritime all-sky retrievals

After highly successful experimentation within the depleted observing system, similar experiments have been performed within the full observing system framework using the IFS cycle 48r1. In these full observing system experiments, the control experiment CTL includes all operationally used observations except for IASI radiances. In the L2 experiments, PCT and PClnq retrievals are added either in clear or in cloud-affected scenes. Additionally, for comparison, the RAD experiment is repeated within the full observing system, meaning that IASI radiances from Metop-C are actively assimilated over the sea.

As expected, the signal is weaker in the full observing system experiments compared to what was observed in the depleted observing system. The L2 assimilation has a negative impact on temperature when cloudy retrievals are ingested, while radiance assimilation remains neutral to positive. The effect of both L1 and L2 assimilation is comparably neutral to slightly positive for humidity forecasts, as evaluated against radiosondes.

Conclusions

• The assimilation of L2 profiles with legitimate mathematical representation (PC) and expression of scene-dependent observation operator (error-normalised averaging kernels) is a functional strategy in an operational NWP system;
• The assimilation of temperature and humidity profiles has a clear positive impact on both temperature and humidity forecasts;
• L2 positive impact is lower but close to L1 in the free troposphere and stratosphere for temperature and humidity;
• Cloud-free L2 impact is equivalent to L1 for temperature and superior to L1 for humidity in the lower troposphere;
• When assimilating cloudy retrievals, L2 outperforms L1 for humidity forecasts throughout the troposphere, but it also leads to small degradations in temperature in the UTLS
• Assimilation in a full observing system exhibits a weaker signal and requires further optimisation of the L2 production and assimilation set-up

Outlook

The assimilation of L2 has the potential to effectively convey the same or more information into an NWP model than through radiances, especially in (regional) systems where stringent trade-offs exist between timeliness requirements, computational costs and the need to manage increasing amount of satellite data.

These first series of experiments over ocean provided very valuable feed-back to L2 products developments and opened a number of potential avenues to explore and characterise further the practicalities and benefits of L2 assimilation: :

• Tuning the observation error;
• Increasing the density of IASI L2 profiles conveyed to the system;
• Pursue assimilation in a full assimilation environment;
• Evaluate the impact over land;
• Evaluate the most effective technical solutions to transmit profiles and scene-dependent observation operators to the system.