Contained Chaos - Ensemble Consistency Testing for the Community Earth System Model

Explore a 45-minute conference talk on ensemble consistency testing for the Community Earth System Model, presented at the Machine Learning for Climate KITP conference. Delve into the challenges of informing society about future climate changes at regional and local scales, and learn how big data and machine learning algorithms are advancing climate science. Discover the Ensemble Consistency Test (ECT) approach for evaluating differences in climate models, including its procedure, effectiveness, and statistical details. Examine counterexamples, challenges in highly accurate testing, and alternative estimators for different ensemble sizes. Gain insights into the intersection of climate modeling, software quality assurance, and statistical analysis in this comprehensive presentation by Dorit Hammerling from the National Center for Atmospheric Research.

Intro
The National Center for Atmospheric Research
Community Earth System Model (CESM)
Need for Software Quality Assurance
Motivation
Evaluating the difference
Our new approach: Ensemble Consistency Test
Ensemble Consistency Test (ECT)
Creation of and comparison with ensemble
Quantity ensemble variability
Hypothesis Testing based on Principal Components
ECT Procedure
How wel does CAM ECT work?
Do we really need year-long runs?
Counterexample 1: HYDRO-BASEFLOW
Counterexample 2: RAND
UF-CAM-ECT and CAM-ECT
A highly accurate test leads to new challenges...
First step: identify affected variables
Next: convert source code to directed graph
Looking at the statistical details of the ECT.
The ECT scheme viewed as a series of RVS
Illustration of estimation bias of eigenvalues
Estimator and ensemble size effect on false positive rate
Alternative estimators for different ensemble sizes