Welcome to the Assessment of Understanding and Quantifying Progress Group working page

Or, as we've been *I hope* fondly nick named, the super topic!

This is your one stop shop to keep up with and communicate with our group. We hope that this will also become a meeting place to facilitate communication between ourselves and the other working topics. If you have questions, comments, or just want to say a friendly "hello", you should be able to find a comment box on each page.

Goals of the group:

  1. (Science) Develop a framework for quantifying the scientific progress in the field of Space Weather.
  2. (Applied Sciences) Develop a framework for generating and tracking ARLs.
  3. (Science and Applied Sciences) Develop a framework for application of metrics to measure performance/progress and construct a library of metric and validation techniques.

International CCMC - LWS Workshop - Assessment of Space Weather Development: Understanding, Operational Readiness, Forecasting Skills.


April 3-7, 2017

At this workshop we will evaluate existing metrics and develop new metrics for measuring advancements, and current capabilities within the field of space weather focusing on three broad topics. Discussions, panels, and working groups will focus on metrics for:

  1. Progress in our research understanding.
  2. The functionality of current models, both physics based and data driven. (including new physics e.g.)
  3. Ability to include current research/physical understanding into models.

With each of these areas comes distinct challenges for developing metrics which we will explore during the workshop. Topic 1 poses unique challenges for quantifying progress. Metrics such as number of papers or citation counts fail to accurately capture improvements in our understanding of the field. Metrics that accurately capture our research progress will allow us as a community to direct our research in an unbiased way towards addressing unanswered questions. This topic will consider questions such as “How critical are we?”, “Are we addressing end user concerns?”, and perhaps most importantly “How are we assessing our progress?”. Other fields have encountered similar problems, and we will consider how they have tracked progress as well as how our own field has attempted to show a continue advancement of understanding. Focus topics 2 and 3 will address questions such as “How should we compare our models against theory and observations?”, “Are our current metrics unbiased, and how can we develop ways to improve them?”, and “Are we consciously or unconsciously ignoring specific issues or topics?”. For topic 2 we will discuss and make progress towards understanding which current metrics are appropriate for the modeler and end user, as well as to identify new metrics that are required to understand model performance and appropriateness for a given problem. For topic 3 we will quantify our ability to model parts of the heliosphere with physics based models, including the time it takes to test and implement new ideas/models/observations into our larger models. We hope to find metrics that will quantify the advancements achieved by including new physics into models, as well as determining which physics is most important for different regions of the heliosphere and under what conditions.