Getting Started

Motivations

A major advantage of our era is the easiness of gathering and recording data. Data can be found literary everywhere, from quantum scale to stars and galaxies. Great challenges are arising from this fact and the biggest of them all is how you can efficient manage all this information.

In particular for space data, technology has advanced so rapidly that every day, petabytes of useful information is recorded through satellites, other sensors around the globe, and simulations. However, a big part of this information is remain unused (or partly used) because the proper tools have not yet been created. New software packages can lead the way to win that challenge and fully (or partly) automate various processes that are necessary in order to transform them in a format that the scientists can understand and analyze.

This is the reason why we developed aidapy which is a high level Python package for the analysis of spacecraft data from heliospheric missions using modern techniques including data assimilation, machine learning (ML), artificial intelligence (AI), and advanced statistical models. The transition from proprietary software based on commercial languages to the new open-source community-developed packages in Python represent one of the main objective of aidapy. Our vision is to combine software tools that are already existing out there for different reasons, in a nice and efficient way. In our minds, this is a crucial and necessary task for the reducing of the knowledge “gap” that exist in several occasions between different scientific fields. aidapy has been created in order to fulfil the needs of a user with high programming skills as well as, for a user with basic programming knowledge.

Finally, we aim to build a new specific database for helping the ongoing trend in database standardization in space. Key heliophysics problems are selected to produce a database (AIDAdb) of new high-level data products that include catalogs of features and events detected by ML and AI algorithms from existing numerical simulations and observations. New simulations are also performed to enrich this database. Moreover, many of the AI methods developed in AIDA represent themselves higher-level data products, for instance in the form of trained models.

The transition to a big-data phase and the use of the big-data language of choice Python prepares the space community to the use of ML and AI. The latest AI developments and especially the methodology of deep neural networks are particularly suited for the identification of physical processes in space images and data. However, there is a barrier to overcome between the two very distinct communities of space scientists and experts in AI. Aidapy aims at bridging this gap linking the software developed in the AI community with heliospheric data.

We are developing aidapy to make it easier for a researcher in space science who does not have a background in computer science or in artificial intelligence to use some of the most advanced tools in this field and provide some concrete examples on how AI can help analyse data from simulations and observations, how AI can help discover physical processes hidden in the data and how it can make space weather forecasts.

~