On May 27, representatives from the German Aerospace Agency (DLR) and the Leibniz Supercomputing Centre (LRZ) signed an agreement to partner on the Terra_Byte project. The multidisciplinary project focuses on interpreting data coming from sources ranging from satellite imaging data to social media.
“The cooperation of two leading research institutions brings together partners who complement each other ideally in their competencies and contribute their expertise, resources, and research areas,” said Prof. Dr. Dieter Kranzlmüller, Director at LRZ as well as the current Chairman of the Board for the Gauss Centre for Supercomputing (GCS). “LRZ has proven experience as an innovative IT service provider and a high performance and data analytics centre, and is a reliable partner for the Bavarian universities and, in the future, for DLR and its institutes in Oberpfaffenhofen.”
The project is focused on better recording and understanding global change in a broader sense. Researchers use imaging data from satellites to study natural disasters, for instance, but understanding large-scale ecological change requires information coming from the ground as well—social media posts, for instance, provide a huge potential for information about these events on a local level. In both cases, though, researchers must be able to efficiently sift through massive amounts of loosely connected data coming from myriad sources.
To date, the European-based Earth observation program Copernicus has generated more than 10 petabytes of data (more than 2 million DVDs worth of information), and researchers expect that by 2024, the program’s satellites will have collected roughly 40 petabytes of data. In order to manage such a massive amount of data, DLR researchers know that they need to employ high-performance data analytics (HPDA) methods to get meaningful information from the data deluge.
In the partnership DLR will be primarily focused on research and algorithm development, and LRZ will be focused on implementing large-scale, reliable IT services, optimizing data analysis and data management processes, and incorporating artificial-intelligence and big-data-related processes.
The partner organizations plan to analyze 40 petabytes of data across thousands of cloud-based computing cores, and will be implementing the first stage of expansion by the end of 2020.