A Benchmark Dataset for Radio Echo Sounding Data - Dreier, Marcel Nicolas; Gourmelon, Nora; Koch, Moritz; Seehaus, Thorsten; Braun, Matthias; Maier, Andreas; Christlein, Vincent (European Geosciences Union 2024, 2024)
Melting glaciers and ice sheets contribute significantly to the global sea level rise, posing an existential threat to coastal regions. For that reason, many expeditions collect radio echo sounding data to track, model, and predict the changes in ice mass and thickness. However, manually analyzing the data collected to determine the thickness of the ice sheet and mountain glaciers is a time-consuming task, as the collected data often extends over tens to hundreds of kilometers. Hence, the development and utilization of automated and semi-automated tools have risen in popularity to alleviate the problem. Especially machine learning-based approaches show promising results as they can capture complex relationships in the data. However, such methods require large labeled datasets to achieve their full potential. Thus, we are currently gathering and cleaning data that will be released as a benchmark dataset for radio echo sounding data to train, test, and compare different approaches. In detail, the dataset will consist of 2D radar data with georeferenced labels for the position of the air-ice and ice-bedrock interface. We plan to incorporate radar data from multiple sources since the underlying radar system and expedition area play a significant role in depicting the data and the general difficulty of the task. That way, the performance of a system can be fairly assessed with the benchmark dataset without the evaluation results being skewed by external factors. We will further ensure a fair comparison by dividing the data of every source in an independent training, validation, and test set. This division will also allow us to use the benchmark dataset to train and compare systems specialized on only a single data source. Thus, future work can also utilize our benchmark dataset to develop systems specialized on only a single data source. Lastly, we will release a baseline model alongside the benchmark dataset to demonstrate its practicality. Thereby giving an initial point of reference that future work can compare to. <

AMD-HookNet for Glacier Front Segmentation - Wu, Fei; Gourmelon, Nora; Seehaus, Thorsten; Zhang, Jianlin; Braun, Matthias; Maier, Andreas; Christlein, Vincent (IEEE Transactions on Geoscience and Remote Sensing, 2023)
Knowledge on changes in glacier calving front positions is important for assessing the status of glaciers. Remote sensing imagery provides the ideal database for monitoring calving front positions; however, it is not feasible to perform this task manually for all calving glaciers globally due to time constraints. Deep-learning-based methods have shown great potential for glacier calving front delineation from optical and radar satellite imagery. The calving front is represented as a single thin line between the ocean and the glacier, which makes the task vulnerable to inaccurate predictions. The limited availability of annotated glacier imagery leads to a lack of data diversity (not all possible combinations of different weather conditions, terminus shapes, sensors, etc. are present in the data), which exacerbates the difficulty of accurate segmentation. In this article, we propose attention-multihooking-deep-supervision HookNet (AMD-HookNet), a novel glacier calving front segmentation framework for synthetic aperture radar (SAR) images. The proposed method aims to enhance the feature representation capability through multiple information interactions between low-resolution and high-resolution inputs based on a two-branch U-Net. The attention mechanism, integrated into the two branch U-Net, aims to interact between the corresponding coarse and fine-grained feature maps. This allows the network to automatically adjust feature relationships, resulting in accurate pixel classification predictions. Extensive experiments and comparisons on the challenging glacier segmentation benchmark dataset CaFFe show that our AMD-HookNet achieves a mean distance error (MDE) of 438 m to the ground truth outperforming the current state of the art by 42%, which validates its effectivenes

CaFFe - a Benchmark Dataset for Glacier Calving Front Extraction from Synthetic Aperture Radar Imagery - Gourmelon, Nora; Seehaus, Thorsten; Klink, Julian; Braun, Matthias; Maier, Andreas; Christlein, Vincent (IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, 2023)

Calving fronts and where to find them: a benchmark dataset and methodology for automatic glacier calving front extraction from synthetic aperture radar imagery - Gourmelon, Nora; Seehaus, Thorsten; Braun, Matthias; Maier, Andreas; Christlein, Vincent (Earth System Science Data, 2022)
Exact information on the calving front positions of marine- or lake-terminating glaciers is a fundamental glacier variable for analyzing ongoing glacier change processes and assessing other variables like frontal ablation rates. In recent years, researchers started implementing algorithms that can automatically detect the calving fronts on satellite imagery. Most studies use optical images, as calving fronts are often easy to distinguish in these images due to the sufficient spatial resolution and the presence of different spectral bands, allowing the separation of ice features. However, detecting calving fronts on synthetic aperture radar (SAR) images is highly desirable, as SAR images can also be acquired during the polar night and are independent of weather conditions (e.g., cloud cover), facilitating year-round monitoring worldwide. In this paper, we present a benchmark dataset (Gourmelon et al., 2022b) of SAR images from multiple regions of the globe with corresponding manually defined labels providing information on the position of the calving front (https://doi.org/10.1594/PANGAEA.940950). With this dataset, different approaches for the detection of glacier calving fronts can be implemented, tested, and their performance fairly compared so that the most effective approach can be determined. The dataset consists of 681 samples, making it large enough to train deep learning segmentation models. It is the first dataset to provide long-term glacier calving front information from multi-mission data. As the dataset includes glaciers from Antarctica, Greenland, and Alaska, the wide applicability of models trained and tested on this dataset is ensured. The test set is independent of the training set so that the generalization capabilities of the models can be evaluated. We provide two sets of labels: one binary segmentation label to discern the calving front from the background, and one label for multi-class segmentation of different landscape classes. Unlike other calving front datasets, the presented dataset contains not only the labels but also the corresponding preprocessed and geo-referenced SAR images as PNG files. The ease of access to the dataset will allow scientists from other fields, such as data science, to contribute their expertise. With this benchmark dataset, we enable comparability between different front detection algorithms and improve the reproducibility of front detection studies. Moreover, we present one baseline model for each kind of label type. Both models are based on the U-Net, one of the most popular deep learning segmentation architectures. In the following two post-processing procedures, the segmentation results are converted into 1-pixel-wide front delineations. By providing both types of labels, both approaches can be used to address the problem. To assess the performance of different models, we suggest first reviewing the segmentation results using the recall, precision, F1 score, and the Jaccard index. Second, the front delineation can be evaluated by calculating the mean distance error to the labeled front. The presented vanilla models provide a baseline of 150 m ± 24 m mean distance error for the Mapple Glacier in Antarctica and 840 m ± 84 m for the Columbia Glacier in Alaska, which has a more complex calving front, consisting of multiple sections, compared with a laterally well constrained, single calving front of Mapple Glacie
Keywords: Deep Learning; Calving Front; Dataset; Computer Vision

Comparability of Deep Learning Techniques for Calving Front Segmentation in SAR Imagery - Gourmelon, Nora; Seehaus, Thorsten; Braun, Matthias; Maier, Andreas; Christlein, Vincent (European Geoscience Union General Assembly 2023, 2023)

Conditional Random Fields for Improving Deep Learning-based Glacier Calving Front Delineations - Gourmelon, Nora; Klink, Julian; Seehaus, Thorsten; Braun, Matthias; Maier, Andreas; Christlein, Vincent (IGARSS 2023 - 2023 IEEE International Geoscience and Remote Sensing Symposium, 2023)

Contextual HookFormer for Glacier Calving Front Segmentation - Wu, Fei; Gourmelon, Nora; Seehaus, Thorsten; Zhang, Jianlin; Braun, Matthias; Maier, Andreas; Christlein, Vincent (IEEE Transactions on Geoscience and Remote Sensing, 2024)
Position changes in glacier calving fronts are important indicators for evaluating the health of ice sheet outlet glaciers and changes in ice dynamics. However, manual delineation of calving fronts in remote sensing imagery is a time-consuming task, resulting in potential large costs. Deep-learning-based methods have made remarkable progress in automatically segmenting and delineating glacier calving fronts from remote sensing imagery. The relatively few remote sensing images and the limited geometric changes for glacier observations both reduce the diversity of the data and exacerbate the difficulty in accurate segmentation. Here we describe a novel automatic method for detecting glacier calving fronts in synthetic aperture radar (SAR) images, termed HookFormer. Our approach processes high-resolution (target) and low-resolution (context) inputs with a unified transformer architecture. The global–local tokens from the context and the target branches are integrated purely by the proposed cross-attention mechanism and cross-interaction module to complement and enhance each other. Moreover, we redesign the HookFormer architecture based on the convolutional neural network (CNN) model AMD-HookNet aiming to improve computational efficiency while achieving significant performance gains with only half of the model parameters/FLOPs. We conduct an in-depth analysis and make extensive comparisons based on the challenging glacier segmentation benchmark dataset CaFFe. As the first pure transformer approach, HookFormer sets a new state of the art with a mean distance error (MDE) of 353m to the ground truth, outperforming the baseline, Swin-Unet, and AMD-HookNet by 53%, 39%, and 19%, respectively.<br

Dissecting Glaciers - Can an Automated Bio-Medical Image Segmentation Tool also Segment Glaciers? - Gourmelon, Nora; Seehaus, Thorsten; Braun, Matthias; Maier, Andreas; Christlein, Vincent (EGU General Assembly 2022, 2022)

Implications of Experiment Set-Ups for Residential Water End-Use Classification - Gourmelon, Nora; Bayer, Siming; Mayle, Michael; Bach, Guy; Bebber, Christian; Munck, Christophe; Sosna, Christoph; Maier, Andreas (5th International Electronic Conference on Water Sciences, 2020)
With the increased population in urban areas worldwide, the security of water supply is gaining in importance. Water scarcity accelerated by climate change poses additional stress to water supply infrastructures. Water consumption data transmitted by smart water meters form the foundation of advanced data analysis, such as water end-use classification, with which the resilience of water supply can be improved. Especially with large amounts of high-resolution data, the accurate categorization of data from smart water meters into different end-uses such as toilets, showers or dishwashers is challenging and cannot be performed by humans. To this end, machine-learning (ML) approaches provide several benefits, such as real-time capability, scalability and generalizability. State-of-the-art methods to identify residential water end-uses include both unsupervised methods and supervised approaches. However, a comprehensive comparison of unsupervised and supervised techniques is still missing. In this study, we are aiming at a quantitative evaluation of various ML techniques for water end-use classification. Furthermore, we focus on deriving general implications on the setting and conduction of ML-based experiments for water end-use classification. For these purposes, a stochastic water consumption simulation tool with high capability to model the real-world water consumption pattern is applied to generate residential data. Subsequently, unsupervised clustering methods, such as dynamic time warping, k-means, DBSCAN, OPTICS and Hough transform, are compared to supervised methods based on SVM. The quantitative results demonstrate that supervised approaches are capable to classify common residential end-uses (toilet, shower, faucet, dishwasher, washing machine, bathtub and mixed water-uses) with accuracies up to 0.99, whereas unsupervised methods fail to detect those consumption categories. The major implications drawn from the quantitative results are two-fold: clustering is not suitable to separate end-use categories. Hence, accurate labels are essential for the end-use classification of water events, where crowdsourcing and citizen science approaches pose feasible solutions for this purpose
Keywords: end-use classification; smart water meter; machine learning

Implications of Experiment Set-Ups for Residential Water End-Use Classification - Gourmelon, Nora; Bayer, Siming; Mayle, Michael; Bach, Guy; Bebber, Christian; Munck, Christophe; Sosna, Christoph; Maier, Andreas (Water, 2021)
With an increasing need for secured water supply, a better understanding of the water consumption behavior is beneficial. This can be achieved through end-use classification, i.e., identifying end-uses such as toilets, showers or dishwashers from water consumption data. Previously, both supervised and unsupervised machine learning (ML) techniques are employed, demonstrating accurate classification results on particular datasets. However, a comprehensive comparison of ML techniques on a common dataset is still missing. Hence, in this study, we are aiming at a quantitative evaluation of various ML techniques on a common dataset. For this purpose, a stochastic water consumption simulation tool with high capability to model the real-world water consumption pattern is applied to generate residential data. Subsequently, unsupervised clustering methods, such as dynamic time warping, k-means, DBSCAN, OPTICS and Hough transform, are compared to supervised methods based on SVM. The quantitative results demonstrate that supervised approaches are capable to classify common residential end-uses (toilet, shower, faucet, dishwasher, washing machine, bathtub and mixed water-uses) with accuracies up to 0.99, whereas unsupervised methods fail to detect those consumption categories. In conclusion, clustering techniques alone are not suitable to separate end-use categories fully automatically. Hence, accurate labels are essential for the end-use classification of water events, where crowdsourcing and citizen science approaches pose feasible solutions for this purpos
Keywords: end-use classification; smart water meter; machine learning; residential water

Out-of-the-box Calving Front Detection Method Using Deep Learning - Herrmann, Oskar; Gourmelon, Nora; Seehaus, Thorsten; Maier, Andreas; Fürst, Johannes; Braun, Matthias; Christlein, Vincent (Cryosphere, 2023)

PASAL: Progress- and sparsity-aware loss balancing for heterogeneous dataset fusion - Gorges, Thomas; Scholz, Teresa; Saloman, Stefan; Zinnen, Mathias; Hoffmann, Juliane; Gourmelon, Nora; Maier, Andreas; Hettenkofer, Sebastian; Christlein, Vincent (Information Fusion, 2025)
Machine learning has seen widespread application in many areas. Despite theoretical advancements, the demand for qualitative and extensive data foundations is increasing. Real-world datasets are often small and combining them is challenging due to the resulting sparsity and heterogeneity. Existing combination techniques merge datasets into a common space before training, causing drawbacks such as data loss and distortion of annotations. To address this, we fuse heterogeneous datasets by jointly training dataset-specific weighted sub-networks. Balancing losses from heterogeneous data sources is challenging, as current techniques are inadequate. We propose a novel progress- and sparsity-aware loss balancing method (PASAL), which adaptively balances sub-network losses based on individual learning progress and sparsity. As an example, we present the application of PASAL to the olfaction domain, where predicting smell properties based on molecular structure is difficult due to subjective impressions, typically limited data, and a lack of unified datasets. By evaluating PASAL on the DREAM Olfaction Prediction Challenge, we improve the current state-of-the-art method from a Z-Score of 9.92 to 10.10. Furthermore, by treating our AI as an annotator, we surpass human performance in the odor and pleasantness categories with statistical significance. Our findings are supported by a feature analysis, indicating that our heterogeneous combination methodology enhances odor predictio
Keywords: Dataset fusion; Heterogeneous data; Deep learning; Loss balancing; Scent prediction; Olfaction

Tapping the Potential of Earth Observation - Calving Front Detection in SAR Images using Deep Learning Techniques - Gourmelon, Nora; Seehaus, Thorsten; Davari, Amirabbas; Braun, Matthias; Maier, Andreas; Christlein, Vincent (EGU General Assembly 2021, 2021)
The calving fronts of lake or marine terminating glaciers provide information about the state of glaciers. A change in its position can affect the flow of the entire glacier system, and the loss of ice mass as icebergs calve-off and discharge into the ocean has a multi-scale impact on the global hydrological cycle. The calving fronts can be manually delineated in Synthetic Aperture Radar (SAR) images. However, this is a time-consuming, tedious and expensive task. As deep learning approaches have achieved tremendous success in various disciplines, such as medical image processing and computer vision, the project Tapping the Potential of Earth Observation (TAPE) is amongst other things dedicated to applying deep learning techniques to calving front detection. So far, all our experiments have employed U-Net based architectures, as the U-Net is state-of-the-art in semantic image segmentation. A major challenge of front detection is the class imbalance: The front has significantly fewer pixels than the remaining parts of the SAR image. Hence, we developed variants of the U-Net specifically addressing this challenge including an Attention U-Net, a probabilistic Bayesian U-Net, as well as a U-Net with a distance map-based binary cross-entropy (BCE) loss function and a Mathews correlation coefficient (MCC) as early stopping criterion. In future work, we plan to investigate multi-task learning and a segmentation of the SAR image into different classes (i.e. ocean, glacier and rocks) to enhance the quality and efficiency of the front detection.<br