2024-03-27T12:55:20+00:00http://mogren.oneOlof MogrenOlof MogrenPublication: From Weak to Strong Sound Event Labels using Adaptive Change-Point Detection and Active Learning2024-03-13T00:00:00+00:00http://mogren.one/publications/2024/from/In this work we propose an audio recording segmentation method based on an adaptive change point detection (A-CPD) for machine guided weak label annotation of audio recording segments. The goal is to maximize the amount of information gained about the temporal activation's of the target sounds. For each unlabeled audio recording, we use a prediction model to derive a probability curve used to guide annotation. The prediction model is initially pre-trained on available annotated sound event data with classes that are disjoint from the classes in the unlabeled dataset. The prediction model then gradually adapts to the annotations provided by the annotator in an active learning loop. The queries used to guide the weak label annotator towards strong labels are derived using change point detection on these probabilities. We show that it is possible to derive strong labels of high quality even with a limited annotation budget, and show favorable results for A-CPD when compared to two baseline query strategies.Publication: Impacts of Color and Texture Distortions on Earth Observation Data in Deep Learning2024-03-07T00:00:00+00:00http://mogren.one/publications/2024/impacts/Land cover classification and change detection are two important applications of remote sensing and Earth observation (EO) that have benefited greatly from the advances of deep learning. Convolutional and transformer-based U-net models are the state-of-the-art architectures for these tasks, and their performances have been boosted by an increased availability of large-scale annotated EO datasets. However, the influence of different visual characteristics of the input EO data on a model's predictions is not well understood. In this work we systematically examine model sensitivities with respect to several color- and texture-based distortions on the input EO data during inference, given models that have been trained without such distortions. We conduct experiments with multiple state-of-the-art segmentation networks for land cover classification and show that they are in general more sensitive to texture than to color distortions. Beyond revealing intriguing characteristics of widely used land cover classification models, our results can also be used to guide the development of more robust models within the EO domain.Publication: Efficient Node Selection in Private Personalized Decentralized Learning2024-01-09T00:00:00+00:00http://mogren.one/publications/2024/efficient/In this paper, we propose a novel approach for privacy-preserving node selection in personalized decentralized learning, which we refer to as Private Personalized Decentralized Learning (PPDL). Our method mitigates the risk of inference attacks through the use of secure aggregation while simultaneously enabling efficient identification of collaborators. This is achieved by leveraging adversarial multi-armed bandit optimization that exploits dependencies between the different arms. Through comprehensive experimentation on various benchmarks under label and covariate shift, we demonstrate that our privacy-preserving approach outperforms previous non-private methods in terms of model performance.Publication: Concept-aware clustering for decentralized deep learning under temporal shift2023-06-22T00:00:00+00:00http://mogren.one/publications/2023/concept-aware/Decentralized deep learning requires dealing with non-iid data across clients, which may also change over time due to temporal shifts. While non-iid data has been extensively studied in distributed settings, temporal shifts have received no attention. To the best of our knowledge, we are first with tackling the novel and challenging problem of decentralized learning with non-iid and dynamic data. We propose a novel algorithm that can automatically discover and adapt to the evolving concepts in the network, without any prior knowledge or estimation of the number of concepts. We evaluate our algorithm on standard benchmark datasets and demonstrate that it outperforms previous methods for decentralized learning.Talk: Tutorial at EUREF 2023: AI for the environment2023-05-23T00:00:00+00:00http://mogren.one/talks/2023/euref/Publication: Fully Convolutional Networks for Dense Water Flow Intensity Prediction in Swedish Catchment Areas2023-04-05T00:00:00+00:00http://mogren.one/publications/2023/dense-water-flow-intensity-prediction/ Intensifying climate change will lead to more extreme weather events, including heavy rainfall and drought. Accurate stream flow prediction models which are adaptable and robust to new circumstances in a changing climate will be an important source of information for decisions on climate adaptation efforts, especially regarding mitigation of the risks of and damages associated with flooding. In this work we propose a machine learning-based approach for predicting water flow intensities in inland watercourses based on the physical characteristics of the catchment areas, obtained from geospatial data (including elevation and soil maps, as well as satellite imagery), in addition to temporal information about past rainfall quantities and temperature variations. We target the one-day-ahead regime, where a fully convolutional neural network model receives spatio-temporal inputs and predicts the water flow intensity in every coordinate of the spatial input for the subsequent day. To the best of our knowledge, we are the first to tackle the task of dense water flow intensity prediction; earlier works have considered predicting flow intensities at a sparse set of locations at a time. An extensive set of model evaluations and ablations are performed, which empirically justify our various design choices. Code and preprocessed data have been made publicly available at <a href="https://github.com/aleksispi/fcn-water-flow">this https URL</a>. Publication: Federated learning for performance prediction in multi-operator environments2023-02-15T00:00:00+00:00http://mogren.one/publications/2023/multi-operator/Telecom vendors and operators deliver services with strict requirements on performance, over complex and sometimes partly shared network infrastructures. A key enabler for network and service management in such environments is knowledge sharing, and the use of data-driven models for performance prediction, forecasting, and troubleshooting. In this paper, we outline a multi-operator service metrics prediction framework using federated learning that allows privacy-preserved knowledge-sharing across operators for improved model performance, and also reduced requirements on data transfer within an operator network. Federated learning is compared against local and central learning strategies for multi-operator performance prediction, and it is shown to balance the requirements on data privacy, model performance, and the network overhead. Further, the paper provides insights on how data heterogeneity affects model performance, where the conclusion is that standard federated learning has certain robustness to data heterogeneity. Finally, we discuss the challenges related to training a federated learning model with a limited budget on the communication rounds. The evaluation is performed using a set of realistic publicly available data traces, that are adapted specifically for the purpose of studying multi-operator service performance prediction.Publication: Private Node Selection in Personalized Decentralized Learning2023-01-30T00:00:00+00:00http://mogren.one/publications/2023/private/In this paper, we propose a novel approach for privacy-preserving node selection in personalized decentralized learning, which we refer to as Private Personalized Decentralized Learning (PPDL). Our method mitigates the risk of inference attacks through the use of secure aggregation while simultaneously enabling efficient identification of collaborators. This is achieved by leveraging adversarial multi-armed bandit optimization that exploits dependencies between the different arms. Through comprehensive experimentation on various benchmarks under label and covariate shift, we demonstrate that our privacy-preserving approach outperforms previous non-private methods in terms of model performance.Publication: EFFGAN: Ensembles of fine-tuned federated GANs2022-12-17T00:00:00+00:00http://mogren.one/publications/2022/effgan/ Generative adversarial networks have proven to be a powerful tool for learning complex and high-dimensional data distributions, but issues such as mode collapse have been shown to make it difficult to train them. This is an even harder problem when the data is decentralized over several clients in a federated learning setup, as problems such as client drift and non-iid data make it hard for federated averaging to converge. In this work, we study the task of how to learn a data distribution when training data is heterogeneously decentralized over clients and cannot be shared. Our goal is to sample from this distribution centrally, while the data never leaves the clients. We show using standard benchmark image datasets that existing approaches fail in this setting, experiencing so-called client drift when the local number of epochs becomes to large. We thus propose a novel approach we call EFFGAN: Ensembles of fine-tuned federated GANs. Being an ensemble of local expert generators, EFFGAN is able to learn the data distribution over all clients and mitigate client drift. It is able to train with a large number of local epochs, making it more communication efficient than previous works. Publication: Few-shot bioacoustic event detection using a prototypical network ensemble with adaptive embedding functions2022-11-03T00:00:00+00:00http://mogren.one/publications/2022/fewshot/ In this report we present our method for the DCASE 2022 challenge on few-shot bioacoustic event detection. We use an ensemble of prototypical neural networks with adaptive embedding functions and show that both ensemble and adaptive embedding functions can be used to improve results from an average F-score of 41.3% to an average F-score of 60.0% on the validation dataset. Publication: Financing Solutions for Circular Business Models: Exploring the Role of Business Ecosystems and Artificial Intelligence2022-10-24T00:00:00+00:00http://mogren.one/publications/2022/financing/ Circular economy promotes a transition away from linear modes of production and consumption to systems with circular material flows that can significantly improve resource productivity. However, transforming linear business models to circular business models posits a number of financial consequences for product companies as they need to secure more capital in a stock of products that will be rented out over time and therefore will encounter a slower, more volatile cash flow in the short term compared to linear direct sales of products. This paper discusses the role of financial actors in circular business ecosystems and alternative financing solutions when moving from product-dominant business models to product-as-a-service or function-based business models and demonstrates a solution where state-of-the-art AI modelling can be incorporated for financial risk assessment. We provide an open implementation and a thorough empirical evaluation of an AI-model which learns to predict residual value of stocks of used items. Furthermore, the paper highlights solutions, managerial implications and potentials for financing circular business models, argues the importance of different forms of data in future business ecosystems, and puts forward recommendations for how AI can help overcoming some of the challenges ahead. Publication: A Novel Method for Smart Fire Detection Using Acoustic Measurements and Machine Learning: Proof of Concept2022-09-09T00:00:00+00:00http://mogren.one/publications/2022/smartfiredetection/ Fires are a major hazard resulting in high monetary costs, personal suffering, and irreplaceable losses. The consequences of a fire can be mitigated by early detection systems which increase the potential for successful intervention. The number of false alarms in current systems can for some applications be very high, but could be reduced by increasing the reliability of the detection system by using complementary signals from multiple sensors. The current study investigates the novel use of machine learning for fire event detection based on acoustic sensor measurements. Many materials exposed to heat give rise to acoustic emissions during heating, pyrolysis and burning phases. Further, sound is generated by the heat flow associated with the flame itself. The acoustic data collected in this study is used to define an acoustic sound event detection task, and the proposed machine learning method is trained to detect the presence of a fire event based on the emitted acoustic signal. The method is able to detect the presence of fire events from the examined material types with an overall F-score of 98.4%. The method has been developed using laboratory scale tests as a proof of concept and needs further development using realistic scenarios in the future. <br /><br /> <strong>Editor's choice: Best paper award in The Journal of Fire Technology 2022.</strong> <a href="https://www.springer.com/journal/10694/updates/19267070">More info.</a> Publication: Decentralized adaptive clustering of deep nets is beneficial for client collaboration2022-07-23T00:00:00+00:00http://mogren.one/publications/2022/decentralized/ We study the problem of training personalized deep learning models in a decentralized peer-to-peer setting, focusing on the setting where data distributions differ between the clients and where different clients have different local learning tasks. We study both covariate and label shift, and our contribution is an algorithm which for each client finds beneficial collaborations based on a similarity estimate for the local task. Our method does not rely on hyperparameters which are hard to estimate, such as the number of client clusters, but rather continuously adapts to the network topology using soft cluster assignment based on a novel adaptive gossip algorithm. We test the proposed method in various settings where data is not independent and identically distributed among the clients. The experimental evaluation shows that the proposed method performs better than previous state-of-the-art algorithms for this problem setting, and handles situations well where previous methods fail. Publication: Adversarial representation learning for synthetic replacement of private attributes2021-12-15T00:00:00+00:00http://mogren.one/publications/2021/adversarial/ Data privacy is an increasingly important aspect of the analysis of big data for many real-world tasks. Privacy enhancing transformations of data can help unlocking the potential in data sources containing sensitive information, but finding the right balance between privacy and utility is often a tricky trade-off. In this work, we study how adversarial representation learning can be used to ensure the privacy of users, and to obfuscate sensitive attributes in existing datasets. While previous methods using this kind of approach only aim at obfuscating the sensitive information, we find that adding new information in its place strengthens the provided privacy. We propose a two step data privatization method that builds on generative adversarial networks: in the first step, sensitive data is removed from the representation, and in the second step, a sample which is independent of the input data is inserted in its place. The result is an approach that can provide stronger privatization on image data, and yet be preserving both the domain and the utility of the inputs. Talk: AI for environment at RISE2021-12-09T00:00:00+00:00http://mogren.one/talks/2021/ai-for-climate/Talk: AI for chemistry and process industry2021-09-15T00:00:00+00:00http://mogren.one/talks/2021/royal-swedish-academy-of-engineering-sciences/Publication: Decentralized federated learning of deep neural networks on non-iid data2021-07-24T00:00:00+00:00http://mogren.one/publications/2021/decentralized/ We tackle the non-convex problem of learning a personalized deep learning model in a decentralized setting. More specifically, we study decentralized federated learning, a peer-to-peer setting where data is distributed among many clients and where there is no central server to orchestrate the training. In real world scenarios, the data distributions are often heterogeneous between clients. Therefore, in this work we study the problem of how to efficiently learn a model in a peer-to-peer system with non-iid client data. We propose a method named Performance-Based Neighbor Selection (PENS) where clients with similar data distributions detect each other and cooperate by evaluating their training losses on each other’s data to learn a model suitable for the local data distribution. Our experiments on benchmark datasets show that our proposed method is able to achieve higher accuracies as compared to strong baselines. Master thesis: Graph neural networks for physics simulations2021-06-30T00:00:00+00:00http://mogren.one/group/2021/lam/Graph neural networks for physics simulationsPublication: Scaling Federated Learning for Fine-Tuning of Large Language Models2021-06-23T00:00:00+00:00http://mogren.one/publications/2021/scaling/ Federated learning (FL) is a promising approach to distributed compute, as well as distributed data, and provides a level of privacy and compliance to legal frameworks. This makes FL attractive for both consumer and healthcare applications. While the area is actively being explored, few studies have examined FL in the context of larger language models and there is a lack of comprehensive reviews of robustness across tasks, architectures, numbers of clients, and other relevant factors. In this paper, we explore the fine-tuning of Transformer-based language models in a federated learning setting. We evaluate three popular BERT-variants of different sizes (BERT, ALBERT, and DistilBERT) on a number of text classification tasks such as sentiment analysis and author identification. We perform an extensive sweep over the number of clients, ranging up to 32, to evaluate the impact of distributed compute on task performance in the federated averaging setting. While our findings suggest that the large sizes of the evaluated models are not generally prohibitive to federated training, we found that the different models handle federated averaging to a varying degree. Most notably, DistilBERT converges significantly slower with larger numbers of clients, and under some circumstances, even collapses to chance level performance. Investigating this issue presents an interesting perspective for future research.Talk: Learned representations and what they encode2021-01-20T00:00:00+00:00http://mogren.one/talks/2021/learned-representations/ Learned continuous embeddings for language units was some of the first trembling steps of making neural networks useful for natural language processing (NLP), and promised a future with semantically rich representations for downstream solutions. NLP has now seen some of the progress that previously happened in image processing: the availability of increased computing power and the development of algorithms have allowed people to train larger models that perform better than ever. Such models also make it possible to use transfer learning for language tasks, thus leveraging large widely available datasets. </p><p> In 2016, Bolukbasi, et.al., presented their paper "Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings", shedding lights on some of the gender bias that was available in trained word embeddings at the time. Datasets obviously encode the social bias that surrounds us, and models trained on that data may expose the bias in their decisions. Similarly, learned representations may encode sensitive details about individuals in the datasets; allowing the disclosure of such information through distributed models or their outputs. All of these aspects are crucial in many application areas, not the least in the processing of medical texts. </p><p> Some solutions have been proposed to limit the expression of social bias in NLP systems. These include techniques such as data augmentation, representation calibration, and adversarial learning. Similar approaches may also be relevant for privacy and disentangled representations. In this talk, we'll discuss some of these issues, and go through some of the solutions that have been proposed recently to limit bias and to enhance privacy in various settings.